An airborne real-time meteorological data acquisition method and device

Abstract

This invention provides a method and apparatus for airborne real-time meteorological data acquisition, relating to the field of meteorological observation technology. The method includes: recording aircraft instrument panel video using a camera; converting the video into an image sequence; classifying the images into alphanumeric and color categories using a convolutional neural network; cropping and recognizing alphanumeric images to extract meteorological parameters; for color radar echo images, extracting echo intensity and mapping geographic coordinates by identifying color markers, establishing a color-intensity mapping, dividing the data into grids, and fusing aircraft position and attitude information; and finally storing the multi-source data in spatiotemporal pairs. This invention overcomes the data isolation limitations between the original aircraft system and meteorological equipment, achieving automated extraction of high-precision data such as airborne radar echoes and aircraft position and attitude, providing reliable meteorological observation data for weather modification operations and atmospheric science research, and has practical application value.

Description

Technical Field

[0001] This invention relates to the field of meteorological observation technology, and in particular to an airborne real-time meteorological data acquisition method and device. Background Technology

[0002] As a meteorological observation and operation platform, weather modification aircraft are typically equipped with two independent equipment systems: one is the aircraft's original aviation equipment system, including airborne radar, navigation equipment, flight attitude monitoring system, etc., mainly for the captain's use, to ensure flight safety; the other is a set of meteorological observation equipment specifically configured for weather modification operations, such as cloud droplet spectrometers, liquid water content measuring instruments, etc., mainly for the onboard meteorological professionals to analyze the physical characteristics of clouds and determine the timing of catalytic operations.

[0003] However, there is a technical bottleneck of data isolation between these two systems. First, from a flight safety perspective, civil aviation regulations strictly limit any form of connection or modification to the original aircraft systems to ensure the independence and reliability of flight safety systems. Second, any modification to aircraft equipment requires strict approval from various civil aviation departments, which is a complex, time-consuming, and costly process. More importantly, many important meteorological parameters of the original aircraft equipment (such as high-resolution echo data acquired by airborne radar, precise aircraft position and attitude information, etc.) are only displayed on the cockpit instrument panel and do not have data storage capabilities.

[0004] Under current technological conditions, meteorological professionals cannot acquire and record high-precision meteorological and location data provided by the aircraft's original systems in real time. This data is of vital value for analyzing meteorological conditions, making decisions about catalyst timing, and conducting subsequent scientific research. Especially during airborne detection operations, this data is often the only available high-precision in-situ observation data. Summary of the Invention

[0005] This invention provides an airborne real-time meteorological data acquisition method and device. By constructing a two-level collaborative architecture, a high-definition camera is installed at an appropriate location in the cockpit to record various meteorological parameters and flight data displayed on the captain's instrument panel throughout the entire flight. Then, computer vision and deep learning technologies are used to automatically identify and extract numerical information from the video images, and this information is correlated with time and space information to form a dataset with meteorological significance.

[0006] This invention's method requires no physical connection or modification to the aircraft's original systems, completely circumventing aviation safety and approval procedures, and achieving effective acquisition of high-value meteorological data at extremely low cost. This data not only serves as an important supplement to existing weather modification operations data but also provides unique data support for atmospheric science research, numerical model validation, weather forecasting, and meteorological equipment evaluation.

[0007] In a first aspect, the present invention provides an airborne real-time meteorological data acquisition method, comprising: When artificial weather modification aircraft are operating, at least one high-definition camera is installed to simultaneously record the meteorological parameters and flight parameters displayed on the aircraft's instrument panel. The recorded video is processed by software and extracted into a series of images according to a set frame rate; Using a pre-trained image classification neural network, the extracted images are classified into alphanumeric images and color images; For the classified alphanumeric images, preliminary cropping is performed according to the area of ​​their displayed physical meaning to obtain sub-images containing independent meteorological element values, and the sub-images are further cropped into unit images containing only a single number or letter; Using a pre-trained digit and letter recognition neural network, specific numbers or letters in the unit image are identified, and then combined to reconstruct the values ​​of various meteorological elements; For the classified color images, a sub-step of radar echo intensity data extraction and geographic coordinate mapping is performed. This sub-step includes: identifying color mark regions in the radar display image and establishing a mapping relationship between the RGB values ​​of the color mark colors and the range of radar echo intensity values; dividing the radar display image into a grid, applying the mapping relationship to the dominant color within each grid cell to obtain the radar echo intensity value corresponding to that grid cell; establishing a correspondence between radar image grid cells and actual geographic coordinates; and training a neural network model for identifying radar echo intensity based on the extracted dataset, and then applying the optimal model in a practical application. The identified alphanumeric meteorological data is paired and deduplicated with radar echo intensity and geographic coordinate data extracted from color images according to time and space to form a structured dataset and store it.

[0008] Furthermore, during weather modification aircraft operations, at least one high-definition camera is installed to simultaneously record video of the meteorological and flight parameters displayed on the aircraft's instrument panel, including: A high-definition camera was set up, facing the original instrument panel of the aircraft, to record video containing multiple meteorological parameters and radar echo information. A second high-definition camera was set up to record video information from the meteorological detection equipment displayed by the weather modification workers. Synchronize the time display of the two cameras with the network time.

[0009] Furthermore, when the recorded video is processed by software and extracted into a series of images according to a set frame rate, video processing software is used to convert the video into batch images, and redundant and invalid images are filtered out according to a set time interval.

[0010] Furthermore, using a pre-trained image classification neural network, the extracted images are classified into alphanumeric images and color images, including: Create training labels, labeling images containing numbers and letters as the first category and colored images as the second category; After binarization preprocessing, the images are divided into training and test sets according to a certain ratio; Multiple convolutional neural network parameter models were evaluated using a test set, and the model with the highest overall score was selected for actual classification based on accuracy and recall metrics.

[0011] Furthermore, color mark regions in the radar display image are identified, and a mapping relationship is established between the RGB values ​​of the color mark and the range of radar echo intensity values, including: To identify color mark regions on an airborne radar display, the following processing steps are performed: Gaussian filtering is applied to video frames for noise reduction, and adaptive histogram equalization is used to enhance contrast; the Canny edge detection algorithm is applied to extract edges, with the high and low thresholds of the Canny edge detection set to 70% and 30% of the image gradient magnitude, respectively, and morphological closing operations are used to connect the edges to form contours; contours are selected based on the following conditions: rectangularity ≥ 0.85; aspect ratio conforms to the expected color mark shape, with vertical color marks being 3:1 to 10:1 and horizontal color marks being 1:3 to 1:10; area occupying 2% to 15% of the entire display area; the region that best meets the above characteristics is selected as the color mark region. Establish a discrete mapping table from RGB color values ​​to echo intensity ranges; For the pixel color within the radar display area, the most similar standard color and its corresponding echo intensity range are determined by calculating its RGB Euclidean distance to the standard color in the mapping table, and the median value of this range is used as the echo intensity value of the pixel.

[0012] Furthermore, the correspondence between radar image grid cells and actual geographic coordinates is established, including: Obtain the aircraft's current latitude and longitude, heading angle, and pitch angle; Calculate the geographic coordinates of each radar image grid cell (i, j), where the radar screen is divided into an m×n grid, and the display range is R kilometers in front of the aircraft. The specific calculation formula is as follows: Calculate normalized coordinates:

[0013] in, For the lateral normalized coordinates of the grid cells on the radar screen, The vertical normalized coordinates of the grid cells on the radar screen. For the horizontal index of the grid cell, The total number of horizontal grids to divide the radar screen. For the vertical index of the grid cell, The total number of vertical grids to be divided for the radar screen; Convert normalized coordinates to polar coordinates:

[0014] in, The actual distance between grid cells. The polar angle of the grid cell; Calculate the actual azimuth angle:

[0015] in, This is the actual azimuth angle. This refers to the aircraft's heading angle; Calculate latitude and longitude offset:

[0016]

[0017] in, This represents the latitudinal offset of the grid cell. This represents the longitude offset of the grid cell. This refers to the aircraft's current actual latitude. Calculate the final geographic coordinates of the grid cells:

[0018]

[0019] in, The final geographic latitude of the grid cell. The final geographic longitude of the grid cell. This is the aircraft's current actual longitude; Record the time, aircraft position, heading angle, and geographic coordinates and echo intensity values ​​of each grid cell to form a geographic information dataset.

[0020] Furthermore, based on the extracted dataset, a neural network model for identifying radar echo intensity is trained, and the optimal model is used in practical applications, including: The dataset is divided into training, validation, and test sets proportionally. The UNet network architecture is adopted. After the data is normalized and the parameters are initialized, it is trained through an encoder-decoder structure. The cross-entropy loss function and stochastic gradient descent optimization algorithm are used to minimize the loss. The test set is used to evaluate multiple network parameter models saved during training, and the optimal model is selected based on the evaluation metrics for actual radar echo intensity identification.

[0021] Furthermore, the identified alphanumeric meteorological data is paired and deduplicated with radar echo intensity and geographic coordinate data extracted from color images according to time and space, forming a structured dataset which is then stored, including: The identified meteorological element values, radar echo intensity, and geographic coordinate data are paired and combined one by one according to the collection time and spatial location. After checking and ensuring the uniqueness of the data at the same time point, the data is stored in the memory.

[0022] In a second aspect, the present invention provides an airborne real-time meteorological data acquisition device for implementing the method described above, the device comprising: The video acquisition module includes at least one high-definition camera for synchronously recording video of meteorological and flight parameters displayed on the aircraft's instrument panel and meteorological equipment displays; The data processing module, which is communicatively connected to the video acquisition module, is used to receive and process video data transmitted by the video acquisition module. It is configured to perform the following: extracting video frames into image sequences; classifying, cropping, and recognizing characters in the images to extract alphanumeric meteorological data; and extracting radar echo intensity, mapping geographic coordinates, and performing data fusion processing on color images. A data storage module, connected to the data processing module, is used to store a structured meteorological dataset generated by the data processing module and paired by time and space.

[0023] Furthermore, the data processing module further includes: The video frame extraction unit is used to convert the video stream into batches of images at set time intervals; The image classification unit integrates a pre-trained image classification convolutional neural network to classify images into alphanumeric and color categories. The character recognition processing unit is used to perform region cropping and unit segmentation on alphanumeric images, and to perform character recognition and combination using a pre-trained alphanumeric recognition convolutional neural network. The radar data extraction and geographic mapping unit is used to perform the following actions: identify radar image color marks and establish a color-intensity mapping relationship; grid the radar echo image and convert the color of each grid into an echo intensity value according to the mapping relationship; and map each grid cell (i, j) to actual geographic coordinates according to the aircraft's real-time position, heading angle, and preset display range parameters using the following geographic mapping calculation model: The data fusion unit is used to perform spatiotemporal matching, deduplication, and integration of meteorological data output by the character recognition processing unit and echo intensity and coordinate data output by the radar data extraction and geographic mapping unit.

[0024] The airborne real-time meteorological data acquisition method provided by this invention has at least the following beneficial effects: 1. This invention achieves seamless integration of data from two independent systems: the original aircraft system (airborne radar, navigation system, flight parameters, etc.) and meteorological equipment. This significantly expands the dimensions of meteorological observation data, creates additional data value for each flight operation, and improves the overall efficiency of flight missions.

[0025] 2. The navigation and radar systems onboard the weather modification aircraft have extremely high accuracy and reliability. After the data generated by these systems is extracted by this invention, higher accuracy and higher resolution observation results can be obtained than those of conventional meteorological equipment, providing a scientific and reliable basis for meteorological scientific research, numerical model improvement and the enhancement of decision-making capabilities for weather modification operations.

[0026] 3. This invention adopts a completely non-invasive data acquisition method, which does not require any physical connection or modification to the original aircraft system, completely avoids flight safety risks and aviation regulations, overcomes the technical barriers of traditional data acquisition methods, and achieves compliant acquisition of key meteorological data.

[0027] 4. This invention applies computer vision and deep learning technologies to the field of meteorological observation, and achieves efficient conversion from video images to structured meteorological data through intelligent recognition and automatic processing technologies.

[0028] 5. The system structure composed of the camera, industrial tablet computer and memory used in this invention is simple, compact and lightweight. It is easy to install and deploy on various artificial weather modification aircraft, requires no professional engineering modification, is easy to operate and has low maintenance costs. It has strong practicality and promotional value.

[0029] 6. Compared with traditional meteorological observation equipment and data acquisition systems, the cost of this invention is significantly reduced (only a few thousand to tens of thousands of yuan), while it can acquire high-quality meteorological data with a value far exceeding the system cost, resulting in a very high input-output ratio and obvious economic benefits. Attached Figure Description

[0030] Figure 1 A flowchart of a data acquisition system for implementing an airborne real-time meteorological data acquisition method, provided in an embodiment of the present invention; Figure 2 A flowchart of an airborne real-time meteorological data acquisition method provided in this embodiment of the invention; Figure 3 This is a flowchart of video acquisition provided in an embodiment of the present invention; Figure 4 This is a flowchart of image classification provided in an embodiment of the present invention; Figure 5 This is a flowchart of an image cropping process provided in an embodiment of the present invention; Figure 6 This is a flowchart of number and letter recognition provided in an embodiment of the present invention; Figure 7 The execution flowchart of the radar echo intensity data extraction and geographic coordinate mapping sub-steps provided in the embodiments of the present invention; Figure 8 This is a flowchart illustrating the identification of radar echo intensity color marks and the establishment of a mapping relationship provided in an embodiment of the present invention; Figure 9 A flowchart for extracting intensity data from radar echo images provided in an embodiment of the present invention; Figure 10 A flowchart for establishing the correspondence between image pixels and geographic coordinates provided in an embodiment of the present invention; Figure 11 A flowchart for establishing and training a network model for recognizing radar echo intensity, provided in an embodiment of the present invention; Figure 12 A flowchart illustrating the data fusion and storage process provided in this embodiment of the invention; Figure 13 A structural diagram of an airborne real-time meteorological data acquisition device provided in this embodiment of the invention; Figure 14 This invention provides a structural diagram of the data processing module in an airborne real-time meteorological data acquisition device.

[0031] The accompanying drawings have illustrated specific embodiments of the invention, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the invention in any way, but rather to illustrate the concept of the invention to those skilled in the art through reference to specific embodiments. Detailed Implementation

[0032] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0033] The technical solution of the present invention and how the technical solution of the present invention solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0034] This invention provides an airborne real-time meteorological data acquisition method, particularly suitable for the real-time acquisition and post-processing of meteorological data on aircraft used for weather modification operations. This airborne real-time meteorological data acquisition method can be implemented through a data acquisition system, such as... Figure 1As shown, the data acquisition system includes a camera for recording high-definition video, an industrial micro-tablet PC for processing real-time meteorological data from the high-definition video, and a large-capacity storage device for storing video data and real-time meteorological data. The high-definition camera records the instrument panel of an aircraft used for weather modification, capturing information such as altitude and latitude / longitude. The industrial micro-tablet PC runs programs such as video-to-image conversion software, image classification networks, color image extraction data networks, number and letter classification networks, and number and letter image cropping software to extract real-time meteorological data and save it to the storage device. Specifically, the camera records the aircraft's instrument panel, acquiring video input and transmitting it to the industrial tablet PC; the tablet PC runs the data extraction program, first converting the video into images, and then using a pre-trained image classification network to classify the images into number / letter images and different color images (i.e., color images). For alphanumeric images, containing information such as date 12-NOV-24, time 02:19, air pressure, temperature, wind speed, wind direction, latitude and longitude, altitude, and humidity, the system first crops them into smaller images containing multiple meteorological elements. This is then further cropped into images containing only a single digit or letter. Finally, a trained alphanumeric classification network is used for recognition, outputting structured results. For color-based images, such as radar echo images, the system processes the data through an extraction network and uses a trained automatic echo intensity recognition network to convert color information into specific echo intensity values. During this process, the system can also perform a step of filtering for spatiotemporal consistency with ground radar, matching and verifying airborne data with ground radar echo images and intensity data. Ultimately, the system pairs and stores all identified data, including alphanumeric information and radar echo intensity, removing duplicate information to form a spatiotemporally consistent dataset stored in memory. This data provides support for weather mechanism and atmospheric science research. The image classification network, digit and letter classification network, and echo intensity recognition network involved in the system all need to be trained in advance using labeled data.

[0035] like Figure 2 As shown, the airborne real-time meteorological data acquisition method includes the following steps S10-S70 in its specific implementation.

[0036] S10: When artificial weather modification aircraft are operating, at least one high-definition camera shall be installed to simultaneously record the meteorological parameters and flight parameters displayed on the aircraft's instrument panel.

[0037] The purpose of step S10 is to acquire video. Each time a weather modification aircraft performs an operation, a high-definition camera is mounted in front of the aircraft's instrument and weather device displays to record video. In some embodiments, such as... Figure 3 As shown, step S10 specifically includes the following steps S101-S102.

[0038] S101: Install cameras to record video information from the instrument panel of aircraft used for weather modification.

[0039] The weather modification aircraft is equipped with two sets of meteorological detection equipment. One set, which comes standard with the aircraft and includes airborne radar, primarily serves the captain and maintenance personnel, ensuring flight safety. The other set is a modified meteorological system used by weather modification personnel for reference. The observation data from the two sets of equipment is not shared; the data from the aircraft's standard equipment is only displayed on the instrument panel and cannot be stored. Considering aviation safety, methods such as connecting split screens are not suitable and would require approval from various civil aviation departments, which is difficult. Therefore, a camera needs to be mounted directly opposite the instrument panel to record video information of multiple meteorological parameters and radar echoes during flight.

[0040] S102: Set up a camera to record information from the display of the meteorological detection equipment carried by the personnel boarding the aircraft for the shadowing operation.

[0041] S103: Debug the time display of the two cameras: ensure that the time display of the two cameras is synchronized with the network time.

[0042] S20: The recorded video is processed by software and extracted into a series of images according to the set frame rate.

[0043] Step S20 is used to convert the video obtained in step S10 into a batch of images by running the video-to-batch-image software in the industrial micro tablet computer to obtain a series of batch images.

[0044] In some embodiments, step S20 can convert the video into a batch of images by using software such as Jinzhou Video Converter, Chuqing Editing Master, Adobe Photoshop, and Adobe Premiere Pro. Taking Jinzhou Video Converter as an example, after preparing the video file, open the software, select the "Video to Image" function, import the video file, and set the relevant output parameters according to your needs, such as the number of frames extracted per second, the format and resolution of the output image, etc. Click the "Start Conversion" button, and the software will automatically process the video file and generate a batch of images. ; S30: Using a pre-trained image classification neural network, the extracted images are classified into alphanumeric images and color images.

[0045] In some embodiments, such as Figure 4 As shown, step S30 includes the following steps S301-S303.

[0046] S301: Create network labels for training the recognition of numbers and letters: Label the selected images one by one, labeling images of numbers and letters as 0 and images of non-numbers and letters (images of different colors) as 1, to complete the manual labeling of the network labels for recognizing numbers and letters, which is used to train the network for recognizing numbers and letters. S302: Preprocessing image data and splitting the dataset: Adaptive thresholding is used to binarize the images to simplify image features and reduce computation. The preprocessed images are then split into training and test sets with a ratio of 70% for the training set and 30% for the test set. ; S303: Selecting the optimal CNN network parameter model: Evaluate the 50 saved CNN network parameter models on the test set data using evaluation metrics. The method is to calculate the accuracy, recall, and other metrics of each CNN network parameter model on the test set, and select the model with the highest comprehensive evaluation score for practical application based on the evaluation results.

[0047] S40: For the classified alphanumeric images, perform preliminary cropping according to the area of ​​their displayed physical meaning to obtain sub-images containing independent meteorological element values, and further crop the sub-images into unit images containing only a single number or letter.

[0048] In some embodiments, such as Figure 5 As shown, step S40 includes the following steps S401-S402.

[0049] S401: Cropping alphanumeric images into small images containing single information about various meteorological elements, time, and space: Using professional image editing software such as XnConvert and Photoshop, alphanumeric images are batch-cropped into small images according to the areas displaying meteorological element data values ​​such as time, latitude and longitude, altitude, wind speed, wind direction, air pressure, and temperature, and saved one by one according to time, location, altitude, etc. S402: Continue cropping small images of single information into small images of single numbers and letters: Continue using professional image editing software to crop the small images after batch cropping in step S401 into small images containing only single numbers and letters, and save them by category and order.

[0050] S50: Using a pre-trained digit and letter recognition neural network, it identifies specific numbers or letters in unit images and then combines them to reconstruct the values ​​of various meteorological elements.

[0051] In some embodiments, such as Figure 6 As shown, step S50 includes the following steps S501-S504.

[0052] S501: Create network labels for recognizing numbers and letters: Manually label the small images after batch cropping in step S402 one by one. The labeling results are used as network labels for training to recognize numbers and letters. The labeling results are 10 numbers from 0 to 1 and 26 letters from A to Z, for a total of 36 categories of sample labels. S502: Divide the training dataset for the digit and letter recognition network: Divide the dataset into a training set, a validation set, and a test set, with a ratio of 70% for the training set, 15% for the validation set, and 15% for the test set. The training set is used to train the network and adjust its parameters. The validation set is used to adjust hyperparameters (such as the learning rate) and does not participate in network training. The test set is used to evaluate the final network performance, and its results represent the network's performance in real-world applications. It does not participate in network training or hyperparameter tuning.

[0053] S503: Determine the architecture of the digit and letter recognition network and train the network: The digit and letter recognition network still adopts the CNN network architecture. Before training the network, the data is normalized and the parameters of the convolutional neural network are initialized. Then, a series of convolution operations, activation functions, pooling operations, and calculations of fully connected layers are performed. After obtaining the network's prediction results, the difference between the prediction results and the true labels is calculated using the cross-entropy loss function. Finally, the stochastic gradient descent algorithm is selected for optimization to calculate the gradient of the loss function with respect to each parameter, and the network parameters are updated to minimize the loss function. The network's hyperparameters are adjusted using a validation set every 5 training epochs, for a total of 50 training epochs, with the network parameters saved in each epoch. S504: Select the optimal model for recognizing numbers and letters: Evaluate the 50 saved CNN network parameter models on the test set data by calculating the accuracy, recall, and other indicators of each CNN network parameter model on the test set, and select the model with the highest comprehensive evaluation score for practical application based on the evaluation results.

[0054] S60: For the classified color images, perform the radar echo intensity data extraction and geographic coordinate mapping sub-steps.

[0055] In some embodiments, such as Figure 7 As shown, the sub-steps for radar echo intensity data extraction and geographic coordinate mapping include S601-S604.

[0056] S601: Identify color mark areas in radar display images and establish a mapping relationship between the RGB values ​​of the color mark and the range of radar echo intensity values.

[0057] In some embodiments, such as Figure 8 As shown, the above mapping relationship is identified through the following steps S6011-S6013.

[0058] S6011: Identifying color mark regions on airborne radar displays: Using combined image segmentation technology, the color mark regions on the radar displays are located and extracted from video frames, specifically including the following sub-steps: S60111: Preprocessing Enhancement: Gaussian filtering is applied to video frames to reduce noise, and contrast is enhanced by adaptive histogram equalization (CLAHE) to make color mark areas more obvious.

[0059] S60112: Edge Detection and Morphological Processing: The Canny edge detection algorithm is applied to extract edge features from the image, and then morphological closing is used to connect adjacent edges to form a complete contour. The high and low thresholds for Canny edge detection are set to 70% and 30% of the image gradient magnitude, respectively.

[0060] S60113: Contour Filtering and Color Mark Candidate Area Identification: Analyze the extracted contours, calculate the shape features of each contour (such as rectangularity, aspect ratio, area, etc.), and filter candidate areas that meet the characteristics of color mark areas. Specific filtering conditions are: rectangularity ≥ 0.85, representing the area ratio of the contour to its smallest bounding rectangle; aspect ratio conforms to the expected color mark shape, with vertical color marks at 3:1 to 10:1 and horizontal color marks at 1:3 to 1:10; area is within the expected range, typically 2% to 15% of the entire display area.

[0061] S60114: Select the region that best matches the color mark features as the final recognition result, and extract the precise boundary of the region for subsequent processing.

[0062] S6012: Based on the color scale range settings of the actual airborne radar image, establish a discrete mapping table from RGB colors to echo intensity ranges, in the following form: {(0, 0, 150): [0-5], (0,150,255): [5-10], ......}.

[0063] S6013: For any pixel color in the radar display area, the similarity of each standard color in the mapping table is evaluated by calculating the Euclidean distance in the RGB space, the most similar standard color is selected, and the corresponding echo intensity range is obtained. To simplify the processing, the median of the range can be directly used as the echo intensity value of the pixel.

[0064] S602: Divide the radar display image into a grid, apply the mapping relationship to the dominant color in each grid cell, and obtain the radar echo intensity value corresponding to that grid cell.

[0065] In some embodiments, such as Figure 9 As shown, intensity data can be extracted from radar echo images through the following steps S6021-S6023.

[0066] S6021: Preprocessing radar echo images: Preprocessing the radar display area in the video frame, including noise reduction, contrast enhancement, and color correction, to facilitate subsequent analysis.

[0067] S6022: Image Segmentation and Color Recognition: The radar echo image is segmented into grid cells, each cell corresponding to a geographical location. Statistical analysis is performed on the colors within each grid cell to extract the RGB values ​​of the dominant color.

[0068] S6023: Color to Intensity Value Conversion: Apply the mapping function established in step S6012 to convert the RGB color value of each grid cell into the corresponding radar echo intensity value.

[0069] S603: Establish the correspondence between radar image grid cells and actual geographic coordinates.

[0070] In some embodiments, a correspondence between image pixels and geographic coordinates is established, mapping the grid cells divided in step S602 to actual geographic coordinates, such as... Figure 10 As shown, the specific steps are as follows: S6031: Obtain basic aircraft information from devices such as Tiantong or Beidou, including the aircraft's current latitude and longitude, heading angle, and pitch angle.

[0071] S6032: Calculate the geographic coordinates of each grid cell: Assuming the radar screen is divided into m×n grid cells (e.g., 60×60), the screen origin is located at the aircraft's position, and the screen display range is R kilometers (e.g., 20 kilometers) in front of the aircraft, then calculate the geographic coordinates of each grid cell (i, j): S60321: Calculate the normalized coordinates of the mesh cells on the screen. The calculation formula is as follows:

[0072]

[0073] in, For the lateral normalized coordinates of the grid cells on the radar screen, The vertical normalized coordinates of the grid cells on the radar screen. For the horizontal index of the grid cell, The total number of horizontal grids to divide the radar screen. For the vertical index of the grid cell, The total number of vertical grids used to divide the radar screen.

[0074] S60322: Convert normalized coordinates to polar coordinates:

[0075]

[0076] in, The actual distance between grid cells. The polar angle of the grid cell.

[0077] S60323: Calculate the actual azimuth angle:

[0078] in, This is the actual azimuth angle. This is the aircraft's heading angle.

[0079] S60324: Convert distance and azimuth to latitude and longitude offset:

[0080]

[0081] in, This represents the latitudinal offset of the grid cell. This represents the longitude offset of the grid cell. This is the aircraft's current actual latitude.

[0082] S60325: Calculate the final geographic coordinates of the grid cells:

[0083]

[0084] in, The final geographic latitude of the grid cell. The final geographic longitude of the grid cell. This is the aircraft's current actual longitude.

[0085] S60326: Record the time corresponding to the current video, the latitude and longitude and heading angle of the aircraft, the latitude and longitude of all grid cells in the current screen and their corresponding pixel values ​​to form a dataset with geographic information.

[0086] S604: Based on the extracted dataset, train a neural network model for identifying radar echo intensity and use the optimal model for practical applications.

[0087] In some embodiments, such as Figure 11 As shown, step S604 includes the following steps S6041-S6043.

[0088] S6041: Divide the training dataset for the radar echo intensity recognition network: Divide the dataset into a training set, a validation set, and a test set, with a ratio of 70% for the training set and 30% for the validation set. The training set is used to train the network and adjust its parameters. The validation set is used to adjust parameters and select the best model; it does not participate in network training. The test set is used to evaluate the final network performance; its results represent the network's performance in real-world applications. It does not participate in network training or parameter tuning.

[0089] S6042: Determine the architecture of the radar echo intensity identification network and train the network: The radar echo intensity identification network adopts the UNet network architecture. Before training the network, the data is normalized and the parameters of the convolutional neural network are initialized. Then, the encoder and decoder are used. After obtaining the network's prediction results, the difference between the prediction results and the true labels is calculated using the cross-entropy loss function. Finally, the stochastic gradient descent algorithm is selected for optimization to calculate the gradient of the loss function with respect to each parameter, and the network parameters are updated to minimize the loss function. .

[0090] S6043: Selecting the optimal radar echo intensity network parameter model: Evaluate the 50 saved CNN network parameter models on the test set data by calculating the accuracy, recall, and other indicators of each CNN network parameter model on the test set, and select the model with the highest comprehensive evaluation score for practical application based on the evaluation results.

[0091] S70: The identified alphanumeric meteorological data is paired and deduplicated with radar echo intensity and geographic coordinate data extracted from color images according to time and space to form a structured dataset and store it.

[0092] In some embodiments, such as Figure 12 As shown, step S70 specifically includes the following steps S701-S702.

[0093] S701: Pairs and combines the recognized numbers and letters one by one; S702: Check the uniqueness of meteorological information at each point in time.

[0094] By employing the above technical solution, this invention transforms video information into image sequences and further utilizes deep learning convolutional neural networks to automatically identify, extract, and convert the image information into structured digital meteorological data, thereby constructing a complete and efficient non-intrusive data acquisition and processing workflow. This method not only provides a simple, reliable, and scientific means of recording and preserving aircraft instrument data that cannot be directly saved during weather modification operations, but more importantly, it successfully achieves information fusion between aircraft system data and professional meteorological equipment data without requiring any physical connection or modification of the original aircraft system. Ultimately, the automated conversion and fusion of video into structured digital information achieved by this invention provides a high-quality, spatiotemporally consistent data foundation for weather mechanism analysis, atmospheric science research, and operational decision support, possessing significant practical value and widespread applicability.

[0095] This invention also provides an airborne real-time meteorological data acquisition device for implementing the methods described in any of the above embodiments, such as... Figure 13 As shown, the device includes: The video acquisition module 131 includes at least one high-definition camera for synchronously recording videos of weather and flight parameters displayed on the aircraft's instrument panel and meteorological equipment display. The data processing module 132, which is communicatively connected to the video acquisition module 131, is used to receive and process video data transmitted by the video acquisition module 131. It is configured to perform the following: extracting video frames into image sequences; classifying, cropping, and recognizing characters in the images to extract alphanumeric meteorological data; and extracting radar echo intensity, mapping geographic coordinates, and performing data fusion processing on color images. The data storage module 132 is connected to the data processing module 132 and is used to store the structured meteorological dataset generated by the data processing module 132 and paired by time and space.

[0096] In some embodiments, such as Figure 14 As shown, the data processing module 132 further includes: The video frame extraction unit 1321 is used to convert the video stream into batch images at set time intervals; Image classification unit 1322 integrates a pre-trained image classification convolutional neural network for classifying images into alphanumeric and color classes; The character recognition processing unit 1323 is used to perform region cropping and unit segmentation on alphanumeric images, and to perform character recognition and combination using a pre-trained alphanumeric recognition convolutional neural network. The radar data extraction and geographic mapping unit 1324 is used to perform the following actions: identify radar image color marks and establish a color-intensity mapping relationship; grid the radar echo image and convert the color of each grid into an echo intensity value according to the mapping relationship; and map each grid cell (i, j) to actual geographic coordinates according to the aircraft's real-time position, heading angle, and preset display range parameters using the following geographic mapping calculation model: The data fusion unit 1325 is used to perform spatiotemporal matching, deduplication, and integration of meteorological data output by the character recognition processing unit and echo intensity and coordinate data output by the radar data extraction and geographic mapping unit.

[0097] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for airborne real-time meteorological data acquisition, characterized in that, include: When artificial weather modification aircraft are operating, at least one high-definition camera is installed to simultaneously record the meteorological parameters and flight parameters displayed on the aircraft's instrument panel. The recorded video is processed by software and extracted into a series of images according to a set frame rate; Using a pre-trained image classification neural network, the extracted images are classified into alphanumeric images and color images; For the classified alphanumeric images, preliminary cropping is performed according to the area of ​​their displayed physical meaning to obtain sub-images containing independent meteorological element values, and the sub-images are further cropped into unit images containing only a single number or letter; Using a pre-trained digit and letter recognition neural network, specific numbers or letters in the unit image are identified, and then combined to reconstruct the values ​​of various meteorological elements; For the classified color images, a sub-step of radar echo intensity data extraction and geographic coordinate mapping is performed. This sub-step includes: identifying color mark regions in the radar display image and establishing a mapping relationship between the RGB values ​​of the color mark colors and the range of radar echo intensity values; dividing the radar display image into a grid, applying the mapping relationship to the dominant color within each grid cell to obtain the radar echo intensity value corresponding to that grid cell; establishing a correspondence between radar image grid cells and actual geographic coordinates; and training a neural network model for identifying radar echo intensity based on the extracted dataset, and then applying the optimal model in a practical application. The identified alphanumeric meteorological data is paired and deduplicated with radar echo intensity and geographic coordinate data extracted from color images according to time and space to form a structured dataset and store it.

2. The airborne real-time meteorological data acquisition method according to claim 1, characterized in that, When weather modification aircraft are operating, at least one high-definition camera should be installed to simultaneously record video of the meteorological and flight parameters displayed on the aircraft's instrument panel, including: A high-definition camera was set up, facing the original instrument panel of the aircraft, to record video containing multiple meteorological parameters and radar echo information. A second high-definition camera was set up to record video information from the meteorological detection equipment displayed by the weather modification workers. Synchronize the time display of the two cameras with the network time.

3. The airborne real-time meteorological data acquisition method according to claim 1, characterized in that, When the recorded video is processed by software and extracted into a series of images according to a set frame rate, the video processing software is used to convert the video into batch images and filter out redundant and invalid images according to a set time interval.

4. The airborne real-time meteorological data acquisition method according to claim 1, characterized in that, Using a pre-trained image classification neural network, the extracted images are classified into alphanumeric images and color images, including: Create training labels, labeling images containing numbers and letters as the first category and colored images as the second category; After binarization preprocessing, the images are divided into training and test sets according to a certain ratio; Multiple convolutional neural network parameter models were evaluated using a test set, and the model with the highest overall score was selected for actual classification based on accuracy and recall metrics.

5. The airborne real-time meteorological data acquisition method according to claim 1, characterized in that, Identify color mark regions in radar display images and establish a mapping relationship between the RGB values ​​of the color mark and the range of radar echo intensity values, including: To identify color mark regions on an airborne radar display, the following processing steps are performed: Gaussian filtering is applied to video frames for noise reduction, and adaptive histogram equalization is used to enhance contrast; the Canny edge detection algorithm is applied to extract edges, with the high and low thresholds of the Canny edge detection set to 70% and 30% of the image gradient magnitude, respectively, and morphological closing operations are used to connect the edges to form contours; contours are selected based on the following conditions: rectangularity ≥ 0.85; aspect ratio conforms to the expected color mark shape, with vertical color marks being 3:1 to 10:1 and horizontal color marks being 1:3 to 1:10; area occupying 2% to 15% of the entire display area; the region that best meets the above characteristics is selected as the color mark region. Establish a discrete mapping table from RGB color values ​​to echo intensity ranges; For the pixel color within the radar display area, the most similar standard color and its corresponding echo intensity range are determined by calculating its RGB Euclidean distance to the standard color in the mapping table, and the median value of this range is used as the echo intensity value of the pixel.

6. The airborne real-time meteorological data acquisition method according to claim 1, characterized in that, Establishing the correspondence between radar image grid cells and actual geographic coordinates includes: Obtain the aircraft's current latitude and longitude, heading angle, and pitch angle; Calculate the geographic coordinates of each radar image grid cell (i, j), where the radar screen is divided into an m×n grid, and the display range is R kilometers in front of the aircraft. The specific calculation formula is as follows: Calculate normalized coordinates: ; ； in, For the lateral normalized coordinates of the grid cells on the radar screen, The vertical normalized coordinates of the grid cells on the radar screen. For the horizontal index of the grid cell, The total number of horizontal grids to divide the radar screen. For the vertical index of the grid cell, The total number of vertical grids to be divided for the radar screen; Convert normalized coordinates to polar coordinates: ; ； in, The actual distance between grid cells. The polar angle of the grid cell; Calculate the actual azimuth angle: ； in, This is the actual azimuth angle. This refers to the aircraft's heading angle; Calculate latitude and longitude offset: ； ； in, This represents the latitudinal offset of the grid cell. This represents the longitude offset of the grid cell. This refers to the aircraft's current actual latitude. Calculate the final geographic coordinates of the grid cells: ； ； in, The final geographic latitude of the grid cell. The final geographic longitude of the grid cell. This is the aircraft's current actual longitude; Record the time, aircraft position, heading angle, and geographic coordinates and echo intensity values ​​of each grid cell to form a geographic information dataset.

7. The airborne real-time meteorological data acquisition method according to claim 1, characterized in that, Based on the extracted dataset, a neural network model for identifying radar echo intensity is trained, and the optimal model is used in practical applications, including: The dataset is divided into training, validation, and test sets proportionally. The UNet network architecture is adopted. After the data is normalized and the parameters are initialized, it is trained through an encoder-decoder structure. The cross-entropy loss function and stochastic gradient descent optimization algorithm are used to minimize the loss. The test set is used to evaluate multiple network parameter models saved during training, and the optimal model is selected based on the evaluation metrics for actual radar echo intensity identification.

8. The airborne real-time meteorological data acquisition method according to claim 1, characterized in that, The identified alphanumeric meteorological data is paired and deduplicated with radar echo intensity and geographic coordinate data extracted from color images, according to time and space, to form a structured dataset, which is then stored. The identified meteorological element values, radar echo intensity, and geographic coordinate data are paired and combined one by one according to the collection time and spatial location. After checking and ensuring the uniqueness of the data at the same time point, the data is stored in the memory.

9. An airborne real-time meteorological data acquisition device, used to implement the method as described in any one of claims 1-8, characterized in that, The device includes: The video acquisition module includes at least one high-definition camera for synchronously recording video of meteorological and flight parameters displayed on the aircraft's instrument panel and meteorological equipment displays; The data processing module, which is communicatively connected to the video acquisition module, is used to receive and process video data transmitted by the video acquisition module. It is configured to perform the following: extracting video frames into image sequences; classifying, cropping, and recognizing characters in the images to extract alphanumeric meteorological data; and extracting radar echo intensity, mapping geographic coordinates, and performing data fusion processing on color images. A data storage module, connected to the data processing module, is used to store a structured meteorological dataset generated by the data processing module and paired by time and space.

10. The airborne real-time meteorological data acquisition device according to claim 9, characterized in that, The data processing module further includes: The video frame extraction unit is used to convert the video stream into batches of images at set time intervals; The image classification unit integrates a pre-trained image classification convolutional neural network to classify images into alphanumeric and color categories. The character recognition processing unit is used to perform region cropping and unit segmentation on alphanumeric images, and to perform character recognition and combination using a pre-trained alphanumeric recognition convolutional neural network. The radar data extraction and geographic mapping unit is used to perform the following actions: identify radar image color marks and establish a color-intensity mapping relationship; grid the radar echo image and convert the color of each grid into an echo intensity value according to the mapping relationship; and map each grid cell (i, j) to actual geographic coordinates according to the aircraft's real-time position, heading angle, and preset display range parameters using the following geographic mapping calculation model: The data fusion unit is used to perform spatiotemporal matching, deduplication, and integration of meteorological data output by the character recognition processing unit and echo intensity and coordinate data output by the radar data extraction and geographic mapping unit.

Images