Method and electronic device for identifying meteorological maps

Abstract

Method and electronic device for identifying weather maps. The method for identifying weather maps includes: obtaining a first weather map; extracting an image fusion feature of the first weather map, so that a trained weather map recognition model predicts and identifies the first weather map according to the image fusion feature, where the image fusion feature is formed by splicing a text-level feature and an image-level feature; and outputting a weather category of the first weather map.

Description

Technical Field

[0001] The present invention relates to the technical field of meteorological map recognition, and in particular, to a method and an electronic device for recognizing meteorological maps. Background Art

[0002] In order to effectively respond to meteorological changes and improve the ability of disaster monitoring and early warning, it is very necessary to recognize meteorological maps. With the development of artificial intelligence, a large number of technologies for recognizing and analyzing meteorological maps using neural networks have emerged one after another. For example, the prior art uses opencv (a cross-platform computer vision and machine learning software library) to process meteorological maps. That is, tesseract OCR (Optical Character Recognition) is used to recognize the text in the meteorological map, and then a neural network is constructed using tensorflow (an end-to-end open-source machine learning platform) to recognize and classify the meteorological map.

[0003] However, the text recognition rate of the tesseract OCR technology is very low and it is not friendly to Chinese recognition. The tesseract OCR technology is too cumbersome for manually selecting features such as the curve angle contour, wind direction angle, air pressure magnitude, and similarity of the picture, and it cannot accurately mine hidden features; moreover, for the same-sized input layer, the feedforward neural network requires too many model parameters, resulting in difficulty in obtaining the optimal parameters. That is, the tesseract OCR itself has shortcomings in the model and ignores the characteristics of the local correlation of meteorological maps.

[0004] Therefore, quickly outputting high-precision meteorological categories by recognizing meteorological maps is the problem to be solved by the present invention. Summary of the Invention

[0005] The purpose of the present invention is to provide a method and an electronic device for recognizing meteorological maps, which can at least recognize the input meteorological map and quickly output high-precision meteorological categories for each area (or specified area) and each geographical orientation of the meteorological map, providing decision-making support for scientifically understanding the formation mechanism of heavy pollution and accurately analyzing the spatial distribution and time-varying characteristics of pollutants during heavy pollution periods.

[0006] According to one aspect of the present invention, at least one embodiment provides a method for recognizing a meteorological map, including: obtaining a first meteorological map; extracting the image fusion features of the first meteorological map, so that a trained meteorological map recognition model predicts and recognizes the first meteorological map according to the image fusion features, wherein the image fusion features are composed of text-level features and image-level features spliced together; outputting the meteorological category of the first meteorological map.

[0007] According to one aspect of the present invention, at least one embodiment further provides a method for training a weather map recognition model, including: obtaining weather map samples; if the data of the weather map samples is balanced and the data volume meets the requirements, extracting training fusion features of the weather map samples, where the training fusion features are formed by splicing text-level features and image-level features; training the weather map recognition model based on the training fusion features, and making the weather map recognition model tend to be stable through the categorical_crossentropy loss function.

[0008] According to another aspect of the present invention, at least one embodiment further provides an electronic device, including: a processor, adapted to implement each instruction; and a memory, adapted to store multiple instructions, the instructions being adapted to be loaded and executed by the processor: the above-mentioned method for recognizing weather maps and / or the method for training a weather map recognition model of the present invention.

[0009] According to another aspect of the present invention, at least one embodiment further provides a system for recognizing weather maps, including: the above-mentioned electronic device of the present invention.

[0010] According to another aspect of the present invention, at least one embodiment further provides a computer-readable non-volatile storage medium, storing computer program instructions, when the computer executes the program instructions, executing: the above-mentioned method for recognizing weather maps and / or the method for training a weather map recognition model of the present invention.

[0011] Through the above-mentioned embodiments of the present invention, using the weather map as the input, extracting the text-level features and image-level features of the weather map, where the EAST text recognition model is used to extract the text-level features (involving text and space), and the VGG16 transfer learning model is used to extract the image-level features, and then predicting and classifying the weather map. The present invention combines some advantages of the convolutional neural network for image data (parameter sharing, local connection, rotation invariance), rather than analyzing the weather map from fixed index angles such as simple curve angle contours, wind direction angles, and air pressure magnitudes. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0013] Figure 1 It is a schematic diagram of weather categories according to an embodiment of the present invention;

[0014] Figure 2 It is a schematic diagram of the application environment according to an embodiment of the present invention;

[0015] Figure 3 It is a schematic diagram of an electronic device according to an embodiment of the present invention;

[0016] Figure 4 It is a flowchart of a method for training a meteorological map recognition model according to an embodiment of the present invention;

[0017] Figure 5 It is a schematic diagram of a meteorological map recognition model according to an embodiment of the present invention;

[0018] Figure 6 It is a flowchart of a method for recognizing a meteorological map according to an embodiment of the present invention;

[0019] Figure 7 It is a schematic diagram of the collected meteorological map according to an embodiment of the present invention;

[0020] Figure 8 It is a schematic diagram of an EAST text recognition model according to an embodiment of the present invention;

[0021] Figure 9 It is a schematic diagram of a VGG16 transfer learning model according to an embodiment of the present invention;

[0022] Figure 10 It is a schematic diagram of a meteorological map recognition model according to an embodiment of the present invention. Detailed implementation manners

[0023] Next, the technical solutions of the present invention will be described clearly and completely with reference to the accompanying drawings. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0024] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention described herein can be implemented in an order different from those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0025] Meteorological maps, also known as weather maps, are a general term for charts used to analyze the physical conditions and characteristics of the atmosphere. There are various meteorological categories according to different requirements and purposes. Generally speaking, as Figure 1 shown, the meteorological categories include: eight categories such as eastern high pressure, low pressure center, inverted trough of low pressure, uniform pressure field of low pressure, northern high pressure, western high pressure, high pressure center, and uniform pressure field of high pressure. By retrospectively analyzing the types of meteorological maps in the region during heavy pollution periods, it can provide decision-making support for the analysis of the pollutant diffusion mechanism and also provide decision-making support for accurately analyzing the spatial distribution and characteristics of pollutants changing over time during heavy pollution periods.

[0026] 1. Eastern high pressure: It is located behind the closed high pressure or behind the subtropical high pressure. The ground blows southerly winds. The ground is controlled by the southerly winds behind the high pressure, and the diffusion conditions are average.

[0027] 2. Low pressure center: It is located inside the closed isobars of the low pressure. The air flow rises, and there are many rainy and cloudy weather. The atmosphere is in an unstable state and is easy to diffuse.

[0028] 3. Inverted trough of low pressure: The isobars on the surface meteorological map are in the shape of an "A" - shaped low - pressure trough. Southerly winds often blow in front of the trough line, and there is upward movement. Abundant water vapor can form clouds and precipitation. Northerly winds often blow behind the trough line, and there is downward movement. The weather is often clear and cloudless. The pressure field is weak. Generally, the diffusion conditions are better in the eastern region and worse in the western region.

[0029] 4. Uniform pressure field of low pressure: It is in the uniform pressure field of an obvious low - pressure system. The atmosphere is stable and not conducive to diffusion.

[0030] 5. Northern high pressure: It is located in the bottom area of the closed high pressure. The ground blows easterly winds. The diffusion conditions are average.

[0031] 6. Western high pressure: It is in front of the continental cold high pressure. The ground blows northerly winds. Affected by the cold air, the near - surface is controlled by northerly winds turning, the wind speed increases, the humidity decreases, and the atmospheric diffusion conditions are better.

[0032] 7. High pressure center: It is near the high - pressure ridge line or inside the subtropical high pressure. The weather is mostly clear, the wind speed is small, the air sinks, and an inversion weather is likely to form, which is not conducive to diffusion.

[0033] 8. Uniform pressure field of high pressure: Generally, the ground is in a weak weather system, the wind speed is less than 1 m / s, the atmosphere is stable, and it is not conducive to diffusion.

[0034] Currently, a large number of technologies for identifying and analyzing meteorological maps using neural networks have emerged. For example, the meteorological map feature type identification system of the prior art includes: a meteorological map upload module, a meteorological map identification module, and a similarity matching module, which are used to identify meteorological maps at two different pressures, 500hPa and surface_pres. Among them, the 500hPa pressure is used to identify the westerly airflow, upper-air ridge, and subtropical high weather in the meteorological map, and the surface_pres pressure is used to identify the high pressure, isobaric, and typhoon weather in the meteorological map. However, the identification of these prior art meteorological maps not only occupies a large amount of computing resources but also has a low identification accuracy. There is an urgent need to provide a method for quickly and accurately identifying the meteorological categories of meteorological maps.

[0035] On this basis, at least one embodiment of the present invention provides a system for identifying meteorological maps, which includes an electronic device for identifying meteorological maps and / or an electronic device for training a meteorological map recognition model. The system for identifying meteorological maps may include an environment as Figure 2 shown. This environment may include a hardware environment and a network environment. The above-mentioned hardware environment includes: an electronic device for identifying meteorological maps and / or an electronic device for training a meteorological map recognition model, hereinafter collectively referred to as the electronic device 100; a server 200. The electronic device 100 can operate the server 200 through corresponding instructions, so as to read, change, add data, etc. The electronic device 100 can be one or more, and may also include multiple processing nodes, and the multiple processing nodes can be regarded as a whole externally.

[0036] Optionally, the electronic device 100 can also send the obtained first meteorological map and / or meteorological map samples to the server 200, so that the server 200 executes the method for identifying meteorological maps and / or the method for training a meteorological map recognition model of the present invention. Optionally, the electronic device 100 can be connected to the server 200 through a network. The above-mentioned network includes a wired network and a wireless network. The wireless network includes but is not limited to: wide area network, metropolitan area network, local area network or mobile data network. Typically, the mobile data network includes but is not limited to: Global System for Mobile Communications (GSM) network, Code Division Multiple Access (CDMA) network, Wideband Code Division Multiple Access (WCDMA) network, Long Term Evolution (LTE) communication network, WIFI network, ZigBee network, network based on Bluetooth technology, etc. Different types of communication networks may be operated by different operators. The type of communication network does not constitute a limitation to the embodiments of the present invention.

[0037] The electronic device 100, such as Figure 3As shown in the figure, it includes: a processor 301; and a memory 303 configured to store computer program instructions, which are suitable for being loaded and executed by the processor to implement the method for identifying meteorological charts and / or the method for training a meteorological chart recognition model developed by the present invention (which will be introduced in detail later). Optionally, at least one embodiment of the present invention further provides a computer-readable non-volatile storage medium storing computer program instructions, and when the computer executes the program instructions, it implements the method for identifying meteorological charts and / or the method for training a meteorological chart recognition model developed by the present invention.

[0038] The processor 301 can be various applicable processors, for example, implemented in the form of a central processing unit, a microprocessor, an embedded processor, etc., and can adopt architectures such as X86 and ARM. The memory 303 can be various applicable storage devices, such as non-volatile storage devices, including but not limited to magnetic storage devices, semiconductor storage devices, optical storage devices, etc., and can be arranged as a single storage device, an array of storage devices, or a distributed storage device, and the embodiments of the present invention do not limit these.

[0039] Those of ordinary skill in the art can understand that the structure of the above-mentioned electronic device 100 is only schematic and does not limit the structure of the device. For example, the device for delaying signals may further include more or fewer components (such as a transmission device) than those shown in Figure 3 the figure. The above-mentioned transmission device is used to receive or send data via a network. In one example, the transmission device is a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0040] Under the above operating environment, at least one embodiment of the present invention proposes a method for training a meteorological chart recognition model, which can be loaded and executed by the processor 301. The meteorological chart recognition model formed by this method can at least quickly and accurately identify the meteorological categories of meteorological charts. As Figure 4 shown in the flowchart of the method for training a meteorological chart recognition model, it should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order from here. The method may include the following steps:

[0041] Step S402, obtaining meteorological chart samples;

[0042] Step S404, if the data of the meteorological chart samples is balanced and the data volume meets the requirements, then extracting the training fusion features of the meteorological chart samples, where the training fusion features are composed of text-level features and image-level features spliced together;

[0043] In step S406, the weather map recognition model is trained based on the training fusion features, and the categorical_crossentropy loss function is used to make the weather map recognition model tend to be stable.

[0044] This training method aims to use a network model built with the ideas of transfer learning and multi-model feature fusion, with weather map samples as the input, to intelligently and efficiently determine the weather categories of the weather map samples, and immediately display the weather categories of each area (or specified area) on the page and show the confidence level, so as to assist the monitoring personnel and speed up the speed of judging the weather categories.

[0045] In step S402, weather map samples are obtained. Optionally, the present invention can obtain the weather map samples that have been manually annotated within the historical range for training and prediction. Among them, after the weather type of the weather map sample is manually annotated, the weather type will be reflected in the file name, and the annotation data is annotated at one time by the same batch of people to achieve uniformity.

[0046] That is to say, in order to maintain the data consistency of the weather map samples, the present invention uses GFS (Global Forecast System) data to draw weather map samples by itself through ncl software (a software commonly used in meteorological and ocean mapping), and the weather map samples are annotated by personnel with professional meteorological experience and stored in a specified disk directory or ftp server. The above storage in the specified disk directory or ftp server can be: the acquisition program recognizes the weather type according to the file name, and then the weather map samples collected for training can be transmitted and distributed to different folders according to the classification data. The above data transmission process can be remote transmission functions or hardware such as ftp, sftp, and ssh.

[0047] In step S404, if the data of the weather map sample is balanced and the data volume meets the requirements, the training fusion features of the weather map sample are extracted, where the training fusion features are composed of text-level features and image-level features spliced together. That is to say, before extracting the fusion features of the weather map sample, the present invention will perform data cleaning and standardization processing on the weather map sample, and then mainly use the pictures after data cleaning and standardization processing to extract the training fusion features.

[0048] Given that meteorological field images are different from other object classifications and there are many isobars, data such as map boundaries in meteorological maps will have a great impact on the model. Therefore, the above cleaning can remove interfering data such as map boundaries, legends, and invalid numbers. When cleaning, the Image method and putpiel method in PIL (Python Imaging Library, a very powerful and easy-to-use image processing library on the Python platform) can be used to read meteorological maps to obtain image pixel information, remove interfering data such as map boundaries, legends, and invalid numbers, and save the processed images to help the model produce high-quality outputs. The above Image method and putpiel method will be elaborated in detail in the subsequent process of the method for identifying meteorological maps described in this article.

[0049] In addition, from the perspective of historical meteorological map samples in the past, the distribution of each sample is extremely unbalanced, and the difference between each category is relatively large. For example, the difference between the low-pressure center and the high-pressure uniform pressure field is more than 60 times. This sample imbalance in small samples will cause the model prediction results to mainly shift towards large categories, resulting in low model accuracy. Therefore, the standardization process of the present invention can determine whether the data of meteorological map samples is balanced and whether the data volume meets the requirements. For unbalanced meteorological map samples, a series of random transformations are used to "expand" them and output the processed data. For example, the ImageDataGenerator method in keras (an open-source Python deep learning framework) is used to perform operations such as rotation, translation, and scaling on the original unbalanced data to upsample the samples, so as to balance the data of meteorological map samples and expand the dataset. In this way, the subsequent trained model will never see two completely identical images, which helps to prevent overfitting and helps the trained model generalize better.

[0050] Therefore, after the data cleaning and standardization process of the meteorological map samples in the present invention, if the data of the meteorological map samples is balanced and the data volume meets the requirements, then the training fusion features of the meteorological map samples are further extracted, where the training fusion features are composed of the text-level feature feature2 and the image-level feature feature1 spliced together. For example, the EAST text recognition model is used to extract the text-level feature feature2 of the meteorological map samples; the VGG16 transfer learning model is used to extract the image-level feature feature1 of the meteorological map samples; the text-level feature feature2 and the image-level feature feature1 are spliced and merged and then normalized to form the image fusion feature.

[0051] That is to say, for the weather map after data cleaning and standardization processing of the present invention, the open-source algorithm EAST text recognition model is used to recognize text-level features in the weather map, such as the high-pressure center, low-pressure center, the values of isobars, and the positions of the high-pressure center and low-pressure center. The transfer learning is used to call the VGG16 transfer learning model (wherein the fully connected module of the VGG16 transfer learning model is removed) to retrain and obtain the deep-level image-level features of the image. This transfer learning reuses the model parameters, which not only appropriately improves the training efficiency and reduces the training time, but also obtains advanced image-level features.

[0052] The present invention uses transfer learning to call the VGG16 transfer learning model, which can reduce the model parameters. The model can be directly trained on the ImageNet dataset. Through fine-tuning, it can learn the features related to our dataset. Without the cumbersome construction of a complex feedforward neural network to learn features, it can better find the hidden features in the data. It should be noted that in the subsequent method for recognizing weather maps of the present invention, the EAST text recognition model and the VGG16 transfer learning model and their parameters can also be reused, and the EAST text recognition model and the VGG16 transfer learning model of the present invention will be elaborated in detail in the process of describing the method for recognizing weather maps.

[0053] In step S406, the weather map recognition model is trained based on the training fusion features, and the weather map recognition model is made to tend to be stable through the categorical_crossentropy loss function. The training result is stored in the image storage unit or the memory 303, and the training result is the predicted weather category result and its confidence. Optionally, the weather map recognition model includes but is not limited to an input layer, a first fully connected layer, a first Dropout layer, a second fully connected layer, a second Dropout layer, and an output layer. Training the weather map recognition model may include: the training fusion features sequentially pass through the input layer, the first fully connected layer, the first Dropout layer, the second fully connected layer, the second Dropout layer, and are output at the output layer. For example, Figure 5As shown, the input layer is, for example, a concat layer. The first fully connected layer includes, but is not limited to, 1024 output neurons (fc 1024). The first Dropout layer is used to delete a certain proportion, such as 20%, of the output neurons (Dropout 0.2). The second fully connected layer includes, but is not limited to, 256 output neurons (fc 256). The second Dropout layer is used to delete a certain proportion, such as 20%, of the output neurons (Dropout 0.2). The output layer includes, but is not limited to, 8 output neurons (fc 8). It should be noted that the above output neurons can be flexibly adjusted according to actual needs. For example, in this article, since 8 types of meteorological categories are to be recognized, the number of output neurons in the output layer is set to 8. If 12 types of meteorological categories are to be recognized, the number of output neurons in the output layer can be set to 12.

[0054] That is to say, the present invention uses the trained fusion features as the input of the neural network - meteorological map recognition model, and through the input layer, two fully connected layers, two Dropout layers, and the Softmax layer, obtains the probability distribution of the meteorological classification categories of the meteorological map. The optional operation method is as follows: construct a model optimizer including a concat layer (i.e., the input layer), two fully connected layers, two dropout layers, and an output layer. The optimizer is SGD (stochastic gradient descent algorithm), the learning rate is 0.0001, and the loss function is categorical_crossentropy neural network training module.

[0055] Through the above manner of the present invention, several fast and high-precision meteorological map recognition models can be trained. The meteorological map recognition model uses models such as the EAST text recognition model and the VGG16 transfer learning model to form a multi-model feature fusion classification and recognition system. Subsequently, the present invention can use the trained meteorological map recognition model to recognize and analyze the meteorological categories of any meteorological map (not limited to the 8 types of meteorological categories listed in this article). It can not only quickly output the high-precision meteorological categories of each region (or designated region) and each geographical orientation of the meteorological map, but also provide decision support for scientifically understanding the formation mechanism of heavy pollution and accurately analyzing the spatial distribution and time-varying characteristics of pollutants during heavy pollution periods.

[0056] Under the above operating environment, at least one embodiment of the present invention proposes a method for recognizing meteorological maps. This method can be loaded and executed by the processor 301, and this method can at least quickly and accurately recognize the meteorological categories of meteorological maps. As Figure 6Flowchart of a method for identifying weather maps. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here. The method may include the following steps:

[0057] Step S602, obtain a first weather map;

[0058] Step S604, extract the image fusion features of the first weather map, so that the trained weather map recognition model can perform predictive recognition on the first weather map according to the image fusion features. Among them, the image fusion features are formed by splicing text-level features and image-level features;

[0059] Step S606, output the weather category of the first weather map.

[0060] Through the above embodiments of the present invention, using the weather map collected in real time or non-real time as the input, the weather category of the weather map is output by the trained weather map recognition model. The model is simple, fast, and has high accuracy.

[0061] In step S602, obtain a first weather map. Optionally, collect weather maps in real time or non-real time; use different colors to remove the interference data of the weather map to form the first weather map. The above collection of weather maps can be: use an image acquisition device to collect the latest generated weather map of a certain province (or a certain region) in real time. The source of the weather map can be a sea level pressure map downloaded from websites such as the Central Meteorological Observatory, the Japan Meteorological Agency, and the South Korea Meteorological Administration, or it can also use GFS data and draw it by itself through software such as ncl, as Figure 7 shown. The above use of different colors to remove the interference data of the weather map can be: use the Image method and putpiel method in PIL (Python Imaging Library, a very powerful and easy-to-use image processing library on the Python platform) to remove the interference data.

[0062] Generally speaking, meteorological maps use different colors to represent the pressure field, pressure data values, isobars, map boundaries, legends, and / or invalid numbers. Therefore, the Image method in PIL is used to obtain the colors of meteorological maps according to different colors, and threshold rules are set according to different colors to remove map boundaries, legends, and invalid numbers. The method is as follows: (1) Use the open method of Image to read the original meteorological map material; (2) Obtain the number of pixel points in the length and width of the meteorological map and traverse the R, G, and B values (the value range is 0-255) of all points with length i and width j; (3) When the R value of each point is within the range of plus or minus 10 of the G value, and the G value is within the range of plus or minus 10 of the B value and the R value is within the range of plus or minus 10 of the B value, use putpixel in Image to assign the position of this point to white (255, 255, 255).

[0063] Meanwhile, for the noise data, the denoising method using the Image method and the putpiel method is as follows: (1) Convert the color image into a grayscale image through the floating-point algorithm Gray = R * 0.3 + G * 0.59 + B * 0.11 for the image after removing the map boundary, legend, and invalid number processing; (2) Threshold the grayscale Figure 2 values, determine a threshold of 115, pixels greater than the threshold are represented as white, and pixels less than the threshold are represented as black. In this way, the pixels (grayscale values) of the image are divided into two parts: 0 and 1. For example, 0 represents black and 1 represents white; (3) Use the isolated point algorithm to remove noise, that is, count the black points in the nine-square grid around the black point. If the number of black points is less than two, it proves that this point is an isolated point, and record the positions of all isolated points; (4) Use the obtained isolated point positions to assign the noise positions to white (255, 255, 255) for the RGB image after the above processing using putpixel in Image. In this way, various interference data in the meteorological map can be removed.

[0064] In step S604, the image fusion features of the first meteorological map are extracted so that the trained meteorological map recognition model can perform prediction and recognition on the first meteorological map according to the image fusion features. The above extraction of the image fusion features of the first meteorological map may include: using the EAST text recognition model to extract the text-level feature feature2 of the first meteorological map; using the VGG16 transfer learning model to extract the image-level feature feature1 of the first meteorological map; splicing and merging the text-level feature and the image-level feature and then performing normalization processing to form the image fusion feature, that is, the image fusion feature is composed of splicing the text-level feature and the image-level feature. The above text-level features include the first feature, the second feature, and the third feature for representing text and space.

[0065] The EAST text recognition model of the present invention includes but is not limited to a feature extraction layer Feature extractor stem (PVANet), a feature fusion layer Feature-merging branch, and an output layer Output layer. Extracting text-level features of the first meteorological map using the EAST text recognition model may include: the feature extraction layer outputs the first feature map f1, the second feature map f2, the third feature map f3, and the fourth feature map f4 through 4 convolutional layers with 64, 128, 256, and 384 channels; the feature fusion layer upsamples f1 by 2 times and concatenates it with f2 to form the first fusion map h1, h1 is upsampled by 2 times and concatenated with f3 to form the second fusion map h2, h2 is upsampled by 2 times and concatenated with f4 to form the third fusion map h3, and h3 forms the fourth fusion map h4 through a convolutional layer, where h1, h2, h3, and h4 are 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the first meteorological map respectively; the output layer outputs the first feature using a 1*1 convolutional layer with 1 channel, outputs the second feature using a 1*1 convolutional layer with 5 channels, and outputs the third feature using a 1*1 convolutional layer with 8 channels, where the first feature represents the probability that each pixel belongs to the text region, the second feature is used to predict the text features of the rotated rectangle, and the third feature is used to predict the text of the irregular quadrilateral.

[0066] As Figure 8 shown, the first meteorological map (such as Immage) forms the feature maps output by each layer through 4 convolutional layers with 64, 128, 256, and 384 channels as f1, f2, f3, and f4. f1 is upsampled by 2 times through the unpoll layer and concatenated with the upper-layer f2, then after passing through a 1*1 convolutional layer (feature dimensionality reduction) and a 3*3 convolutional layer, it is upsampled by 2 times through the unpoll layer and concatenated with f3; then after passing through a 1*1 convolutional layer (feature dimensionality reduction) and a 3*3 convolutional layer, it is upsampled by 2 times through the unpoll layer and concatenated with f4, fusing the four-layer feature maps, and the sizes of the respective fusion maps are 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the original image. The specific formula is:

[0067]

[0068]

[0069] The output layer after the feature fusion layer has three parts: score map, RBOX, and QUAD. The score map outputs a score map through a 1*1 convolutional layer with an output channel of 1, representing the probability that each pixel belongs to the text region, that is, the text feature featureA is obtained; RBOX is generated by two 1*1 convolutional layers with 5 channels. Among them, 4 channels respectively represent the 4 distances from the pixel position to the top, right, bottom, and left boundaries of the rectangle, and 1 channel represents the rotation angle of the bounding box. This part is used to predict the text feature featureB of the rotated rectangle; QUAD uses 8 numbers to represent the coordinate offsets from the four corner vertices {pi|i∈{1,2,3,4}} of the quadrilateral to the pixel position. Since each distance offset contains two numbers (Δxi, Δyi), the output contains 8 channels, and this part can predict the text featureC of the irregular quadrilateral.

[0070] That is to say, the present invention uses the EAST text recognition model to obtain the text features, the probability featureA that each pixel belongs to the text region, the text featureB of the rotated rectangle, and the text featureC of the irregular quadrilateral. After vectorizing these three feature data, they are merged into the text space feature feature2. It can be seen from the structure of the EAST text recognition model that the EAST text recognition model of the present invention adopts a multi-scale fusion method such as FCN for feature extraction, which is used for subsequent pixel-level text region prediction; at the same time, the text recognition model can detect oblique text. Due to considering the direction information, it can detect text in all directions. The present invention adopts the idea of FPN to extract multi-scale fusion features. Simply put, it is to generate pictures of different sizes, generate different features for each picture, make predictions respectively, and finally count the prediction results of all sizes, which can make the EAST text detector very robust. Even if the text is blurred, reflected, or partially occluded, it can still locate the text.

[0071] The VGG16 transfer learning model of the present invention includes but is not limited to the first convolutional module, the second convolutional module, the third convolutional module, the fourth convolutional module, the fifth convolutional module, and 1 fully connected module. Among them, the first 5 convolutional modules are responsible for feature extraction, and the last fully connected module is responsible for completing the classification task. Each convolutional module includes multiple convolutional layers and one pooling layer; the fully connected module contains one flatten layer and multiple fully connected layers. The number of channels in each block structure is the same. Each convolutional module uses a 3*3 convolutional kernel, and the padding is same. The pooling layer is used to compress data and parameter quantities when constructing the neural network, reducing overfitting. Here, the present invention can also delete the last fully connected module of the VGG16 transfer learning model and add a global average pooling layer GlobalAveragePooling2D to output image-level features.

[0072] Optionally, extracting the image-level features of the first meteorological map using the VGG16 transfer learning model may include: the first meteorological map sequentially passes through a first convolutional module, a second convolutional module, a third convolutional module, a fourth convolutional module, and a fifth convolutional module to generate image-level features. Among them, the first convolutional module includes, but is not limited to, 2 cascaded 3×3 convolutional kernels with 64 channels, the second convolutional module includes, but is not limited to, 2 cascaded 3×3 convolutional kernels with 128 channels, the third convolutional module includes, but is not limited to, 3 cascaded 3×3 convolutional kernels with 256 channels, the fourth convolutional module includes, but is not limited to, 2 cascaded 3×3 convolutional kernels with 512 channels, and the fifth convolutional module includes, but is not limited to, 2 cascaded 3×3 convolutional kernels with 512 channels. As Figure 9 shown:

[0073] For example, the first convolutional module includes two convolutional layers, each convolutional layer contains 64 3×3 convolutional kernels with a stride of 1 and a padding of "same", the activation function is ReLU, and the output size is 224×224×64; the pooling layer is max pooling, the filter is 2×2, and the stride is 2. Max pooling can reduce the deviation of the estimated value mean caused by the convolutional layer parameter error and retain more texture information; after being processed by the first convolutional module, the image size is halved, and the size after pooling becomes 112×112×64.

[0074] For example, the second convolutional module includes two convolutional layers, each convolutional layer contains 128 3×3 convolutional kernels with a stride of 1 and a padding of "same", the activation function is ReLU, and the output size is 112×112×128; the pooling layer is max pooling, the filter is 2×2, and the stride is 2; after being processed by the second convolutional module, the image size is halved, and the size after pooling becomes 56×56×128.

[0075] For example, the third convolutional module includes three convolutional layers, each convolutional layer contains 256 3×3 convolutional kernels with a stride of 1 and a padding of "same", the activation function is ReLU, and the output size is 56×56×256; the pooling layer is max pooling, the filter is 2×2, and the stride is 2; after being processed by the third convolutional module, the image size is halved, and the size after pooling becomes 28×28×256.

[0076] For example, the fourth convolutional module includes three convolutional layers. Each convolutional layer contains 512 3*3 convolutional kernels with a channel of 3 and a stride of 1, padding = same filling, the activation function is ReLU, and the output size is 28*28*512. The pooling layer is max pooling (maximum pooling), the filter is 2*2, and the stride is 2; after being processed by the fourth convolutional module, the image size is halved, and the size after pooling becomes 14*14*512.

[0077] For example, the fifth convolutional module includes three convolutional layers. Each convolutional layer contains 512 3*3 convolutional kernels with a channel of 3 and a stride of 1, padding = same filling, the activation function is ReLU, and the output size is 14*14*512; the pooling layer is max pooling (maximum pooling), the filter is 2*2, and the stride is 2; after being processed by the fifth convolutional module, the image size is halved, and the size after pooling becomes 7*7*512.

[0078] In the above manner of the present invention, input the first meteorological map, build a VGG16 transfer learning model without a fully connected layer and load the weights. At the last layer of the VGG16 transfer learning model, add GlobalAveragePooling2D to output the image-level feature feature1.

[0079] The present invention splices and combines the text-level feature feature2 and the image-level feature feature1 and then performs normalization processing. For example, use the concatenate method to combine feature1 and feature2 and then perform min-max normalization to form an image fusion feature. Furthermore, the trained meteorological map recognition model predicts and recognizes the first meteorological map based on this image fusion feature. The method for generating or training the training fusion feature in the method for training the meteorological map recognition model of the present invention is similar, and the relevant content can be referred to in this part, such as Figure 10 shown.

[0080] In step S606, output the meteorological category of the first meteorological map, and the meteorological category includes an eastern high pressure, a low pressure center, a low pressure inverted trough, a low pressure uniform pressure field, a northern high pressure, a western high pressure, a high pressure center, and / or a high pressure uniform pressure field. The above output of the meteorological category of the first meteorological map may include: displaying the meteorological category and its confidence level of each region (or a specified region) of the meteorological map.

[0081] In the above manner of the present invention, using the trained meteorological map recognition model to identify and analyze the meteorological category of any meteorological map can not only quickly output the high-precision meteorological categories of each region (or a specified region) and each geographical orientation of the meteorological map, but also provide decision support for scientifically understanding the formation mechanism of heavy pollution and accurately analyzing the spatial distribution and time-varying characteristics of pollutants during the heavy pollution period.

[0082] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for identifying meteorological maps, characterized in that, Including: Obtain a first meteorological map; Extract the image fusion features of the first meteorological map, so that a trained meteorological map recognition model predicts and recognizes the first meteorological map according to the image fusion features. Among them, the image fusion features are composed of text-level features and image-level features spliced together. Use the EAST text recognition model to extract the text-level features of the first meteorological map. The text-level features are sequentially composed of the probability features of each pixel belonging to the text area, the text features of the rotated rectangle, and the text features of the irregular quadrilateral. The text-level features include the first feature, the second feature, and the third feature. The first feature represents the probability of each pixel belonging to the text area. The second feature is used to predict the text features of the rotated rectangle. The third feature is used to predict the text of the irregular quadrilateral. The image-level features are obtained by transfer learning calling a transfer learning model, and the transfer learning reuses the transfer learning model parameters; Output the meteorological category of the first meteorological map.

2. The method according to claim 1, wherein Obtaining a first meteorological map includes: Collect meteorological maps in real time, where the meteorological maps use different colors to represent the pressure field, pressure data values, isobars, map boundaries, legends, and / or invalid numbers; Use the different colors to remove the interference data of the meteorological map to form a first meteorological map, where the interference data includes map boundaries, legends, and / or invalid numbers.

3. The method according to claim 1, wherein Extracting the image fusion features of the first meteorological map includes: Use the VGG16 transfer learning model to extract the image-level features of the first meteorological map; Splice and merge the text-level features and the image-level features and then perform normalization processing to form image fusion features.

4. The method according to claim 1, wherein the EAST text recognition model comprises a feature extraction layer, a feature fusion layer, and an output layer, characterized in that, Using the EAST text recognition model to extract the text-level features of the first meteorological map includes: The feature extraction layer outputs the first layer of feature map f1, the second layer of feature map f2, the third layer of feature map f3, and the fourth layer of feature map f4 through 4 convolutional layers with 64, 128, 256, and 384 channels; The feature fusion layer upsamples f1 by 2 times and splices it with f2 to form the first layer of fusion map h1. h1 is upsampled by 2 times and spliced with f3 to form the second layer of fusion map h2. h2 is upsampled by 2 times and spliced with f4 to form the third layer of fusion map h3. h3 passes through a convolutional layer to form the fourth layer of fusion map h4, where h1, h2, h3, and h4 are 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the first meteorological map respectively; The output layer outputs the first feature using a 1*1 convolutional layer with 1 channel, outputs the second feature using a 1*1 convolutional layer with 5 channels, and outputs the third feature using a 1*1 convolutional layer with 8 channels.

5. According to the method described in claim 3, the VGG16 transfer learning model includes a first convolutional module, a second convolutional module, a third convolutional module, a fourth convolutional module, and a fifth convolutional module, characterized in that, Using the VGG16 transfer learning model to extract the image-level features of the first meteorological map includes: The first meteorological map successively passes through a first convolutional module, a second convolutional module, a third convolutional module, a fourth convolutional module, and a fifth convolutional module to generate image-level features. Among them, the first convolutional module includes 2 cascaded 64-channel 3*3 convolutional kernels, the second convolutional module includes 2 cascaded 128-channel 3*3 convolutional kernels, the third convolutional module includes 3 cascaded 256-channel 3*3 convolutional kernels, the fourth convolutional module includes 2 cascaded 512-channel 3*3 convolutional kernels, and the fifth convolutional module includes 2 cascaded 512-channel 3*3 convolutional kernels.

6. According to the method described in claim 2, the meteorological categories include eastern high pressure, low pressure center, inverted trough of low pressure, uniform pressure field of low pressure, northern high pressure, western high pressure, high pressure center, and / or uniform pressure field of high pressure, characterized in that The meteorological categories output from the first meteorological map include: Display the meteorological categories and their confidence levels of each region of the meteorological map.

7. A method for training a meteorological map recognition model, characterized in that, Include: Obtain meteorological map samples; If the data of the meteorological map sample is balanced and the data volume meets the requirements, extract the training fusion features of the meteorological map sample. Among them, the training fusion features are composed of text-level features and image-level features spliced together. Use the EAST text recognition model to extract the text-level features of the meteorological map sample. The text-level features are successively merged by the probability features of each pixel point belonging to the text region, the text features of the rotated rectangle, and the text features of the irregular quadrilateral. The text-level features include a first feature, a second feature, and a third feature. The first feature represents the probability of each pixel point belonging to the text region. The second feature is used to predict the text features of the rotated rectangle. The third feature is used to predict the text of the irregular quadrilateral. The image-level features are obtained by transfer learning calling the transfer learning model, and the transfer learning reuses the transfer learning model parameters; Train the meteorological map recognition model based on the training fusion features, and make the meteorological map recognition model tend to be stable through the categorical_crossentropy loss function.

8. The method according to claim 7, wherein the weather map recognition model comprises an input layer, a first fully connected layer, a first Dropout layer, a second fully connected layer, a second Dropout layer, and an output layer, characterized in that Training the meteorological map recognition model based on the training fusion features includes: The training fusion features successively pass through an input layer, a first fully connected layer, a first Dropout layer, a second fully connected layer, a second Dropout layer, and are output at the output layer. Among them, the first fully connected layer includes 1024 output neurons. The first Dropout layer is used to delete 20% of the output neurons. The second fully connected layer includes 256 output neurons. The second Dropout layer is used to delete 20% of the output neurons. The output layer includes 8 output neurons.

9. The method according to claim 7, wherein the Keras ImageDataGenerator is used to rotate, translate, and scale the imbalanced data to balance the data of the meteorological map samples, characterized in that When the data of the meteorological map sample is balanced and the data volume meets the requirements, extracting the training fusion features of the meteorological map sample includes: Use the VGG16 transfer learning model to extract the image-level features of the meteorological map sample; Splice and merge the text-level features and the image-level features and then perform normalization processing to form image fusion features.

10. An electronic device, including: A processor, adapted to implement each instruction; and a memory, adapted to store a plurality of instructions, the instructions being adapted to be loaded and executed by a processor: the method for identifying a meteorological map according to any one of claims 1-6, and / or the method for training a meteorological map recognition model according to any one of claims 7-9.

Images