Deep processing method and system based on multi-modal data

Abstract

The application discloses a deep processing method and system based on multi-modal data, and relates to the technical field of meteorological data management. The method comprises the following steps: collecting multiple meteorological monitoring data; performing spatio-temporal attention analysis to obtain multiple spatio-temporal attention parameter sets; fusing multiple basic meteorological data distribution sets according to the multiple spatio-temporal attention parameter sets to obtain multiple meteorological data distributions of multiple meteorological data; constructing a temperature integrated prediction model for temperature distribution prediction; calculating multiple temperature prediction fusion coefficients; and fusing the multiple temperature prediction fusion coefficients to obtain the predicted temperature distribution of a prediction area. The method solves the technical problems that the existing meteorological data processing cannot effectively utilize multi-class meteorological variable prediction data, cannot accurately capture the spatial distribution characteristics and correlation of meteorological variables, and thus leads to insufficient accuracy, real-time performance and reliability of meteorological prediction. The method achieves the technical effects of improving the accuracy, real-time performance and reliability of meteorological prediction.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of meteorological data management, and particularly relates to a deep processing method and system based on multi-modal data. BACKGROUND

[0002] In the field of meteorological science, the accurate prediction of meteorological data has always been a hot and difficult point of research. As an important indicator for in-depth research on the climate system, the change direction and amplitude of air temperature can reflect climate variability. Nowadays, air temperature services cover many fields such as agriculture, tourism, shipping, etc. Especially under the background of continuous rise of global temperature, air temperature change has attracted global attention. Along with the global warming phenomenon, sudden and severe climate disasters occur frequently in a large range, sea level rises, and extreme weather occurs frequently, leading to an imbalance in the ecological environment. With the intensification of global climate change, the accuracy of meteorological prediction is increasingly important for agricultural production, transportation, energy scheduling, etc. However, the traditional meteorological prediction mainly relies on numerical weather prediction models. When dealing with multi-source and multi-class meteorological data, it is difficult to fully utilize the rich meteorological information in multi-class meteorological data and accurately capture the complex spatio-temporal correlation, making it difficult to meet the actual demand for prediction accuracy and real-time performance.

[0003] Therefore, in the current meteorological data processing related technology, there is a technical problem that multi-class meteorological variable prediction data cannot be effectively utilized, it is difficult to accurately capture the spatial distribution characteristics and correlation of meteorological variables, and thus the accuracy, real-time performance and reliability of meteorological prediction are insufficient. SUMMARY

[0004] The present application provides a deep processing method and system based on multi-modal data, which solves the technical problem that the existing meteorological data processing cannot effectively utilize multi-class meteorological variable prediction data, cannot accurately capture the spatial distribution characteristics and correlation of meteorological variables, and thus the accuracy, real-time performance and reliability of meteorological prediction are insufficient, and achieves the technical effect of improving the accuracy, real-time performance and reliability of meteorological prediction.

[0005] The application provides a deep processing method based on multi-modal data, which comprises: collecting a plurality of meteorological monitoring data monitored by a plurality of monitoring points, wherein each meteorological monitoring data comprises a plurality of types of meteorological data; performing spatio-temporal attention analysis on the plurality of types of meteorological data according to the monitoring coordinates of the plurality of monitoring points and in combination with the time stamps of the plurality of meteorological monitoring data, to obtain a plurality of spatio-temporal attention parameter sets; performing meteorological data distribution adversarial generation of a forecast area according to the plurality of meteorological monitoring data, to obtain a plurality of basic meteorological data distribution sets, and fusing the plurality of basic meteorological data distribution sets according to the plurality of spatio-temporal attention parameter sets, to obtain a plurality of meteorological data distributions of the plurality of types of meteorological data, wherein each meteorological data distribution comprises the distribution of one type of meteorological data in the forecast area; constructing a temperature integrated prediction model for temperature distribution prediction based on ensemble learning, wherein the temperature integrated prediction model comprises a plurality of temperature prediction channels, and each temperature prediction channel comprises a plurality of temperature prediction branches; analyzing a plurality of key variable parameters of the plurality of types of meteorological data based on historical temperature prediction data of the forecast area, and calculating a plurality of temperature prediction fusion coefficients in combination with the plurality of spatio-temporal attention parameter sets; calculating a plurality of prediction ratios according to the plurality of temperature prediction fusion coefficients, inputting the plurality of meteorological data distributions into the temperature prediction branches of the corresponding prediction ratios in the plurality of temperature prediction channels to perform temperature distribution prediction, to obtain a plurality of predicted temperature distributions, and fusing the plurality of predicted temperature distributions according to the plurality of temperature prediction fusion coefficients to obtain a predicted temperature distribution of the forecast area.

[0006] In a possible implementation, according to the monitoring coordinates of the plurality of monitoring points and in combination with the time stamps of the plurality of meteorological monitoring data, the spatio-temporal attention analysis of the plurality of types of meteorological data is performed to obtain a plurality of spatio-temporal attention parameter sets, and the following processing is performed: obtaining a plurality of monitoring coordinates of the plurality of monitoring points and a plurality of time stamps of the plurality of meteorological monitoring data; obtaining a sample monitoring coordinate set, a sample time stamp set, and a plurality of sample spatio-temporal attention parameter sets according to the historical meteorological monitoring data of the forecast area and according to the change amplitudes of the plurality of types of meteorological data under different coordinates and different times; training a spatio-temporal attention analyzer by using the sample monitoring coordinate set, the sample time stamp set, and the plurality of sample spatio-temporal attention parameter sets; and performing spatio-temporal attention analysis on the plurality of monitoring coordinates and the plurality of time stamps by using the spatio-temporal attention analyzer to obtain the plurality of spatio-temporal attention parameter sets.

[0007] In a possible implementation, according to the plurality of meteorological monitoring data, the meteorological data distribution of the forecast area is generated by confrontation, and a plurality of basic meteorological data distribution sets are obtained. The following processing is performed: according to the historical meteorological data monitoring of the plurality of monitoring points and the forecast area, a plurality of sample meteorological monitoring data sets and a plurality of sample meteorological data distribution sets are collected; a first sample meteorological monitoring data set of a first monitoring point and the plurality of sample meteorological data distribution sets are used to construct a first meteorological distribution generation confrontation path based on a generative adversarial network; the first meteorological monitoring data of the first monitoring point is input into the first meteorological distribution generation confrontation path for confrontation generation, and a first basic meteorological data distribution set is obtained; according to the plurality of meteorological monitoring data, the meteorological data distribution of the forecast area is generated by confrontation, and a plurality of basic meteorological data distribution sets are obtained.

[0008] In a possible implementation, the first sample meteorological monitoring data set of the first monitoring point and the plurality of sample meteorological data distribution sets are used to construct the first meteorological distribution generation confrontation path based on the generative adversarial network, and the following processing is further performed: the first generator and the first discriminator in the first meteorological distribution generation confrontation path are constructed based on the generative adversarial network; the first sample meteorological monitoring data set is input into the first generator for generation training of the meteorological data distribution set, the plurality of sample meteorological data distribution sets are used for discrimination training in the first discriminator, and the network parameters of the first meteorological distribution generation confrontation path are trained and updated; until the training meets the convergence requirement, the first meteorological distribution generation confrontation path is obtained.

[0009] In a possible implementation, the temperature integrated prediction model for temperature distribution prediction is constructed based on ensemble learning, and the following processing is further performed: according to the temperature prediction historical data of the forecast area, a plurality of sample meteorological data distribution sets of a plurality of types of meteorological data are collected, and a sample temperature distribution set is collected; a first sample meteorological data distribution set of a first type of meteorological data and the sample temperature distribution set are used to construct a plurality of first temperature prediction branches based on ensemble learning, and a first temperature prediction channel is obtained, wherein different groups of first temperature prediction construction data are collected multiple times in the first sample meteorological data distribution set and the sample temperature distribution set, and the plurality of first temperature prediction branches are constructed and trained to obtain; the plurality of sample meteorological data distribution sets and the plurality of sample temperature distribution sets of other types of meteorological data are further used to construct a plurality of temperature prediction branches based on ensemble learning, and a plurality of temperature prediction channels are obtained.

[0010] In a possible implementation, based on the historical temperature prediction data of the forecast area, a plurality of key variable parameters of the multi-type meteorological data are analyzed, a plurality of temperature prediction fusion coefficients are calculated in combination with the plurality of spatio-temporal attention parameter sets, and the following processing is further performed: based on the historical temperature prediction data of the forecast area, a plurality of meteorological data sequences of multi-type meteorological data and a temperature data sequence are collected; according to the plurality of meteorological data sequences and the temperature data sequence, variable correlation analysis is performed to obtain a plurality of key variable parameters; according to the spatio-temporal attention parameters of the multi-type meteorological data in the plurality of spatio-temporal attention parameter sets, a plurality of average spatio-temporal attention parameters are calculated; and the plurality of key variable parameters and the plurality of average spatio-temporal attention parameters are weighted and calculated to obtain the plurality of temperature prediction fusion coefficients.

[0011] In a possible implementation, according to the plurality of temperature prediction fusion coefficients, a plurality of prediction ratios are calculated, and the plurality of meteorological data distributions are input into temperature prediction branches of the corresponding prediction ratios in the plurality of temperature prediction channels to perform temperature distribution prediction, and the following processing is further performed: according to the plurality of temperature prediction fusion coefficients, in combination with the number of temperature prediction branches in each temperature prediction channel, a plurality of prediction ratios are obtained by integer calculation; the plurality of meteorological data distributions are respectively input into temperature prediction branches of the corresponding prediction ratios in the plurality of temperature prediction channels to perform temperature distribution prediction, a plurality of predicted temperature distribution sets are obtained, and a plurality of predicted temperature distributions are fused in the plurality of temperature prediction channels; and according to the plurality of temperature prediction fusion coefficients, the plurality of predicted temperature distributions are weighted and fused to obtain a predicted temperature distribution of the forecast area.

[0012] The application also provides a deep processing system based on multi-modal data, comprising: a meteorological monitoring data acquisition module, configured to acquire a plurality of meteorological monitoring data monitored and collected by a plurality of monitoring points, wherein each meteorological monitoring data comprises a plurality of types of meteorological data; a spatio-temporal attention parameter set obtaining module, configured to perform spatio-temporal attention analysis on the plurality of types of meteorological data according to monitoring coordinates of the plurality of monitoring points and in combination with time stamps of the plurality of meteorological monitoring data, and obtain a plurality of spatio-temporal attention parameter sets; a meteorological data distribution obtaining module, configured to perform meteorological data distribution generative adversarial generation in a forecast area according to the plurality of meteorological monitoring data, obtain a plurality of basic meteorological data distribution sets, fuse the plurality of basic meteorological data distribution sets according to the plurality of spatio-temporal attention parameter sets, and obtain a plurality of meteorological data distributions of the plurality of types of meteorological data, wherein each meteorological data distribution comprises a distribution of one type of meteorological data in the forecast area; a temperature integrated prediction model construction module, configured to construct a temperature integrated prediction model for temperature distribution prediction based on integrated learning, wherein the temperature integrated prediction model comprises a plurality of temperature prediction channels, and each temperature prediction channel comprises a plurality of temperature prediction branches; a temperature prediction fusion coefficient obtaining module, configured to analyze a plurality of key variable parameters of the plurality of types of meteorological data based on historical temperature prediction data of the forecast area, calculate a plurality of temperature prediction fusion coefficients in combination with the plurality of spatio-temporal attention parameter sets; and a forecast temperature distribution obtaining module, configured to calculate a plurality of prediction ratios according to the plurality of temperature prediction fusion coefficients, input the plurality of meteorological data distributions into temperature prediction branches of the corresponding prediction ratios in the plurality of temperature prediction channels to perform temperature distribution prediction, obtain a plurality of predicted temperature distributions, and fuse the plurality of predicted temperature distributions according to the plurality of temperature prediction fusion coefficients to obtain a forecast temperature distribution of the forecast area.

[0013] The deep processing method and system based on multi-modal data provided in the application can collect multiple meteorological monitoring data, perform spatio-temporal attention analysis on multiple meteorological data, obtain multiple spatio-temporal attention parameter sets, fuse multiple basic meteorological data distribution sets according to the multiple spatio-temporal attention parameter sets, obtain multiple meteorological data distributions of multiple meteorological data, construct a temperature integrated prediction model for temperature distribution prediction, calculate multiple temperature prediction fusion coefficients in combination with the multiple spatio-temporal attention parameter sets, obtain multiple predicted temperature distributions, and fuse the multiple predicted temperature distributions according to the multiple temperature prediction fusion coefficients to obtain a predicted temperature distribution of a predicted area. The technical problems that the existing meteorological data processing cannot effectively utilize multiple meteorological variable prediction data, cannot accurately capture the spatial distribution characteristics and correlation of meteorological variables, and thus cannot improve the accuracy, real-time performance and reliability of meteorological prediction are solved, and the technical effects of improving the accuracy, real-time performance and reliability of meteorological prediction are achieved. BRIEF DESCRIPTION OF DRAWINGS

[0014] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments of the present disclosure will be briefly introduced below. Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the foregoing or the following operations are not necessarily performed in sequence. On the contrary, various steps can be processed in reverse order or simultaneously as needed. Meanwhile, other operations can be added to these processes, or one or more steps can be removed from these processes.

[0015] Figure 1 The deep processing method based on multi-modal data provided in the embodiments of the present application is shown in the flowchart.

[0016] Figure 2 The structure of the deep processing system based on multi-modal data provided in the embodiments of the present application is shown in the schematic diagram.

[0017] Reference signs: meteorological monitoring data acquisition module 10, spatio-temporal attention parameter set obtaining module 20, meteorological data distribution obtaining module 30, temperature integrated prediction model constructing module 40, temperature prediction fusion coefficient obtaining module 50, and predicted temperature distribution obtaining module 60. DETAILED DESCRIPTION

[0018] The above description is only a summary of the technical solutions of the present application. In order to more clearly understand the technical means of the present application, the embodiments of the present application can be implemented according to the content of the specification, and in order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application are described below.

[0019] In order to make the purposes, technical solutions and advantages of the present application clearer, the following will further describe the present application in conjunction with the accompanying drawings, the described embodiments should not be regarded as limitations to the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0020] In the following description, "some embodiments" are involved, which describe a subset of all possible embodiments, but it can be understood that "some embodiments" can be the same subset or different subset of all possible embodiments, and can be combined with each other without conflict, the term "first\second" involved only distinguishes similar objects, and does not represent a specific order for the objects. The terms "include" and "have" and any variations, are intended to cover non-exclusive inclusion, for example, a process, method, system, product or server including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or modules not clearly listed or inherent to these processes, methods, products or devices. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as understood by those skilled in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application.

[0021] The embodiments of the present application provide a deep processing method based on multi-modal data, as shown in the following Figure 1 The method comprises the following steps:

[0022] In step S100, a plurality of meteorological monitoring data monitored and collected by a plurality of monitoring points are collected, wherein each meteorological monitoring data includes a plurality of types of meteorological data. Collecting a plurality of meteorological monitoring data means collecting meteorological monitoring data information from meteorological stations or sensor networks distributed in various geographical locations, and each meteorological monitoring data includes a plurality of types of meteorological data, that is, a plurality of different types of elements affecting air temperature, such as atmospheric pressure, geopotential height, wind direction and speed, precipitation, etc. Specifically, the change of atmospheric pressure can reflect the change of weather system, high pressure is usually related to sunny weather, and low pressure is related to storm and precipitation; geopotential height refers to the work done by a unit mass of air block rising to a certain height from sea level, also known as gravitational potential, and the change of geopotential height reflects the vertical structure and dynamic change of weather system; wind speed and wind direction together describe the state and characteristics of wind, and the size and direction of wind speed are crucial for predicting weather changes.

[0023] In step S200, according to the monitoring coordinates of the plurality of monitoring points, combined with the time stamps of the plurality of meteorological monitoring data, spatio-temporal attention analysis of the plurality of meteorological data is performed to obtain a plurality of spatio-temporal attention parameter sets. The spatio-temporal attention analysis combines the monitoring coordinates of the plurality of monitoring points and the time stamps of the meteorological monitoring data to achieve in-depth analysis of the plurality of meteorological data. Specifically, according to the coordinates of the plurality of monitoring points and the time of data collection, the change amplitude of the meteorological data that may change under the spatio-temporal is analyzed as a spatio-temporal attention parameter, that is, by analyzing the geographical coordinates (longitude, latitude, etc.) of the monitoring points, the distribution and change rule of the meteorological data at different geographical positions are analyzed, for example, the pressure distribution of a certain area, the change of wind direction and wind speed, etc.; by using the time stamp information of the meteorological data, the change trend of the meteorological elements with time is analyzed, for example, the real-time, seasonal change, long-term trend of the meteorological data, etc.; the spatio-temporal attention parameter set is the result of spatio-temporal attention analysis, which contains important parameters describing the spatio-temporal characteristics of meteorological data, for example, weight parameters are used to measure the importance of meteorological data of different monitoring points or different time points in the overall analysis, correlation parameters reflect the correlation of meteorological data between different monitoring points or different time points, and feature parameters represent key indicators of spatio-temporal characteristics of meteorological data. Through spatio-temporal attention analysis, the spatio-temporal distribution and change rule of meteorological data can be more deeply understood, and the spatio-temporal attention parameter set can be used in subsequent meteorological data fusion, construction of prediction model, etc. to improve the accuracy and real-time of meteorological prediction.

[0024] In a possible implementation, step S200 further includes step S210 of obtaining a plurality of monitoring coordinates of the plurality of monitoring points and a plurality of time stamps of the plurality of meteorological monitoring data. The monitoring point refers to a specific location or place for collecting meteorological data, which can be distributed in different geographical positions such as cities, rural areas, mountainous areas, oceans, etc. The monitoring coordinates are information used to identify the specific position of each monitoring point in the geographical space. Common coordinate systems include longitude and latitude coordinates and geographical coordinate systems. Each monitoring point has its own unique set of coordinate values. The time stamp is time information associated with meteorological monitoring data, which is used to identify the exact time when the data is collected or recorded. The time stamp is usually represented in the format of date and time, such as year-month-day hour:minute:second, which can help understand the change of meteorological conditions over time.

[0025] Step S200 further includes step S220 of obtaining a sample monitoring coordinate set, a sample timestamp set, and a plurality of sample spatiotemporal attention parameter sets according to the change amplitudes of the plurality of types of meteorological data at different coordinates and different times based on the historical meteorological monitoring data of the forecast area. The sample monitoring coordinate set refers to a set of geographic coordinates of each monitoring point extracted from the historical meteorological monitoring data of the forecast area, used to identify the specific position of each monitoring point in the geographical space. The sample timestamp set refers to a set of timestamp information corresponding to each data point extracted from the meteorological monitoring data, used to identify the exact time when the meteorological data is collected or recorded. For each monitoring point, the change amplitude is calculated based on the plurality of types of meteorological data (such as temperature, humidity, wind speed, etc.) at different times. If the calculated value is negative, it generally indicates that the data value has decreased, but when identifying the spatiotemporal attention parameter, it may need to be converted to an absolute value. Specifically, according to the change amplitudes of the plurality of types of meteorological data at different coordinates and different times, combined with the spatial distribution and time sequence characteristics of the meteorological data, a plurality of sample spatiotemporal attention parameter sets are identified, i.e. the smaller the change amplitude, the larger the spatiotemporal attention parameter, such as 1-change amplitude, which may include various types of parameters, such as weight parameters (used to measure the importance of meteorological data at different monitoring points or different time points in the overall analysis), correlation parameters (reflecting the correlation between meteorological data at different monitoring points or different time points), etc., used to describe the importance and correlation of meteorological data in time and space, reflecting the reliability of meteorological data in this space-time, the smaller the change amplitude, the larger the spatiotemporal attention parameter, the greater the reliability.

[0026] Step S200 further includes step S230 of training the spatiotemporal attention analyzer using the sample monitoring coordinate set, the sample timestamp set, and the plurality of sample spatiotemporal attention parameter sets. Based on a recurrent neural network (RNN) or a long short-term memory network (LSTM), a model of the spatiotemporal attention analyzer is designed, including a spatiotemporal convolution layer and an attention mechanism. The prepared sample monitoring coordinate set, sample timestamp set, and plurality of sample spatiotemporal attention parameter sets are used to train and verify the spatiotemporal attention analyzer until a satisfactory accuracy is achieved.

[0027] Step S200 further comprises step S240, performing spatio-temporal attention analysis on the plurality of monitoring coordinates and the plurality of timestamps by the spatio-temporal attention analyzer to obtain a plurality of spatio-temporal attention parameter sets. The plurality of monitoring coordinates and the plurality of timestamps are provided as input data to the spatio-temporal attention analyzer. The spatio-temporal attention analyzer first performs feature extraction on the input data, i.e., capturing useful spatio-temporal features such as spatial distribution and temporal variation of temperature, humidity, wind speed, etc. from the original data. An attention mechanism is applied to evaluate the importance of different features in time and space. According to the calculation result of the attention mechanism, the spatio-temporal attention analyzer generates a plurality of spatio-temporal attention parameter sets, reflecting the importance and relevance of meteorological data at different monitoring coordinates and timestamps.

[0028] Step S300, according to the plurality of meteorological monitoring data, the weather data distribution of the forecast area is generated, and a plurality of basic meteorological data distribution sets are obtained. The plurality of basic meteorological data distribution sets are fused according to the plurality of spatio-temporal attention parameter sets to obtain a plurality of meteorological data distributions of multiple weather data, wherein each meteorological data distribution includes the distribution of a type of weather data in the forecast area. According to the plurality of weather data in each meteorological monitoring data, the generation of the weather data distribution of the forecast area is generated, and then the plurality of spatio-temporal attention parameter sets are used to fuse the plurality of basic meteorological data distributions of each type of weather data to obtain a fused weather data distribution. Specifically, using the generative adversarial network (GAN) or its variant technology, based on the plurality of meteorological monitoring data (including atmospheric pressure, geopotential height, wind direction and speed, precipitation, etc.), simulated or enhanced weather data is generated. The generative adversarial network generally includes a generator and a discriminator. The generator is responsible for generating new weather data distribution according to the input data (which can be random noise or real data, i.e. input meteorological monitoring data). The discriminator is responsible for judging whether the generated data is real or close to real. By training the generator and the discriminator, a plurality of basic meteorological data distribution sets are generated, which simulate the potential distribution of weather data in the forecast area. After obtaining the plurality of basic meteorological data distribution sets, the plurality of spatio-temporal attention parameter sets obtained by the spatio-temporal attention analysis are used to fuse the basic distribution sets, and finally a plurality of weather data distributions are generated, each of which corresponds to a type of weather data (such as atmospheric pressure distribution, precipitation distribution, etc.) in the forecast area. It not only reflects the change of weather data in space and time, but also considers the correlation and importance between different data. Specifically, according to the spatio-temporal attention parameter set, the data distributions of the same category in different basic meteorological data distribution sets are fused, for example, using weighted average, according to the weight or importance provided by the spatio-temporal attention parameter set, the data distributions of the same category are combined or integrated to obtain a plurality of more comprehensive and accurate weather data distributions, i.e. a plurality of basic meteorological data distribution sets of meteorological parameters (such as atmospheric pressure, wind speed, etc.) can more accurately reflect the distribution and change of weather data in space and time.

[0029] In a possible implementation, step S300 further includes step S310 of collecting a plurality of sample meteorological monitoring data sets and a plurality of sample meteorological data distribution sets according to historical meteorological data monitoring of the plurality of monitoring points and the forecast area. From the historical meteorological data, meteorological monitoring data of a plurality of representative time periods or events and meteorological monitoring points are selected as samples to form a plurality of sample meteorological monitoring data sets, each sample set containing a set of meteorological data collected at a specific time and place, such as temperature, humidity, air pressure, precipitation, and other meteorological elements of the monitoring point in a plurality of time periods; and spatial and temporal distribution analysis is performed on the meteorological data to form a plurality of sample meteorological data distribution sets, which describe the change rules and characteristics of the meteorological data in geographical space and time sequence.

[0030] Step S300 further includes step S320 of constructing a first meteorological distribution generative adversarial path based on the first sample meteorological monitoring data set of the first monitoring point and the plurality of sample meteorological data distribution sets using a generative adversarial network. The first monitoring point is any one of the plurality of monitoring points in the forecast area, and the first sample meteorological monitoring data set contains meteorological data of the monitoring point in a certain time period in the past. The first sample meteorological monitoring data set of the first monitoring point is used as a representative of real data to train a discriminant model so that it can identify real meteorological data distribution. The generative model will try to generate meteorological data samples as similar as possible to the real data distribution. During the training process, the discriminant model continuously evaluates the data samples generated by the generative model and gives feedback, and the generative model adjusts according to the feedback of the discriminant model to generate more realistic data samples. The process is iterated until the generative model can generate meteorological data samples that the discriminant model cannot distinguish between true and false. Finally, the first meteorological distribution generative adversarial path is obtained.

[0031] Step S300 further includes step S330 of inputting the first meteorological monitoring data of the first monitoring point into the first meteorological distribution generative adversarial path for adversarial generation to obtain a first basic meteorological data distribution set. The first meteorological monitoring data of the first monitoring point is input into the first meteorological distribution generative adversarial path for adversarial generation to obtain a new set of meteorological data samples, i.e., the first basic meteorological data distribution set, which may contain similar meteorological characteristics as the original monitoring data, but may also introduce some new or changed meteorological characteristics due to the characteristics of the generative adversarial network.

[0032] Step S300 further includes step S340 of performing meteorological data distribution adversarial generation of the forecast area according to a plurality of other meteorological monitoring data to obtain a plurality of basic meteorological data distribution sets.

[0033] In a possible implementation, step S320 further includes step S321 of constructing, based on a generative adversarial network, a first generator and a first discriminator in the first weather distribution generative adversarial path. Two main neural network components are defined and trained in this generative adversarial network structure, which together constitute a model for simulating and generating weather data, including the first generator and the first discriminator. Specifically, the first generator learns the distribution of real weather data and generates new weather data samples according to the learned distribution. The training goal of the generator is to make the generated data samples as realistic as possible to deceive the discriminator and make the discriminator unable to distinguish between the generated data and the real data. The first discriminator is a binary classifier, and its task is to determine whether the input data sample is from the real weather data distribution or is false data generated by the generator. The training goal of the discriminator is to accurately determine the authenticity of the data, that is, to distinguish between real data and generated data.

[0034] Step S320 further includes step S322 of inputting the first sample weather monitoring data set into the first generator for generation training of the weather data distribution set, inputting the plurality of sample weather data distribution sets into the first discriminator for discrimination training, and training and updating network parameters of the first weather distribution generative adversarial path. In the training process, the two networks are in mutual confrontation. The discriminator will constantly try to identify the false data generated by the generator, while the generator will constantly try to generate more realistic data to deceive the discriminator. Through this competition and confrontation, the performance of the two networks will gradually improve, and finally the generator can generate simulated data very close to the real weather data distribution. Step S323 is also included, which is until the training meets the convergence requirement, and the first weather distribution generative adversarial path is obtained.

[0035] Step S400, based on ensemble learning, constructing a temperature ensemble prediction model for temperature distribution prediction, wherein the temperature ensemble prediction model comprises a plurality of temperature prediction channels, and each temperature prediction channel comprises a plurality of temperature prediction branches. The temperature ensemble prediction model is a general framework model integrating a plurality of temperature prediction channels, which improves the overall prediction performance by integrating the prediction results of a plurality of models; the temperature ensemble prediction model contains a plurality of prediction channels, each of which can use different data sets, feature sets, model architectures or parameter settings, for example, the plurality of temperature prediction channels can be parallel or serial, and each channel will independently generate a prediction result of the temperature distribution; within each temperature prediction channel, a plurality of temperature prediction branches are included, which can be based on different machine learning models or physical models, and each branch will generate a specific temperature prediction result according to the input features and data sets, and the temperature prediction results of these temperature prediction branches will be integrated (such as weighted average) within the channel to produce the final prediction result of the channel. Overall, the temperature ensemble prediction model realizes comprehensive and multi-angle prediction of temperature distribution through the construction of multiple temperature prediction channels and temperature prediction branches, and each temperature prediction branch of each temperature prediction channel is an independent prediction of temperature change. Through ensemble learning, these prediction results are combined to obtain a more comprehensive and accurate prediction result.

[0036] In a possible implementation, step S400 further comprises step S410 of collecting a plurality of sample meteorological data distribution sets of a plurality of meteorological data and collecting a sample temperature distribution set according to the temperature prediction history data of the forecast area. The temperature prediction history data refers to the temperature prediction records of the forecast area in the past period of time, for example, from meteorological observation stations or meteorological model monitoring results, etc. For each type of meteorological data (such as temperature), a representative time period or event is selected from the temperature prediction history data as a sample to form a plurality of sample meteorological data distribution sets, each sample set contains a set of meteorological data collected at a specific time and place, for example, for temperature data, representative temperature data of different seasons such as spring, summer, autumn and winter is selected as a sample to form a plurality of sample distribution sets of temperature data; among the plurality of meteorological data, special attention is paid to temperature data, and its sample distribution set is collected, which contains a plurality of temperature data samples of different time periods and different places, reflecting the distribution rules and characteristics of temperature in time and space, for example, historical weather data of a certain area within a certain period of time is obtained, including maximum temperature, minimum temperature, weather conditions, etc., which is used to construct a sample meteorological data distribution set, and for temperature data, further extract average temperature, extreme temperature, etc. information of a specific time period to form a sample temperature distribution set.

[0037] Step S400 further includes step S420, using the first sample meteorological data distribution set and the sample air temperature distribution set of the first type of meteorological data, based on ensemble learning, constructing a plurality of first air temperature prediction branches to obtain a first air temperature prediction channel. Different groups of first air temperature prediction construction data are collected multiple times in the first sample meteorological data distribution set and the sample air temperature distribution set, and the plurality of first air temperature prediction branches are constructed and trained. The first type of meteorological data is any one of the plurality of types of meteorological data, the first sample meteorological data distribution set is a sample set extracted from historical data related to the first type of meteorological data, and contains meteorological data under different time, different place and different weather conditions; in the first sample meteorological data distribution set and the sample air temperature distribution set, different groups of first air temperature prediction construction data are collected multiple times, that is, multiple groups of data are selected randomly or according to certain rules from the two sets, each group of data contains all features (such as temperature, humidity, wind speed, etc.) and target variables (i.e. air temperature) required for constructing air temperature prediction model, based on the first air temperature prediction construction data collected each time, a plurality of base learners (such as decision tree model or neural network) are trained to form an air temperature prediction branch, and a plurality of first air temperature prediction branches are combined to form a first air temperature prediction channel.

[0038] Step S400 further includes step S430, continuing to use a plurality of sample meteorological data distribution sets and a plurality of sample air temperature distribution sets of other types of meteorological data, based on ensemble learning, respectively constructing a plurality of air temperature prediction branches to obtain a plurality of air temperature prediction channels. Based on machine learning algorithms (such as neural network, decision tree, etc.), a model is constructed, and a plurality of sample meteorological data distribution sets and a plurality of sample air temperature distribution sets of each type of meteorological data are used for training to construct a plurality of air temperature prediction branches to capture the air temperature change law under different weather conditions. Specifically, the plurality of sample meteorological data distribution sets and the plurality of sample air temperature distribution sets are divided into training set and validation set, and the training set is used to train the model and the validation set is used to evaluate the performance of the model. After training, a plurality of air temperature prediction branches are obtained, and finally a plurality of air temperature prediction channels are obtained. Each air temperature prediction channel can output corresponding air temperature prediction results according to input meteorological data, reflecting the influence of different weather conditions on air temperature.

[0039] Step S500, based on the historical temperature forecast data of the forecast area, analyzing a plurality of key variable parameters of the plurality of meteorological data, combining the plurality of spatio-temporal attention parameter sets, and calculating a plurality of temperature forecast fusion coefficients. The historical temperature forecast data of the forecast area includes information such as past temperature changes, trends, periodicity, etc. Specifically, the historical temperature forecast data is analyzed to extract features related to temperature changes, and the plurality of meteorological data is analyzed to identify relevant variable parameters of meteorological changes. The correlation between meteorological data changes and temperature changes is analyzed to obtain a plurality of key variable parameters of meteorological data. The spatio-temporal attention parameter set is combined to consider the importance and relevance of different meteorological data at different times and spatial locations, i.e., the average spatio-temporal attention parameter of each type of meteorological data at multiple monitoring points is combined to obtain a plurality of temperature forecast fusion coefficients, which represent the contribution degree or weight of different meteorological data or different prediction models in temperature prediction. The size of the temperature forecast fusion coefficient reflects the reliability of the current predicted temperature of the corresponding type of meteorological data.

[0040] In one possible implementation, step S500 further includes step S510, based on the historical temperature forecast data of the forecast area, collecting a plurality of meteorological data sequences of the plurality of meteorological data, and a temperature data sequence. The historical temperature forecast data refers to the temperature forecast data for the forecast area in the past period of time, which can come from weather forecast records or historical output records of meteorological models. The plurality of meteorological data refers to a plurality of different types of meteorological data collected at the same time (such as temperature, humidity, air pressure, precipitation, etc.). Each meteorological data sequence represents the change of one type of meteorological data over time. For example, the temperature data sequence can record the maximum temperature or average temperature of each day in the past few years; the humidity data sequence can record the humidity value of each day in the same time period; the temperature data sequence is a meteorological data sequence specifically for temperature, which records the change of temperature over time, which can be actual observed temperature data (such as observation data of a meteorological station) or predicted values in the historical temperature forecast data.

[0041] The step S500 further comprises a step S520 of performing variable correlation analysis on the plurality of meteorological data sequences and the air temperature data sequence to obtain a plurality of key variable parameters. The variable correlation analysis on the plurality of meteorological data sequences and the air temperature data sequence refers to identifying meteorological variables that have a significant impact on the air temperature by using correlation analysis (such as Pearson correlation coefficient, Spearman correlation coefficient, etc.), that is, obtaining a plurality of key variable parameters. The step S530 further comprises a step of calculating a plurality of average spatio-temporal attention parameters according to the spatio-temporal attention parameters of the plurality of meteorological data in the plurality of spatio-temporal attention parameter sets. After obtaining the spatio-temporal attention parameters of the plurality of meteorological data, in order to further comprehensively consider the importance of all meteorological data at different spatio-temporal positions, a plurality of average spatio-temporal attention parameters are calculated. For example, the spatio-temporal attention parameters of all meteorological data at the same time point are averaged to obtain an average spatio-temporal attention parameter at the time point, the spatio-temporal attention parameters of all meteorological data at the same spatial position are averaged to obtain an average spatio-temporal attention parameter at the spatial position, and then the dimensions of time and space are combined to calculate the average spatio-temporal attention parameter, and finally a plurality of average spatio-temporal attention parameters are obtained.

[0042] The step S500 further comprises a step S540 of performing weighted calculation on the plurality of key variable parameters and the plurality of average spatio-temporal attention parameters to obtain a plurality of air temperature prediction fusion coefficients. The key variable parameter is the correlation size between the corresponding meteorological data and the air temperature. The key variable parameter and the average spatio-temporal attention parameter of each type of meteorological data are weighted to obtain a plurality of air temperature prediction fusion coefficients, which reflect the comprehensive importance of different meteorological data in air temperature prediction and the reliability of the model on them at different spatio-temporal positions, and are used for subsequent construction of an air temperature prediction model for air temperature prediction. By weighting and fusing the prediction results of different meteorological data, a more accurate air temperature prediction is obtained.

[0043] Step S600, according to the plurality of air temperature prediction fusion coefficients, a plurality of prediction ratios are calculated, the plurality of meteorological data distributions are input into the air temperature prediction branches of the corresponding prediction ratios in the plurality of air temperature prediction channels to perform air temperature distribution prediction, a plurality of predicted air temperature distributions are obtained, and the plurality of air temperature prediction fusion coefficients are fused to obtain a predicted air temperature distribution of the prediction area. According to the obtained plurality of air temperature prediction fusion coefficients, the prediction ratio, that is, the corresponding weight, of each air temperature prediction channel and the air temperature prediction branch in the final air temperature prediction is calculated and determined, which represents the importance or contribution of different air temperature prediction channels or air temperature prediction branches in generating the final air temperature distribution prediction. The plurality of meteorological data distributions (including atmospheric pressure, geopotential height, wind direction and speed, precipitation, etc.) are input into the plurality of air temperature prediction channels of the air temperature integrated prediction model. In each air temperature prediction channel, according to the determined prediction ratio, the corresponding air temperature prediction branch is selected to perform air temperature distribution prediction. Each air temperature prediction branch generates a predicted air temperature distribution using the input meteorological data distribution, and a plurality of predicted air temperature distributions are obtained. Then, according to the predicted air temperature distribution generated by each air temperature prediction branch and the corresponding air temperature prediction fusion coefficient, the air temperature prediction results are weighted and fused to obtain the predicted air temperature distribution of the prediction area, which comprehensively considers the contribution of a plurality of prediction channels and branches and improves the prediction accuracy, that is, the prediction results of a plurality of different sources and different methods are integrated to reduce the error of the air temperature prediction result.

[0044] In a possible implementation, step S600 further includes step S610, according to the plurality of air temperature prediction fusion coefficients, the number of air temperature prediction branches in each air temperature prediction channel is combined to obtain a plurality of prediction ratios by integer calculation. Each air temperature prediction fusion coefficient is multiplied by the total number of each corresponding prediction branch, and an integer operation is performed to obtain the prediction ratio of each prediction branch in the final prediction, wherein the integer operation is to ensure that the sum of the ratios of all prediction branches is equal to or close to the total number of prediction branches.

[0045] Step S600 further includes step S620, inputting the plurality of weather data distributions into corresponding prediction proportion temperature prediction branches in a plurality of temperature prediction channels respectively, performing temperature distribution prediction to obtain a plurality of predicted temperature distribution sets, and fusing the plurality of predicted temperature distributions in the plurality of temperature prediction channels. The plurality of weather data distributions are input into a plurality of temperature prediction channels constructed in advance, and then the weather data distributions are input into the temperature prediction branches of corresponding prediction proportions according to the plurality of prediction proportions calculated, that is, for each temperature prediction channel, different temperature prediction branches will receive different proportions of weather data input, each temperature prediction branch processes and analyzes the input weather data to generate a respective temperature prediction result in the form of a temperature distribution, reflecting the possible changes in temperature in a future period of time, that is, a plurality of predicted temperature distribution sets are obtained. Then, in the plurality of temperature prediction channels, the prediction results of the temperature prediction branches are fused, for example, weighted average, to obtain a plurality of predicted temperature distributions by fusing the prediction results of the plurality of temperature prediction branches, which contain the possible changes in temperature in a future period of time, and have high accuracy and stability due to the fusion of the prediction results of the plurality of prediction branches. Step S630 further includes weighting and fusing the plurality of predicted temperature distributions according to the plurality of temperature forecast fusion coefficients to obtain the forecast temperature distribution of the forecast area. The plurality of temperature forecast fusion coefficients are used to weight and fuse the predicted temperature distributions of each temperature prediction branch to obtain the forecast temperature values of the plurality of regions of the forecast area, and finally form the forecast temperature distribution.

[0046] In the foregoing, reference Figure 1 The deep processing method based on multi-modal data according to the embodiments of the present application is described in detail. Next, the deep processing system based on multi-modal data according to the embodiments of the present application will be described with reference to Figure 2 The deep processing system based on multi-modal data according to the embodiments of the present application is described in detail. Next, the deep processing system based on multi-modal data according to the embodiments of the present application will be described with reference to

[0047] The deep processing system based on multi-modal data according to the embodiments of the present application is used to solve the technical problems that the existing weather data processing cannot effectively utilize multi-type weather variable prediction data, cannot accurately capture the spatial distribution characteristics and correlation of weather variables, and further leads to insufficient accuracy, real-time performance and reliability of weather prediction, and achieves the technical effects of improving the accuracy, real-time performance and reliability of weather prediction. The deep processing system based on multi-modal data includes a weather monitoring data acquisition module 10, a spatio-temporal attention parameter set obtaining module 20, a weather data distribution obtaining module 30, a temperature integrated prediction model construction module 40, a temperature forecast fusion coefficient obtaining module 50, and a forecast temperature distribution obtaining module 60.

[0048] The weather monitoring data collection module 10 is configured to collect a plurality of weather monitoring data collected by a plurality of monitoring points, wherein each weather monitoring data comprises a plurality of weather data;

[0049] The spatio-temporal attention parameter set obtaining module 20 is configured to perform spatio-temporal attention analysis on the plurality of weather data according to the monitoring coordinates of the plurality of monitoring points and the timestamps of the plurality of weather monitoring data, and obtain a plurality of spatio-temporal attention parameter sets;

[0050] The weather data distribution obtaining module 30 is configured to perform weather data distribution generative adversarial network on the plurality of weather monitoring data, obtain a plurality of basic weather data distribution sets, fuse the plurality of basic weather data distribution sets according to the plurality of spatio-temporal attention parameter sets, and obtain a plurality of weather data distributions of the plurality of weather data, wherein each weather data distribution comprises a distribution of one type of weather data in the forecast area.

[0051] The temperature integrated prediction model construction module 40 is configured to construct a temperature integrated prediction model for predicting temperature distribution based on integrated learning, wherein the temperature integrated prediction model comprises a plurality of temperature prediction channels, and each temperature prediction channel comprises a plurality of temperature prediction branches.

[0052] The temperature prediction fusion coefficient obtaining module 50 is configured to analyze a plurality of key variable parameters of the plurality of weather data based on historical temperature prediction data of the forecast area, and calculate a plurality of temperature prediction fusion coefficients according to the plurality of spatio-temporal attention parameter sets.

[0053] The predicted temperature distribution obtaining module 60 is configured to calculate a plurality of prediction ratios according to the plurality of temperature prediction fusion coefficients, input the plurality of weather data distributions into temperature prediction branches of the plurality of temperature prediction channels corresponding to the prediction ratios to predict temperature distribution, obtain a plurality of predicted temperature distributions, and fuse the predicted temperature distributions according to the plurality of temperature prediction fusion coefficients to obtain a predicted temperature distribution of the forecast area.

[0054] In the following, the specific configuration of the spatio-temporal attention parameter set obtaining module 20 will be described in detail. The spatio-temporal attention parameter set obtaining module 20 further comprises: obtaining a plurality of monitoring coordinates of the plurality of monitoring points and a plurality of timestamps of the plurality of meteorological monitoring data; obtaining a sample monitoring coordinate set, a sample timestamp set, and a plurality of sample spatio-temporal attention parameter sets according to the historical meteorological monitoring data of the forecast area and the variation amplitude of the plurality of types of meteorological data at different coordinates and different times; training a spatio-temporal attention analyzer using the sample monitoring coordinate set, the sample timestamp set, and the plurality of sample spatio-temporal attention parameter sets; and performing spatio-temporal attention analysis on the plurality of monitoring coordinates and the plurality of timestamps using the spatio-temporal attention analyzer to obtain a plurality of spatio-temporal attention parameter sets.

[0055] In the following, the specific configuration of the meteorological data distribution obtaining module 30 will be described in detail. The meteorological data distribution obtaining module 30 can further comprise: collecting a plurality of sample meteorological monitoring data sets and a plurality of sample meteorological data distribution sets according to historical meteorological data monitoring of the plurality of monitoring points and the forecast area; constructing a first meteorological distribution generative adversarial path based on a generative adversarial network using a first sample meteorological monitoring data set of a first monitoring point and the plurality of sample meteorological data distribution sets; inputting the first meteorological monitoring data of the first monitoring point into the first meteorological distribution generative adversarial path for adversarial generation to obtain a first basic meteorological data distribution set; and performing adversarial generation of meteorological data distribution of the forecast area according to the plurality of meteorological monitoring data to obtain a plurality of basic meteorological data distribution sets.

[0056] In the following, the specific configuration of the meteorological data distribution obtaining module 30 will be described in detail. The meteorological data distribution obtaining module 30 can further comprise: constructing a first generator and a first discriminator within the first meteorological distribution generative adversarial path based on a generative adversarial network; inputting the first sample meteorological monitoring data set into the first generator for generation training of the meteorological data distribution set, and inputting the plurality of sample meteorological data distribution sets into the first discriminator for discriminant training, to update the network parameters of the first meteorological distribution generative adversarial path; and obtaining the first meteorological distribution generative adversarial path until the training meets the convergence requirement.

[0057] Below, the specific configuration of the air temperature integrated prediction model construction module 40 will be described in detail. The air temperature integrated prediction model construction module 40 further comprises: collecting a plurality of sample meteorological data distribution sets of multiple types of meteorological data and a sample air temperature distribution set according to air temperature prediction historical data of the forecast area; using a first sample meteorological data distribution set of a first type of meteorological data and the sample air temperature distribution set, constructing a plurality of first air temperature prediction branches based on ensemble learning to obtain a first air temperature prediction channel, wherein different groups of first air temperature prediction construction data are collected multiple times in the first sample meteorological data distribution set and the sample air temperature distribution set to construct and train the plurality of first air temperature prediction branches; and continuing to use a plurality of sample meteorological data distribution sets of other types of meteorological data and a plurality of sample air temperature distribution sets to respectively construct a plurality of air temperature prediction branches based on ensemble learning to obtain a plurality of air temperature prediction channels.

[0058] Below, the specific configuration of the air temperature prediction fusion coefficient obtaining module 50 will be described in detail. The air temperature prediction fusion coefficient obtaining module 50 further comprises: collecting a plurality of meteorological data sequences of multiple types of meteorological data and an air temperature data sequence based on historical air temperature prediction data of the forecast area; performing variable correlation analysis according to the plurality of meteorological data sequences and the air temperature data sequence to obtain a plurality of key variable parameters; calculating a plurality of average spatiotemporal attention parameters according to the spatiotemporal attention parameters of the multiple types of meteorological data in the plurality of spatiotemporal attention parameter sets; and performing weighted calculation on the plurality of key variable parameters and the plurality of average spatiotemporal attention parameters to obtain a plurality of air temperature prediction fusion coefficients.

[0059] Below, the specific configuration of the predicted air temperature distribution obtaining module 60 will be described in detail. The predicted air temperature distribution obtaining module 60 can further comprise: according to the plurality of air temperature prediction fusion coefficients, integrating the number of air temperature prediction branches in each air temperature prediction channel to obtain a plurality of prediction ratios; inputting the plurality of meteorological data distributions into the air temperature prediction branches of the corresponding prediction ratios in the plurality of air temperature prediction channels to perform air temperature distribution prediction and obtain a plurality of predicted air temperature distribution sets, and fusing the plurality of predicted air temperature distributions in the plurality of air temperature prediction channels to obtain a predicted air temperature distribution of the forecast area.

[0060] The deep processing system based on multi-modal data provided in the embodiments of the present application can execute the deep processing method based on multi-modal data provided in any of the embodiments of the present application, and has the corresponding function modules and beneficial effects of the execution method.

[0061] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, however, any number of different modules can be used and run on the user terminal and / or the server, the various units and modules included are only divided according to the functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific name of each functional unit is only for the convenience of mutual differentiation, and is not used to limit the protection scope of the present application.

[0062] The above detailed description does not constitute a limitation on the protection scope of the present application. Those skilled in the art should understand that various modifications, combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A deep processing method based on multimodal data, characterized in that, The method includes: Multiple meteorological monitoring data points are collected from multiple monitoring points, and each meteorological monitoring data point includes multiple types of meteorological data. Based on the monitoring coordinates of the multiple monitoring points and the timestamps of the multiple meteorological monitoring data, spatiotemporal attention analysis of multiple types of meteorological data is performed to obtain multiple spatiotemporal attention parameter sets; Based on the multiple meteorological monitoring data, adversarial generation of meteorological data distribution in the forecast area is performed to obtain multiple basic meteorological data distribution sets. The multiple basic meteorological data distribution sets are then fused according to the multiple spatiotemporal attention parameter sets to obtain multiple meteorological data distributions of multiple types of meteorological data. Each meteorological data distribution includes the distribution of one type of meteorological data within the forecast area. Based on ensemble learning, an integrated temperature prediction model is constructed to predict temperature distribution. The integrated temperature prediction model includes multiple temperature prediction channels, and each temperature prediction channel includes multiple temperature prediction branches. Based on historical temperature forecast data of the forecast area, multiple key variable parameters of the various types of meteorological data are analyzed, and multiple spatiotemporal attention parameter sets are combined to calculate multiple temperature forecast fusion coefficients. Multiple forecast ratios are calculated based on the multiple temperature forecast fusion coefficients. The multiple meteorological data distributions are input into the temperature forecast branches of the corresponding forecast ratios in the multiple temperature forecast channels to predict the temperature distribution, thereby obtaining multiple predicted temperature distributions. The multiple temperature forecast fusion coefficients are then used to fuse the data to obtain the predicted temperature distribution of the forecast area.

2. The deep processing method based on multimodal data according to claim 1, characterized in that, Based on the monitoring coordinates of the multiple monitoring points and the timestamps of the multiple meteorological monitoring data, spatiotemporal attention analysis of various types of meteorological data is performed to obtain multiple spatiotemporal attention parameter sets, including: Obtain multiple monitoring coordinates of the multiple monitoring points, and multiple timestamps of the multiple meteorological monitoring data; Based on historical meteorological monitoring data of the forecast area, obtain a set of sample monitoring coordinates and a set of sample timestamps, and identify and obtain multiple sets of sample spatiotemporal attention parameters according to the variation range of multiple types of meteorological data at different coordinates and times; A spatiotemporal attention analyzer is trained using the aforementioned set of sample monitoring coordinates, set of sample timestamps, and multiple sets of sample spatiotemporal attention parameters. The spatiotemporal attention analyzer is used to perform spatiotemporal attention analysis on the multiple monitoring coordinates and multiple timestamps to obtain multiple spatiotemporal attention parameter sets.

3. The deep processing method based on multimodal data according to claim 1, characterized in that, Based on the aforementioned multiple meteorological monitoring data, adversarial generation of meteorological data distribution for the forecast area is performed to obtain multiple basic meteorological data distribution sets, including: Based on historical meteorological data from multiple monitoring points and forecast areas, multiple sample meteorological monitoring data sets and multiple sample meteorological data distribution sets were collected. Using the first sample meteorological monitoring data set of the first monitoring point and multiple sample meteorological data distribution sets, a first meteorological distribution generative adversarial path is constructed based on a generative adversarial network; Using the first meteorological distribution to generate an adversarial path, the first meteorological monitoring data of the first monitoring point is input to generate an adversarial path to obtain the first basic meteorological data distribution set; Based on multiple other meteorological monitoring data, the meteorological data distribution of the forecast area is generated to obtain multiple basic meteorological data distribution sets.

4. The deep processing method based on multimodal data according to claim 3, characterized in that, Using the first sample meteorological monitoring data set from the first monitoring point and multiple sample meteorological data distribution sets, a first meteorological distribution generative adversarial path is constructed based on a generative adversarial network, including: Based on generative adversarial networks, a first generator and a first discriminator are constructed within the first meteorological distribution generative adversarial path. The first sample meteorological monitoring data set is input into the first generator to generate and train the meteorological data distribution set. The multiple sample meteorological data distribution sets are used to perform discrimination training in the first discriminator. The network parameters of the first meteorological distribution generation adversarial path are trained and updated. The training continues until the convergence requirement is met, at which point the first weather distribution is obtained to generate an adversarial path.

5. The deep processing method based on multimodal data according to claim 1, characterized in that, Based on ensemble learning, an ensemble temperature prediction model is constructed to predict temperature distribution, including: Based on historical temperature forecast data for the forecast area, multiple sample meteorological data distribution sets of various types of meteorological data are collected, and sample temperature distribution sets are also collected. Using the first sample meteorological data distribution set and sample temperature distribution set of the first type of meteorological data, multiple first temperature prediction branches are constructed based on ensemble learning to obtain the first temperature prediction channel. In this process, different sets of first temperature prediction construction data are collected multiple times from the first sample meteorological data distribution set and sample temperature distribution set, and the multiple first temperature prediction branches are constructed and trained. We continue to use multiple sample meteorological data distribution sets and multiple sample temperature distribution sets of other types of meteorological data. Based on ensemble learning, we construct multiple temperature prediction branches and obtain multiple temperature prediction channels.

6. The deep processing method based on multimodal data according to claim 1, characterized in that, Based on historical temperature forecast data for the forecast area, multiple key variable parameters of the various meteorological data are analyzed, and combined with multiple spatiotemporal attention parameter sets, multiple temperature forecast fusion coefficients are calculated, including: Based on historical temperature forecast data for the forecast area, multiple meteorological data sequences of various types of meteorological data, as well as temperature data sequences, are collected; Based on the multiple meteorological data sequences and the temperature data sequences, a correlation analysis of variables was performed to obtain multiple key variable parameters; Based on the spatiotemporal attention parameters of various meteorological data within the multiple spatiotemporal attention parameter sets, multiple average spatiotemporal attention parameters are calculated and obtained. Multiple temperature forecast fusion coefficients are obtained by weighting the multiple key variable parameters and multiple average spatiotemporal attention parameters.

7. The deep processing method based on multimodal data according to claim 1, characterized in that, Multiple prediction ratios are calculated based on the multiple temperature forecast fusion coefficients. The distribution of these multiple meteorological data is then input into the corresponding prediction ratio's temperature prediction branch within the multiple temperature prediction channels to predict temperature distribution, including: Based on the multiple temperature forecast fusion coefficients and the number of temperature forecast branches in each temperature forecast channel, multiple forecast ratios are obtained by rounding down. The multiple meteorological data distributions are respectively input into the temperature prediction branches of the corresponding prediction proportions in multiple temperature prediction channels to perform temperature distribution prediction, and multiple predicted temperature distribution sets are obtained. These sets are then fused within the multiple temperature prediction channels to obtain multiple predicted temperature distributions. Based on the multiple temperature forecast fusion coefficients, the multiple predicted temperature distributions are weighted and fused to obtain the predicted temperature distribution of the forecast area.

8. A deep processing system based on multimodal data, characterized in that, The system is used to implement the deep processing method based on multimodal data as described in any one of claims 1-7, and the system comprises: A meteorological monitoring data acquisition module is used to collect multiple meteorological monitoring data from multiple monitoring points, wherein each meteorological monitoring data includes multiple types of meteorological data. A spatiotemporal attention parameter set acquisition module is used to perform spatiotemporal attention analysis on multiple types of meteorological data based on the monitoring coordinates of the multiple monitoring points and the timestamps of the multiple meteorological monitoring data, and to obtain multiple spatiotemporal attention parameter sets. The meteorological data distribution acquisition module is used to generate multiple basic meteorological data distribution sets by performing adversarial generation of meteorological data distribution in the forecast area based on the multiple meteorological monitoring data, and to fuse the multiple basic meteorological data distribution sets according to the multiple spatiotemporal attention parameter sets to obtain multiple meteorological data distributions of multiple types of meteorological data, wherein each meteorological data distribution includes the distribution of one type of meteorological data in the forecast area. A temperature integrated prediction model construction module is used to construct a temperature integrated prediction model for predicting temperature distribution based on ensemble learning. The temperature integrated prediction model includes multiple temperature prediction channels, and each temperature prediction channel includes multiple temperature prediction branches. A temperature forecast fusion coefficient acquisition module is used to analyze multiple key variable parameters of the various meteorological data based on historical temperature forecast data of the forecast area, and calculate multiple temperature forecast fusion coefficients by combining the multiple spatiotemporal attention parameter sets. The forecast temperature distribution acquisition module is used to calculate multiple prediction ratios based on the multiple temperature forecast fusion coefficients, input the multiple meteorological data distributions into the temperature prediction branches of the corresponding prediction ratios in the multiple temperature prediction channels to predict the temperature distribution, obtain multiple predicted temperature distributions, and fuse them according to the multiple temperature forecast fusion coefficients to obtain the forecast temperature distribution of the forecast area.

Images