Meteorological environment information crawling and analyzing method based on large language model

Abstract

The invention relates to a meteorological environment information crawling and analyzing method based on a large language model, belongs to the technical field of meteorological data acquisition and analysis, and solves the problems that the efficiency of acquiring meteorological data by depending on manual data search is low, and unstructured data cannot be fully utilized. Comprising the steps of obtaining a meteorological environment information crawler demand, a meteorological environment information analysis demand and a meteorological environment information report generation demand, and constructing a meteorological environment information crawling cue word, a meteorological environment information analysis cue word and a meteorological environment information report generation cue word in a grading manner based on respective preset cue word templates; performing data crawling on the data source website based on the meteorological environment information crawling cue word and a preset meteorological environment information extraction cue word, and performing similarity measurement on a crawling result and an ERA5 data set to obtain meteorological environment information; and generating cue words based on the meteorological environment information analysis cue words and the meteorological environment information report, and performing information analysis on the meteorological environment information to obtain a meteorological environment information analysis report.

Description

Technical Field

[0001] The present invention relates to the technical field of meteorological data acquisition and analysis, and in particular to a method for crawling and analyzing meteorological environmental intelligence based on a large language model. Background Art

[0002] At present, the requirements for obtaining meteorological environmental intelligence are getting higher and higher. Accurate acquisition of meteorological data can provide key meteorological parameters such as wind speed, wind direction, temperature, humidity, etc. in a timely manner, help analyze the laws of weather changes, predict future weather conditions, and thus issue weather forecasts and alarms in a timely manner, improving the accuracy of weather forecasts.

[0003] However, the acquisition of traditional meteorological data mostly relies on manually collecting scattered information on the Internet or manually modifying crawler strategies according to different meteorological websites. These methods are often cumbersome in processing procedures, resulting in slow and incomplete data acquisition. In a rapidly changing meteorological environment, this defect may affect the timeliness and accuracy of decision-making. At the same time, traditional methods have great difficulty in processing unstructured meteorological data. Traditional methods are often unable to cope with these unstructured data and it is difficult to extract valuable information from them.

[0004] In addition, due to the many limitations of traditional methods in processing data, the collected meteorological data are often not fully utilized. A large amount of data is idle or only used for simple statistical analysis. The utilization efficiency of meteorological data is low and its potential value and role cannot be brought into play. Summary of the invention

[0005] In view of the above analysis, an embodiment of the present invention aims to provide a method for crawling and analyzing meteorological environmental intelligence based on a large language model, so as to solve the problem that the existing method of obtaining meteorological data by manual search is inefficient and cannot fully utilize unstructured data.

[0006] The purpose of the present invention is mainly achieved through the following technical solutions:

[0007] The present invention provides a method for crawling and analyzing meteorological environment intelligence based on a large language model, comprising the following steps:

[0008] Obtaining meteorological environment intelligence crawler requirements, meteorological environment intelligence analysis requirements, and meteorological environment intelligence report generation requirements, and hierarchically constructing meteorological environment intelligence crawler prompt words, meteorological environment intelligence analysis prompt words, and meteorological environment intelligence report generation prompt words based on respective preset prompt word templates; wherein, hierarchically constructing prompt words is to generate corresponding prompt words based on the matching degree of each meteorological environment intelligence requirement and the corresponding preset prompt word template;

[0009] Based on the meteorological environment intelligence crawling prompt word and the preset meteorological environment intelligence extraction prompt word, data crawling is performed on the data source website, and similarity measurement is performed between the crawling result and the ERA5 data set to obtain meteorological environment intelligence;

[0010] Based on the meteorological environment intelligence analysis prompt word and the meteorological environment intelligence report generation prompt word, the meteorological environment intelligence is analyzed to obtain a meteorological environment intelligence analysis report.

[0011] Furthermore, the hierarchical construction prompt words include:

[0012] When the preset prompt word template completely includes the corresponding meteorological environment information requirements, the preset prompt word template is used as the corresponding prompt word;

[0013] When the preset prompt word template includes part of the corresponding meteorological environment intelligence requirements, the semi-fuzzy generated prompt word template is used to optimize the preset prompt word template based on the corresponding meteorological environment intelligence requirements to generate the corresponding prompt word;

[0014] When the preset prompt word template does not include the corresponding meteorological environment intelligence demand at all, the fuzzy generated prompt word template is used to generate the meteorological environment intelligence prompt words based on the meteorological environment intelligence demand.

[0015] Furthermore, crawling data from the data source website based on the meteorological environment intelligence crawling prompt word and the preset meteorological environment intelligence extraction prompt word includes:

[0016] Input the meteorological environment intelligence crawling prompt words into the large language model to obtain crawling demand keywords;

[0017] Based on the crawling requirement keywords and preset meteorological environment intelligence extraction prompt words, a large language model is used to crawl meteorological environment intelligence from the source code of the data source website.

[0018] Furthermore, when the meteorological environment intelligence crawling prompt words do not include the data source URL, an intelligent agent is used to search for the data source URL through a search engine based on the crawling requirement keywords.

[0019] Furthermore, the crawling results are measured similarly with the ERA5 dataset to obtain meteorological environment intelligence, including:

[0020] Performing similarity measurement on each meteorological indicator in the crawled result and the corresponding meteorological indicator in the ERA5 data set to obtain a similarity value of the crawled result;

[0021] When the similarity value of the crawled result is greater than or equal to a preset similarity threshold, the crawled result is used as the crawled meteorological environment intelligence;

[0022] When the similarity value of the crawling result is less than a preset similarity threshold, the meteorological environment intelligence extraction prompt word is optimized to generate a new meteorological environment intelligence extraction prompt word, and then based on the crawler requirement keywords and the new meteorological environment intelligence extraction prompt word, a large language model is used to crawl meteorological environment intelligence from the source code of the data source website;

[0023] When the number of optimizations for the meteorological environment intelligence extraction prompt words exceeds a preset number, a new data source URL is set, and based on the crawler requirement keywords and the meteorological environment intelligence extraction prompt words, a large language model is used to crawl meteorological environment intelligence from the source code of the new data source URL.

[0024] Furthermore, the similarity measurement of each meteorological indicator in the crawling result and the corresponding meteorological indicator in the ERA5 data set includes:

[0025] Performing qualitative data similarity measurement on the qualitative data of each meteorological indicator in the crawled result to obtain a qualitative data similarity value;

[0026] Performing quantitative data similarity measurement on the quantitative data of each meteorological indicator in the crawled result to obtain a quantitative data similarity value;

[0027] The similarity value of the qualitative data and the similarity value of the quantitative data are averaged to obtain the similarity value of the crawling result.

[0028] Furthermore, the qualitative data similarity measurement includes:

[0029] The qualitative data of each meteorological indicator in the crawled result is compared with the ERA5 data set and then binarized; where the data is consistent and recorded as 1, and the data is inconsistent and recorded as 0;

[0030] For the binarized data, the following formula is used to calculate the similarity value of the qualitative data of the crawling results:

[0031]

[0032] Among them, p represents the qualitative data similarity value; x k,i represents the i-th qualitative data of the k-th meteorological index; n k represents the number of qualitative data of the kth meteorological indicator; N represents the number of meteorological indicators with qualitative data in the crawled results.

[0033] Furthermore, the quantitative data similarity measurement includes:

[0034] For the quantitative data of each meteorological indicator in the crawled results, the similarity value of each quantitative data is calculated using the following formula:

[0035]

[0036] Among them, q represents the quantitative data similarity value; x l,j represents the value of the jth quantitative data of the lth meteorological index; y l,j represents the value of the jth quantitative data of the lth meteorological indicator in the ERA5 dataset; m l represents the number of quantitative data of the lth meteorological indicator; M represents the number of meteorological indicators with quantitative data in the crawled results.

[0037] Furthermore, after obtaining the meteorological environment information, the method further includes sorting the meteorological environment information to obtain regular meteorological environment information;

[0038] Based on the meteorological environment information of the rule and the meteorological environment information analysis prompt words, the meteorological data is analyzed using a large language model to obtain an analysis result;

[0039] Prompt words are generated based on the analysis results and the meteorological environment intelligence report, and a meteorological environment intelligence analysis report is generated using a large language model.

[0040] Furthermore, the meteorological environment information is sorted to obtain regular meteorological environment information, including:

[0041] Removing irrelevant characters from the meteorological environment information;

[0042] Unify the data format of the meteorological environment information.

[0043] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

[0044] 1. The present invention integrates natural language processing (NLP) and a large language model to achieve efficient and comprehensive collection, intelligent parsing and in-depth analysis of massive, multimodal meteorological data on the Internet, thereby quickly and accurately acquiring meteorological data, deeply analyzing meteorological data and generating reports based on user needs.

[0045] 2. The present invention realizes intelligent meteorological data crawler based on large language model, and utilizes its powerful language understanding and generation ability to automatically crawl the optimal meteorological data, thereby improving the efficiency and quality of data crawling.

[0046] 3. The present invention realizes the hierarchical prompt word generation for meteorological data crawling and analysis, formulates prompt word templates in the whole process of meteorological data crawling and analysis, and can optimize prompt words according to specific user needs or regenerate prompt words according to the requirements.

[0047] 4. The present invention verifies the accuracy of the meteorological crawler by comparing the acquired crawler data with the ERA5 data, thereby realizing the accuracy verification of the meteorological crawler data.

[0048] In the present invention, the above-mentioned technical solutions can also be combined with each other to achieve more preferred combination solutions. Other features and advantages of the present invention will be described in the subsequent description, and some advantages can become obvious from the description, or can be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained through the contents particularly pointed out in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The drawings are only for the purpose of illustrating particular embodiments and are not to be considered limiting of the present invention. Like reference symbols denote like components throughout the drawings.

[0050] Figure 1 A schematic diagram of a flow chart of a method for crawling and analyzing meteorological environment information based on a large language model in an embodiment of the present invention;

[0051] Figure 2 The figure is a flow chart of crawling result similarity measurement in an embodiment of the present invention. DETAILED DESCRIPTION

[0052] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0053] A specific embodiment of the present invention discloses a method for crawling and analyzing meteorological environment information based on a large language model, such as Figure 1 As shown, the steps S1 to S3 are included:

[0054] Step S1, obtain the meteorological environment intelligence crawler requirements, meteorological environment intelligence analysis requirements and meteorological environment intelligence report generation requirements, and based on the respective preset prompt word templates, hierarchically construct meteorological environment intelligence crawler prompt words, meteorological environment intelligence analysis prompt words and meteorological environment intelligence report generation prompt words; wherein, the hierarchical construction of prompt words is to generate corresponding prompt words based on the matching degree of each meteorological environment intelligence requirement and the corresponding preset prompt word template.

[0055] Specifically, the meteorological environment intelligence crawler demand is the meteorological data required by the user to be crawled from the website, including: meteorological data type (such as temperature, humidity, wind speed, wind direction, air pressure, precipitation, UV index, visibility, etc.), region (such as the world, a specified country, region or city) and time (such as a month, a week or a specified period of time) and other information, the content of which meets the needs of specific analysis or prediction. Exemplarily, the meteorological environment intelligence crawler demand obtained is "Please crawl the temperature of Beijing, China in the past week."

[0056] Meteorological environment intelligence crawling prompt words are used to guide the large language model to generate prompt words for the web crawler according to user needs, so that the crawler can locate and crawl the required meteorological data.

[0057] The meteorological environment intelligence analysis requirement is a requirement for in-depth analysis of the extracted meteorological environment intelligence, and the analysis may include trend prediction, anomaly detection, pattern recognition, etc. For example, the acquired meteorological environment intelligence analysis requirement may be "Please predict the weather in Beijing, China in the next week.".

[0058] The meteorological environment intelligence analysis prompt words are used to guide the large language model to generate prompt words for data analysis, so that the large language model can perform complex data analysis and interpretation.

[0059] The meteorological environment intelligence report generation requirement is a requirement for integrating analysis results and generating a report, which may include charts, forecasts, abnormal event descriptions, etc. For example, the obtained meteorological environment intelligence report generation requirement may be "Please generate a weather report for Beijing, China, for the next week.".

[0060] The meteorological environment intelligence report generation prompt words are used to guide the large language model to generate report prompt words so that the large language model can convert the analysis results into easy-to-understand reports.

[0061] Furthermore, the hierarchical construction prompt words include:

[0062] When the preset prompt word template completely includes the corresponding meteorological environment intelligence demand, the preset prompt word template is used as the corresponding prompt word; the meteorological environment intelligence demand refers to the aforementioned meteorological environment intelligence crawler demand, meteorological environment intelligence analysis demand and meteorological environment intelligence report generation demand; the corresponding preset prompt word template refers to the preset meteorological environment intelligence crawling prompt word template, the preset meteorological environment intelligence analysis prompt word template and the preset meteorological environment intelligence report generation prompt word template; the meteorological environment intelligence crawling prompt word template includes information such as the type of data to be crawled, time range, geographical area and data source; the meteorological environment intelligence analysis prompt word template includes information such as analysis objectives, required data types, time range, geographical area, analysis methods (such as trend analysis, anomaly detection, etc.) and result format requirements; the preset meteorological environment intelligence report generation prompt word template includes information such as the structural layout of the generated report, content highlights (such as summary, method, results, conclusion), data visualization requirements, language style and format requirements.

[0063] When the preset prompt word template includes part of the corresponding meteorological environment intelligence requirements, the semi-fuzzy generated prompt word template is used to optimize the preset prompt word template based on the corresponding meteorological environment intelligence requirements to generate the corresponding prompt word;

[0064] When the preset prompt word template does not include the corresponding meteorological environment intelligence demand at all, the fuzzy generated prompt word template is used to generate the meteorological environment intelligence prompt words based on the meteorological environment intelligence demand.

[0065] Specifically, the use of preset prompt word templates can be quickly applied to common needs and reduce the time for generating prompt words. When the user's needs are not in the preset prompt word templates, prompt words can be generated according to specific needs. Providing customized prompt words for different needs and scenarios can ensure that the large language model can understand the task objectives more accurately, thereby improving the accuracy of the output results.

[0066] For example, the preset meteorological environment intelligence crawling prompt word template is: "You are an artificial intelligence engineer in the meteorological field. Please carefully read the specific requirements of the meteorological data crawler below {meteorological data crawler requirements}, and use your in-depth understanding of the meteorological field to generate a set of keywords for the meteorological data crawler.

[0067] Please generate the following types of keywords:

[0068] 1) Keywords related to meteorological data types, including but not limited to temperature, humidity, wind speed, wind direction, precipitation, air pressure, visibility and other meteorological related indicators.

[0069] 2) Region-related keywords, such as countries or regions.

[0070] 3) Time-related keywords, such as 'weather data for the past week', 'historical weather records', 'start date to end date', etc.

[0071] 4) Keywords related to data source. If the user specifies a data source website, the keyword will be output.

[0072] Output in the following format: {Weather data type: [Weather data type related keywords], region: [region related keywords], time: [time related keywords], data source: [data source related keywords]}".

[0073] When the obtained meteorological environment intelligence crawler demand is "Please crawl the temperature of Beijing, China in the past week.", the keywords in the demand are included in the preset meteorological environment intelligence crawler prompt word template. Therefore, the obtained meteorological environment intelligence crawler demand is input into the "{meteorological data crawler demand}" of the prompt word template to obtain the meteorological environment intelligence crawler prompt word.

[0074] When the acquired meteorological environment intelligence crawler demand is "Please crawl the ozone concentration in Beijing, China from August 1, 2024 to August 5, 2024.", only a part of the keywords in the demand are in the preset meteorological environment intelligence crawling prompt word template, and the keyword "ozone concentration" that needs to be obtained is not in the preset template. Therefore, it is necessary to use the preset semi-fuzzy generated prompt word template to obtain an input sentence that optimizes the preset meteorological environment intelligence crawling prompt word template based on the meteorological environment intelligence crawler demand, input a large language model to optimize the preset meteorological environment intelligence crawling prompt word template, and obtain a meteorological environment intelligence crawling prompt word that includes the acquired meteorological environment intelligence crawler demand. Exemplarily, the semi-fuzzy generated prompt word template can be "You are a large language model prompt word engineer, please optimize the prompt word template: {Meteorological data crawling and analysis prompt word template} according to user needs: {user specific needs}.".

[0075] When the acquired meteorological environment intelligence crawler demand is "Please collect different types of meteorological information from multiple public meteorological sites and organize the data by geographical area.", the keywords in the demand do not appear in the preset meteorological environment intelligence crawling prompt word template. At this time, the preset fuzzy generated prompt word template is used to obtain the input sentence for regenerating the prompt word based on the meteorological environment intelligence crawler demand, and the large language model is input to obtain the meteorological environment intelligence crawling prompt word including the acquired meteorological environment intelligence crawler demand. Exemplarily, the fuzzy generated prompt word template can be "You are a large language model prompt word engineer. Please design a clear and detailed prompt word based on user needs: {user specific needs}.".

[0076] Specifically, the fuzzy generated meteorological environment intelligence crawling prompt words can be "You are an artificial intelligence engineer in the meteorological field. Please generate a set of keywords for meteorological data crawler according to the following fuzzy requirement description {meteorological data crawler requirements}.

[0077] 1) Keywords related to meteorological data types, including commonly used meteorological related indicators.

[0078] 2) Geographically related keywords, covering a wide geographical area.

[0079] 3) Time-related keywords, covering different time spans.

[0080] 4) Keywords related to data sources, covering different data providers.

[0081] Output in the following format: {Weather data type: [Weather data type related keywords], region: [region related keywords], time: [time related keywords], data source: [data source related keywords]}".

[0082] The preset meteorological environment intelligence analysis prompt word template can be: "You are an artificial intelligence engineer in the meteorological field. Please design an algorithm that can combine the following input data to predict the trend of the meteorological environment in the future, and identify potential abnormal meteorological phenomena such as extreme weather events, sudden climate change, etc. by comparing the differences between current data and historical data, as well as other analysis needs required by users: {meteorological data analysis needs}.

[0083] 1. Input Data

[0084] 1) Current meteorological data: meteorological parameters such as temperature, humidity, air pressure, wind speed, wind direction, precipitation, etc., obtained in real time through crawlers, covering specific geographical areas and time periods;

[0085] 2) Historical meteorological data: meteorological records from past years that have been collected and organized, containing the same parameters as current meteorological data, for comparison and analysis with current data.

[0086] 2. Task Requirements

[0087] 1) Trend forecast:

[0088] ① Using the statistical characteristics and patterns of current and historical meteorological data, establish mathematical models (such as time series analysis, machine learning algorithms, etc.) to predict the trend of meteorological changes in the future (such as the next week or month);

[0089] ② The forecast results should include the forecast values ​​of each meteorological parameter and its confidence interval in order to evaluate the accuracy of the forecast;

[0090] ③ When making trend forecasts, the seasonality, periodicity, nonlinearity and other characteristics of meteorological data should be considered and appropriate mathematical models and parameters should be selected.

[0091] 2) Difference comparison and anomaly identification:

[0092] ① Compare the current meteorological data with the historical data of the same period, and identify meteorological phenomena that deviate significantly from the historical average level or normal fluctuation range;

[0093] ② Use statistical tests (such as Z test, t test) or anomaly detection algorithms (such as DBSCAN, etc.) to automatically identify and mark potential abnormal meteorological phenomena, such as extreme high / low temperatures, heavy rains, droughts, hurricanes, etc.;

[0094] ③ Analyze the characteristics, impact scope and possible causes of abnormal phenomena to provide a scientific basis for further early warning and response measures;

[0095] ④ In the process of anomaly identification, the anomaly threshold and detection algorithm should be set reasonably to avoid false positives and missed positives.

[0096] 3. Output results

[0097] 1) Future meteorological trends: including the forecast values, forecast time range, confidence interval or probability distribution of each meteorological parameter;

[0098] 2) Abnormal meteorological phenomenon identification information: List the identified abnormal meteorological phenomena, occurrence time, location, impact range, possible causes, suggestions and response measures;

[0099] 3) Data comparison and analysis information: summarize the differences between current data and historical data, and analyze the changing trends and periodic laws of the meteorological environment.

[0100] Please design a detailed implementation plan based on the above requirements, including data preprocessing, model selection, algorithm implementation, result verification and other steps, and provide the required output results. "

[0101] When the acquired meteorological environment intelligence analysis demand is "Please predict the weather in Beijing, China for the next week.", the keywords in the demand are included in the preset meteorological environment intelligence analysis prompt word template. Therefore, the acquired meteorological environment intelligence analysis demand is input into the "{meteorological data analysis demand}" of the preset meteorological environment intelligence analysis prompt word template to obtain the meteorological environment intelligence analysis prompt word; or, when the meteorological environment intelligence analysis demand is not acquired, the preset meteorological environment intelligence analysis prompt word template can also be used to obtain the meteorological environment intelligence analysis prompt word.

[0102] When the acquired meteorological environment intelligence analysis demand is "Please analyze the patterns of abnormal climate occurrence", there is only a part of the keywords in the demand, namely "abnormal climate". In the preset meteorological environment intelligence analysis prompt word template, therefore, the preset semi-fuzzy generation prompt word template is used to obtain an input sentence for optimizing the preset meteorological environment intelligence analysis prompt word template based on the meteorological environment intelligence analysis demand, and the large language model is input to optimize the preset meteorological environment intelligence analysis prompt word template to obtain the meteorological environment intelligence analysis prompt words including the meteorological environment intelligence analysis demand.

[0103] When the acquired meteorological environment intelligence analysis requirement is "Please develop a meteorological analysis tool that can mine data information based on the latest meteorological observation data and historical data, conduct comparative analysis, and help decision makers make more accurate judgments.", the keywords in the requirement do not appear in the preset meteorological environment intelligence analysis prompt word template, so the preset fuzzy generation prompt word template is used to obtain an input sentence for regenerating the prompt word based on the meteorological environment intelligence analysis requirement, and input into the large language model to obtain the meteorological environment intelligence analysis prompt word including the acquired meteorological environment intelligence analysis requirement. .

[0104] Specifically, the fuzzy generated meteorological environment intelligence analysis prompt words can be: "You are an artificial intelligence engineer in the meteorological field. According to the following fuzzy requirement description {meteorological environment intelligence analysis requirements}, please design an algorithm that can use current and historical meteorological data to analyze and predict future meteorological trends and identify possible abnormal meteorological phenomena.

[0105] 1. Requirement description

[0106] We need a system that can analyze current weather conditions and predict future weather trends. The system should also be able to identify any significant deviations from normal weather phenomena by comparing current data with historical data. In addition, the system needs to provide detailed analysis reports to help users understand the relationship between current weather conditions and historical data.

[0107] 2. Input data

[0108] (1) Real-time meteorological data: including but not limited to temperature, humidity, air pressure, wind speed, wind direction, precipitation, etc., covering specific geographical areas and time periods.

[0109] (2) Historical meteorological records: meteorological records over the years that have been collected and organized, containing the same parameters as real-time data.

[0110] 3. Task requirements

[0111] (1) Trend prediction

[0112] Using current and historical meteorological data, mathematical models (such as time series analysis, machine learning, etc.) are established to predict future meteorological trends. The forecast results should include the predicted values ​​of each meteorological parameter and uncertainty estimates.

[0113] (2) Abnormality Identification

[0114] By comparing current data with historical data, any significant deviation from the norm can be identified. Appropriate statistical methods or algorithms can be applied to automatically detect abnormal meteorological phenomena and provide detailed analysis reports.

[0115] 4. Output results

[0116] Future meteorological trend analysis report, identification of abnormal meteorological phenomena and their characteristic analysis, and data comparison and analysis report.

[0117] Please design a detailed implementation plan based on the above requirements, including data preprocessing, model selection, algorithm implementation, result verification and other steps, and provide corresponding output results. ".

[0118] For example, the preset meteorological environment intelligence report generation prompt template is: "You are an expert in the meteorological field. Please generate a meteorological report according to the user's report requirements {meteorological data report generation requirements} and the following requirements

[0119] 1. Input data:

[0120] 1) Future weather trends: including the forecast values, forecast time range, confidence interval or probability distribution of each weather parameter.

[0121] 2) Identification information of abnormal meteorological phenomena: List the identified abnormal meteorological phenomena, time of occurrence, location, scope of impact, possible causes, suggestions and response measures.

[0122] 3) Data comparison and analysis information: summarize the differences between current data and historical data, and analyze the changing trends and periodic laws of the meteorological environment.

[0123] 2. Report content requirements:

[0124] 1) Cover: Contains the report title (such as "Analysis report on meteorological trends and abnormal phenomena in a certain area in the next week"), compilation date and other information;

[0125] 2) Abstract: A brief overview of future meteorological trends, major anomalies, and the main findings of data comparison and analysis;

[0126] 3) Forecast of future weather trends: Detailed description of the forecast results of various meteorological parameters, including charts showing forecast values ​​and confidence intervals, and analysis of possible weather changes;

[0127] 4) Identification of abnormal meteorological phenomena: List the identified abnormal phenomena in detail, mark their geographical locations using maps or icons, and analyze the possible causes of the abnormal phenomena and their impact on the local society, economy, environment, etc.;

[0128] 5) Data comparison and analysis: Compare current data with historical data through charts and analyze the changing trend of meteorological environment, especially any significant deviation or periodic change;

[0129] 6) Conclusion and recommendations: Summarize the main findings of the report and propose countermeasures or recommendations for future meteorological trends and abnormal phenomena;

[0130] 7) Appendix: Contains supplementary information such as data sources, prediction model description, references, etc.

[0131] 3. Report style: The report content should be accurate, objective, and comprehensive, the language should be clear and easy to understand, and the charts should be clear and beautiful, so that non-professionals can understand it easily.

[0132] Please generate a complete weather report according to the above requirements."

[0133] When the acquired meteorological environment intelligence report generation demand is "Please generate a weather report for Beijing, China for the next week.", the keywords in the demand are included in the preset meteorological environment intelligence report generation prompt word template. Therefore, the acquired meteorological environment intelligence report generation demand is input into the "{meteorological data report generation demand}" of the prompt word template to obtain the meteorological environment intelligence report generation prompt word; or, when the meteorological environment intelligence report generation demand is not acquired, the preset meteorological environment intelligence report generation prompt word template is also used to obtain the meteorological environment intelligence report generation prompt word.

[0134] When the meteorological environment intelligence report generation requirement obtained is "Please add an introduction to the report. The introduction needs to describe the purpose of the report, background information and the data source used." Only a part of the keywords in the requirement are in the preset meteorological environment intelligence report generation prompt word template. Therefore, the preset semi-fuzzy generation prompt word template is used to obtain an input sentence for optimizing the preset meteorological environment intelligence report generation prompt word template based on the meteorological environment intelligence report generation requirement. The large language model is input to optimize the preset meteorological environment intelligence report generation prompt word template to obtain the meteorological environment intelligence report generation prompt words including the meteorological environment intelligence report generation requirement.

[0135] When the meteorological environment intelligence report generation requirement obtained is "Please generate a detailed meteorological analysis report. This report needs to integrate the latest meteorological observation data and historical data, display the results of data comparison and analysis and put forward relevant suggestions. The report should help decision makers better understand the current meteorological conditions and its relationship with historical data, and prepare for possible changes in the future.", the keywords in the requirement do not appear in the preset meteorological environment intelligence analysis prompt word template. Therefore, the preset fuzzy generation prompt word template is used to regenerate the input sentence of the prompt word based on the meteorological environment intelligence report generation requirement, and the large language model is input to obtain the meteorological environment intelligence report generation prompt word including the meteorological environment intelligence report generation requirement.

[0136] Specifically, the fuzzy generated meteorological environment intelligence report generation prompt can be: "You are an expert in the meteorological field. Please generate a detailed meteorological analysis report according to the following requirement description {meteorological environment intelligence report generation requirement}.

[0137] 1. Report outline

[0138] (1) Introduction

[0139] Briefly describe the purpose of the report, background information, and data sources used.

[0140] (2) Current weather conditions

[0141] Describe the key indicators of the current meteorological data, such as temperature, humidity, wind speed, etc., and provide a visual display of the current meteorological data so that you can intuitively understand the current conditions.

[0142] (3) Data comparison and analysis

[0143] Compare current meteorological data with historical data to reveal any important trends, use charts to show historical trends in the data, and discuss possible cyclical changes, and analyze the differences between current and historical data and their significance.

[0144] (4) Analysis results

[0145] Summarize the main findings of the data comparison and analysis, including any significant deviations or periodic patterns, and explore the impact of these changes on current and future meteorological conditions.

[0146] (5) Suggestions

[0147] Based on the analysis results, specific recommendations are made to help decision makers take action and propose prevention and response measures for future meteorological trends.

[0148] (6) Conclusion

[0149] Summarize the report's key findings and recommendations, emphasizing their importance and urgency.

[0150] (7) Supplementary information

[0151] Include additional information such as data sources, analysis methods, references, etc.

[0152] 2. Report style

[0153] The content should be accurate, objective, and comprehensive, the language should be clear and easy to understand, suitable for non-professionals to read, and the charts should be clear, beautiful, and easy to understand.

[0154] Please generate a detailed meteorological analysis report based on the above requirements."

[0155] Step S2: crawling the data source website based on the meteorological environment intelligence crawling prompt word and the preset meteorological environment intelligence extraction prompt word, and measuring the similarity between the crawling result and the ERA5 data set to obtain meteorological environment intelligence.

[0156] Furthermore, the meteorological environment intelligence crawling prompt words are input into a large language model to obtain crawling demand keywords.

[0157] Specifically, the keywords and data source URLs of the meteorological environment intelligence in the meteorological environment intelligence crawling prompt words are extracted through the large language model and output in a fixed format. The crawling demand keywords obtained are based on the specific needs of the acquisition, reflecting the information data that is desired to be obtained from the input data source URL.

[0158] Exemplarily, the large language model may be ChatGPT, BERT, LlaMA, etc., which is not limited in this embodiment.

[0159] Furthermore, when the meteorological environment intelligence crawling prompt words do not include the data source URL, an intelligent agent is used to search for the data source URL through a search engine based on the crawling requirement keywords.

[0160] Specifically, specifying the data source URL can ensure that the crawler obtains data from reliable and authoritative sources, reducing the risk of obtaining erroneous information from unreliable or low-quality websites, and a clear data source URL can reduce the search scope of the crawler and improve the efficiency of data crawling. The crawler can directly access the specified URL without having to search extensively on the Internet. Therefore, when the data source URL is not included in the obtained meteorological environment intelligence crawling prompt words, an intelligent agent is used to obtain a list of data source URLs based on the crawling requirement keywords. Exemplarily, the intelligent agent can use Gnomic intelligent agent, Baidu Wenxin intelligent agent, etc., which is not specified in this embodiment.

[0161] Furthermore, based on the crawling requirement keywords and preset meteorological environment intelligence extraction prompt words, a large language model is used to crawl meteorological environment intelligence from the source code of the data source website.

[0162] Specifically, the intelligent agent is used again to obtain the HTML source code of all web pages from the above-mentioned data source URL, and the crawling requirement keywords and the HTML source code of the web pages are input into the preset meteorological environment intelligence extraction prompt words, and then input into the large language model to obtain the meteorological environment intelligence included in the crawling requirement keywords.

[0163] For example, the preset meteorological environment intelligence extraction prompt is "You are an artificial intelligence engineer in the meteorological field. Please read the following meteorological data crawler requirement keywords {crawling requirement keywords} carefully, use your in-depth understanding of the meteorological field to directly extract meteorological data from the webpage HTML source code {webpage HTML source code}, and output it in JSON format."

[0164] Furthermore, the crawling results are measured for similarity with the ERA5 dataset to obtain meteorological environmental intelligence.

[0165] Specifically, the ERA5 dataset is the fifth-generation reanalysis dataset provided by the European Centre for Medium-Range Weather Forecasts (ECMWF). It is a combination of observational data and data generated by numerical weather forecast model inversion, and is real-time and representative. The crawled data can be compared with the ERA5 dataset to measure the accuracy of the meteorological crawled data.

[0166] Furthermore, the crawling results are measured similarly with the ERA5 dataset to obtain meteorological environment intelligence, such as Figure 2 As shown, it includes steps S201 to S204:

[0167] Step S201: perform similarity measurement on each meteorological indicator in the crawled result and the corresponding meteorological indicator in the ERA5 data set to obtain a similarity value of the crawled result.

[0168] Specifically, the meteorological indicators in the crawling results include but are not limited to temperature, humidity, weather conditions, wind direction, etc. Comparing them with the data of the corresponding time and region in the ERA5 dataset can verify whether the obtained meteorological indicators are accurate.

[0169] Furthermore, the similarity measurement includes:

[0170] The qualitative data of each meteorological indicator in the crawling result is subjected to a qualitative data similarity measurement to obtain a qualitative data similarity value.

[0171] Specifically, qualitative data is descriptive meteorological data, such as weather conditions such as sunny, cloudy, overcast, rainy, snowy, etc., wind direction such as east wind, southwest wind, northeast wind, etc. This type of data does not exist in numerical form.

[0172] Furthermore, the qualitative data of each meteorological indicator in the crawled result is compared with the ERA5 data set and then binarized; wherein, consistent data is recorded as 1, and inconsistent data is recorded as 0.

[0173] Specifically, since qualitative data does not exist in numerical form, when the acquired data is consistent with the corresponding data in the ERA5 dataset, the data similarity is set to 1; when the acquired data is inconsistent with the corresponding data in the ERA5 dataset, the data similarity is set to 0.

[0174] Furthermore, for the binarized data, the following formula is used to calculate the similarity value of the qualitative data of the crawling results:

[0175]

[0176] Among them, p represents the qualitative data similarity value; x k,i represents the i-th qualitative data of the k-th meteorological index; n k represents the number of qualitative data of the kth meteorological indicator; N represents the number of meteorological indicators with qualitative data in the crawled results.

[0177] Specifically, the number of meteorological indicators of qualitative data in the crawled results indicates the number of meteorological indicators represented by qualitative data in the crawled results. It should be noted that the higher the similarity value of qualitative data in the crawled results, the higher the accuracy of the crawled results.

[0178] Furthermore, the quantitative data of each meteorological indicator in the crawled result is measured for quantitative data similarity using the following formula to obtain a quantitative data similarity value.

[0179]

[0180] Among them, q represents the quantitative data similarity value; x l,j represents the value of the jth quantitative data of the lth meteorological index; y l,j represents the value of the jth quantitative data of the lth meteorological indicator in the ERA5 dataset; m l represents the number of quantitative data of the lth meteorological indicator; M represents the number of meteorological indicators with quantitative data in the crawled results.

[0181] Specifically, the number of meteorological indicators of quantitative data in the crawled results indicates the number of meteorological indicators represented by quantitative data in the crawled results. It should be noted that the higher the similarity value of the quantitative data of the crawled results, the higher the accuracy of the crawled results.

[0182] Furthermore, the qualitative data similarity value and the quantitative data similarity value are averaged to obtain the similarity value of the crawling result.

[0183] Exemplarily, the crawling results are shown in Table 1.

[0184] Table 1 Meteorological environment information crawling results

[0185]

[0186] Among them, temperature and humidity are quantitative data, and their similarity value calculation results are:

[0187]

[0188] Wind direction and air quality are qualitative data, and their similarity value calculation results are:

[0189]

[0190] The final similarity value of the crawling result is P = (p + q) / 2 = (0.996315 + 0.5) / 2 = 0.748158.

[0191] Step S202: When the similarity value of the crawled result is greater than or equal to a preset similarity threshold, the crawled result is used as crawled meteorological environment intelligence.

[0192] Specifically, when the calculated similarity value of the crawled result is greater than or equal to a preset similarity threshold, it means that the crawled result is authentic and can be used as meteorological environment intelligence for subsequent analysis. For example, the similarity threshold can be set to 0.7.

[0193] Step S203: When the similarity value of the crawling result is less than a preset similarity threshold, the meteorological environment intelligence extraction prompt word is optimized, and after generating a new meteorological environment intelligence extraction prompt word, based on the crawler requirement keywords and the new meteorological environment intelligence extraction prompt word, a large language model is used to crawl meteorological environment intelligence from the source code of the data source URL.

[0194] Specifically, when the calculated similarity value of the crawling result is less than the preset similarity threshold, it means that the crawled result at this time is quite different from the real result and is not accurate and reliable enough. At this time, in order to improve the accuracy of data extraction, the large language model is used to optimize the meteorological environment intelligence extraction prompt words and generate new meteorological environment intelligence extraction prompt words. Exemplarily, the large language prompt words for optimizing the meteorological environment intelligence extraction prompt words can be "You are a large language model prompt word engineer, please optimize the prompt words {meteorological data extraction prompt words} so that the large language model can more accurately extract meteorological data according to the meteorological data crawler demand keywords {crawling demand keywords}."

[0195] For the newly generated meteorological and environmental intelligence extraction prompt words and the crawling requirement keywords, the large language model is used again to crawl the meteorological and environmental intelligence from the source code of the data source URL, and the re-crawled results are measured for similarity with the ERA5 data set. If the similarity value obtained is greater than or equal to the preset similarity threshold, the result crawled this time can be used as meteorological and environmental intelligence, and the crawling is ended; if the similarity value obtained this time is still less than the preset similarity threshold, it is necessary to optimize the meteorological and environmental intelligence extraction prompt words again and then crawl the data.

[0196] Step S204: When the number of optimizations for the meteorological environment intelligence extraction prompt words exceeds a preset number, a new data source URL is set, and based on the crawler requirement keywords and the meteorological environment intelligence extraction prompt words, a large language model is used to crawl meteorological environment intelligence from the source code of the new data source URL.

[0197] Specifically, when the number of optimizations for extracting prompt words for meteorological environment intelligence reaches the upper limit and the similarity value of the crawled results still does not meet the requirement, it means that the meteorological environment intelligence data provided by the current data source website is inaccurate. In order to improve the quality and accuracy of the data, it is necessary to reselect the data source website, crawl the meteorological environment intelligence again, and measure the similarity of the crawled results.

[0198] Step S3: based on the meteorological environment intelligence analysis prompt words and the meteorological environment intelligence report generating prompt words, the meteorological environment intelligence is analyzed to obtain a meteorological environment intelligence analysis report.

[0199] Furthermore, after obtaining the meteorological environment information, the method also includes sorting the data of the meteorological environment information to obtain regular meteorological environment information.

[0200] Specifically, for the obtained meteorological environment information, meaningless characters are filtered out and spelling errors are corrected to improve the quality and usability of the data. In the subsequent processing process, regular meteorological environment information can improve the accuracy and efficiency of analysis.

[0201] Furthermore, the data sorting includes removing irrelevant characters in the meteorological environment information and unifying the data format of the meteorological environment information.

[0202] Specifically, the irrelevant characters include but are not limited to HTML tags (such as , , ), script code, special symbols (such as #, *, &, %, $, @, !, etc.), extra spaces or control characters, comments in source code files, etc. For example, the OpenRefine tool can be used to remove irrelevant characters and perform data format conversion and unification.

[0203] Furthermore, based on the meteorological environment information of the rule and the meteorological environment information analysis prompt words, the meteorological data is analyzed using a large language model to obtain an analysis result.

[0204] Specifically, the meteorological environment information of the rule and the meteorological environment information analysis prompt words are input into the large language model together. The large language model analyzes the meteorological environment information of the rule based on the guidance of the meteorological environment information analysis prompt words to obtain the content that needs to be output in the prompt words.

[0205] Furthermore, prompt words are generated based on the analysis results and the meteorological environment intelligence report, and a meteorological environment intelligence analysis report is generated using a large language model.

[0206] Specifically, the analysis result and the prompt words generated by the meteorological environment intelligence report are input into the large language model together. The large language model processes the analysis result based on the guidance of the prompt words generated by the meteorological environment intelligence report to obtain the meteorological environment intelligence analysis report.

[0207] In summary, the method for crawling and analyzing meteorological environment intelligence based on a large language model according to an embodiment of the present invention has the following beneficial effects:

[0208] 1. The present invention integrates natural language processing (NLP) and a large language model to achieve efficient and comprehensive collection, intelligent parsing and in-depth analysis of massive, multimodal meteorological data on the Internet, thereby quickly and accurately acquiring meteorological data, deeply analyzing meteorological data and generating reports based on user needs.

[0209] 2. The present invention realizes intelligent meteorological data crawler based on large language model, and utilizes its powerful language understanding and generation ability to automatically crawl the optimal meteorological data, thereby improving the efficiency and quality of data crawling.

[0210] 3. The present invention realizes the hierarchical prompt word generation for meteorological data crawling and analysis, formulates prompt word templates in the whole process of meteorological data crawling and analysis, and can optimize prompt words according to specific user needs or regenerate prompt words according to the requirements.

[0211] 4. The present invention verifies the accuracy of the meteorological crawler by comparing the acquired crawler data with the ERA5 data, thereby realizing the accuracy verification of the meteorological crawler data.

[0212] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A method for crawling and analyzing meteorological environment information based on a large language model, characterized in that: The steps include: Obtaining meteorological environment intelligence crawler requirements, meteorological environment intelligence analysis requirements, and meteorological environment intelligence report generation requirements, and hierarchically constructing meteorological environment intelligence crawler prompt words, meteorological environment intelligence analysis prompt words, and meteorological environment intelligence report generation prompt words based on respective preset prompt word templates; wherein, hierarchically constructing prompt words is to generate corresponding prompt words based on the matching degree of each meteorological environment intelligence requirement and the corresponding preset prompt word template; Based on the meteorological environment intelligence crawling prompt word and the preset meteorological environment intelligence extraction prompt word, data crawling is performed on the data source website, and similarity measurement is performed between the crawling result and the ERA5 data set to obtain meteorological environment intelligence; Based on the meteorological environment intelligence analysis prompt word and the meteorological environment intelligence report generation prompt word, the meteorological environment intelligence is analyzed to obtain a meteorological environment intelligence analysis report.

2. The method according to claim 1, characterized in that: The hierarchical construction prompt words include: When the preset prompt word template completely includes the corresponding meteorological environment information requirements, the preset prompt word template is used as the corresponding prompt word; When the preset prompt word template includes part of the corresponding meteorological environment intelligence requirements, the semi-fuzzy generated prompt word template is used to optimize the preset prompt word template based on the corresponding meteorological environment intelligence requirements to generate the corresponding prompt word; When the preset prompt word template does not include the corresponding meteorological environment intelligence demand at all, the fuzzy generated prompt word template is used to generate the meteorological environment intelligence prompt words based on the meteorological environment intelligence demand.

3. The method according to claim 2, characterized in that: The method of crawling data from the data source website based on the meteorological environment intelligence crawling prompt word and the preset meteorological environment intelligence extraction prompt word includes: Input the meteorological environment intelligence crawling prompt words into the large language model to obtain crawling demand keywords; Based on the crawling requirement keywords and preset meteorological environment intelligence extraction prompt words, a large language model is used to crawl meteorological environment intelligence from the source code of the data source website.

4. The method according to claim 3, characterized in that: When the meteorological environment intelligence crawling prompt words do not include the data source URL, an intelligent agent is used to search for the data source URL through a search engine based on the crawling requirement keywords.

5. The method according to claim 3, characterized in that: The crawling results are measured similarly with the ERA5 dataset to obtain meteorological environment intelligence, including: Performing similarity measurement on each meteorological indicator in the crawled result and the corresponding meteorological indicator in the ERA5 data set to obtain a similarity value of the crawled result; When the similarity value of the crawled result is greater than or equal to a preset similarity threshold, the crawled result is used as the crawled meteorological environment intelligence; When the similarity value of the crawling result is less than a preset similarity threshold, the meteorological environment intelligence extraction prompt word is optimized to generate a new meteorological environment intelligence extraction prompt word, and then based on the crawler requirement keywords and the new meteorological environment intelligence extraction prompt word, a large language model is used to crawl meteorological environment intelligence from the source code of the data source website; When the number of optimizations for the meteorological environment intelligence extraction prompt words exceeds a preset number, a new data source URL is set, and based on the crawler requirement keywords and the meteorological environment intelligence extraction prompt words, a large language model is used to crawl meteorological environment intelligence from the source code of the new data source URL.

6. The method according to claim 5, characterized in that: The similarity measurement of each meteorological indicator in the crawling result and the corresponding meteorological indicator in the ERA5 data set includes: Performing qualitative data similarity measurement on the qualitative data of each meteorological indicator in the crawled result to obtain a qualitative data similarity value; Performing quantitative data similarity measurement on the quantitative data of each meteorological indicator in the crawled result to obtain a quantitative data similarity value; The qualitative data similarity value and the quantitative data similarity value are averaged to obtain the similarity value of the crawling result.

7. The method according to claim 6, characterized in that: The qualitative data similarity measure includes: The qualitative data of each meteorological indicator in the crawled result is compared with the ERA5 data set and then binarized; where the data is consistent and recorded as 1, and the data is inconsistent and recorded as 0; For the binarized data, the following formula is used to calculate the similarity value of the qualitative data of the crawling results: Among them, p represents the qualitative data similarity value; x k,i represents the i-th qualitative data of the k-th meteorological index; n k represents the number of qualitative data of the kth meteorological indicator; N represents the number of meteorological indicators with qualitative data in the crawled results.

8. The method according to claim 6, characterized in that: The quantitative data similarity measurement includes: For the quantitative data of each meteorological indicator in the crawled results, the similarity value of each quantitative data is calculated using the following formula: Among them, q represents the quantitative data similarity value; x l,j represents the value of the jth quantitative data of the lth meteorological index; y l,j represents the value of the jth quantitative data of the lth meteorological indicator in the ERA5 dataset; m l represents the number of quantitative data of the lth meteorological indicator; M represents the number of meteorological indicators with quantitative data in the crawled results.

9. The method according to claim 1, characterized in that: After obtaining the meteorological environment information, the method further includes sorting the meteorological environment information to obtain regular meteorological environment information; Based on the meteorological environment information of the rule and the meteorological environment information analysis prompt words, the meteorological data is analyzed using a large language model to obtain an analysis result; Prompt words are generated based on the analysis results and the meteorological environment intelligence report, and a meteorological environment intelligence analysis report is generated using a large language model.

10. The method according to claim 9, characterized in that: The step of arranging data of the meteorological environment information to obtain regular meteorological environment information includes: Removing irrelevant characters from the meteorological environment information; Unify the data format of the meteorological environment information.

Images