Big data analysis and prediction system based on cloud computing

Abstract

The invention discloses a big data analysis and prediction system based on cloud computing, which comprises a data acquisition module, a data storage module, a big data analysis module, a prediction result display module and a command and control module, and is characterized in that the data acquisition module is used for acquiring real-time meteorological, traffic flow and road condition related data through a sensor and a meteorological station; the data storage module is used for realizing efficient storage and management of data by using a cloud computing platform and ensuring the security and reliability of the data, and the big data analysis module is used for analyzing and calculating various data and realizing accurate prediction of the influence of snow disaster and snow accumulation conditions on traffic. And the prediction result display module is used for displaying the analysis result on an interface of a command and control center in a visual form, providing reference for a decision maker, and conveniently taking countermeasures in time, and the system has the characteristics of high prediction precision and strong practicability.

Description

Technical Field

[0001] The present invention relates to the field of computer science and technology, and specifically to a big data analysis and prediction system based on cloud computing. Background Art

[0002] With the acceleration of the urbanization process and the increasingly significant impact of climate change, the rare heavy snow in southern cities has become a prominent problem. At present, southern cities lack experience in dealing with heavy snow. Especially during peak hours, such as the morning and evening rush hours, they face the dilemmas of tight time and heavy tasks. Traditional methods for heavy snow prevention and traffic control often lack systematicness and accuracy, and are prone to causing waste of resources and ineffective prevention.

[0003] Some current weather forecasting and traffic control systems are mainly based on historical data and meteorological predictions, and their ability to respond to rare events is relatively weak. Therefore, it is necessary to design a big data analysis and prediction system based on cloud computing with high prediction accuracy and strong practicability. Summary of the Invention

[0004] The purpose of the present invention is to provide a big data analysis and prediction system based on cloud computing to solve the problems raised in the above background art.

[0005] To solve the above technical problems, the present invention provides the following technical solution: A big data analysis and prediction system based on cloud computing, including a data acquisition module, a data storage module, a big data analysis module, a prediction result display module, and a command and control module. The data acquisition module is used to collect real-time meteorological, traffic flow, and road condition related data through sensors and weather stations, and establish a data set. The data storage module is used to use the cloud computing platform to achieve efficient storage and management of data, ensuring the security and reliability of the data. The big data analysis module is used to analyze and calculate various types of data to achieve accurate prediction of the impact of snow disaster and snow accumulation on traffic. The prediction result display module is used to display the analysis results in an intuitive form on the interface of the command and control center, providing reference for decision-makers and facilitating timely adoption of response measures. The command and control module is used to automatically provide targeted snow removal and traffic control suggestions based on the results of big data analysis, and also supports manual intervention to ensure the timeliness and accuracy of decisions.

[0006] According to the above technical solution, the big data analysis module further includes a data preprocessing sub-module, a feature extraction sub-module, and a model analysis sub-module. The data preprocessing sub-module is used to preprocess the collected data. The feature extraction sub-module is used to extract key features from the preprocessed data to establish a data model. The model analysis sub-module is used to analyze and predict the impact of subsequent snowfall on urban roads based on the established data model.

[0007] According to the above technical solution, the prediction result display module further includes a visualization sub-module and a real-time update sub-module. The visualization sub-module is used to display the analysis results in an intuitive form such as charts and maps on the interface of the command and control center for decision-makers to refer to. The real-time update sub-module is used to regularly update the analysis results to ensure that decision-makers obtain the latest prediction information.

[0008] According to the above technical solution, the command and control module further includes a road warning generation sub-module and a command suggestion generation sub-module. The road warning generation sub-module is used to judge the road warning timing according to the analysis results and generate corresponding warning information. The command suggestion generation sub-module is used to generate targeted snow removal and traffic control suggestions according to the big data analysis results.

[0009] According to the above technical solution, the operation method of the big data analysis and prediction system based on cloud computing includes the following steps:

[0010] Step S1: Start the data collection module to collect data related to meteorology, traffic flow, and road conditions in real time;

[0011] Step S2: Store the collected data through the cloud computing platform to ensure the security and reliability of the data;

[0012] Step S3: Start the big data analysis module to perform in-depth analysis and calculation on the stored data using distributed computing and machine learning algorithms;

[0013] Step S4: Display the analysis results in the form of maps and charts on the interface of the command and control center;

[0014] Step S5: According to the displayed results, the system automatically generates targeted snow removal and traffic control suggestions to support manual intervention.

[0015] According to the above technical solution, step S3 further includes the following steps:

[0016] Step S31: Obtain multi-dimensional data related to real-time collected meteorological data, traffic flow data, and road condition data from the data storage module;

[0017] Step S32: Clean the obtained data, process missing values and outliers to ensure data quality, and then perform normalization and standardization preprocessing on the data for subsequent analysis;

[0018] Step S33: Extract key features from various types of data, where the key features include temperature, snowfall, traffic flow, and road humidity, to establish a data model;

[0019] Step S34: Through task division, the data is divided into multiple small tasks so that each node can process them in parallel, making full use of the computing resources in the cluster;

[0020] Step S35: Start the distributed computing framework and distribute the tasks to each node in the cluster for parallel processing;

[0021] Step S36: Integrate the results obtained by each node to form a comprehensive analysis result.

[0022] According to the above technical solution, step S35 further includes the following steps:

[0023] Step S351: Use historical data for model training, considering meteorological, traffic, and road condition factors, so that the model can better predict possible future snow disaster situations;

[0024] Step S352: Use the validation data set to evaluate the trained model to ensure its generalization ability on unknown data;

[0025] Step S353: Apply the trained model to real-time data to generate a real-time prediction of the impact of snow disaster snow accumulation on traffic.

[0026] According to the above technical solution, in step S353, the key features including temperature, snowfall, traffic flow, and road humidity are respectively marked as feature X1, X2, X3, X4, and then the corresponding weight values W1, W2, W3, W4, and a preset bias parameter c are given. The impact index of the current road traffic is calculated by the formula Y = W1X1 + W2X2 + W3X3 + W4X4 + c. In the formula, Y is the impact index of the current road traffic affected by snowfall; this impact index is affected by temperature, snowfall, traffic flow, and road humidity. When the temperature is lower, the greater the feature X1, when the snowfall is greater, the greater the feature X2, when the traffic flow is greater, the greater the probability of traffic accidents caused by snowfall and congestion, and thus the greater the impact index of the current road traffic affected by snowfall. Therefore, the greater the feature value X3, when the road humidity is greater, it is easier to cause road icing, and the greater the feature value X4, where the bias parameter is the compensation value of the impact index after predicting the transmission execution cycle time U according to big data.

[0027] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: In the present invention, by providing a data acquisition module, a data storage module, a big data analysis module, a prediction result display module, and a command and control module, it is possible to quickly analyze and predict rare blizzards during the morning and evening rush hours, and make preparations for snow removal in advance, so as to effectively cope with the characteristic problems of tight time and heavy tasks, improve the urban snow disaster prevention ability, and reduce the impact on traffic and life. Description of the Drawings

[0028] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. They are used together with the embodiments of the present invention to explain the present invention and do not constitute a limitation to the present invention. In the accompanying drawings:

[0029] Figure 1 is a schematic diagram of the system module composition of the present invention. Specific embodiments

[0030] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the scope of protection of the present invention.

[0031] Please refer to Figure 1 , the present invention provides a technical solution: a big data analysis and prediction system based on cloud computing, including a data acquisition module, a data storage module, a big data analysis module, a prediction result display module, and a command and control module. The data acquisition module is used to collect real-time meteorological, traffic flow, and road condition-related data through sensors and weather stations and establish a data set. The data storage module is used to use the cloud computing platform to achieve efficient storage and management of data, ensuring the security and reliability of the data. The big data analysis module is used to analyze and calculate various types of data to achieve accurate prediction of the impact of snow disaster and snow accumulation on traffic. The prediction result display module is used to display the analysis results in an intuitive form on the interface of the command and control center for reference by decision-makers, facilitating timely adoption of response measures. The command and control module is used to automatically provide targeted snow removal and traffic control suggestions based on the results of big data analysis and also support manual intervention to ensure the timeliness and accuracy of decision-making; by setting up a data acquisition module, a data storage module, a big data analysis module, a prediction result display module, and a command and control module, it is possible to quickly analyze and predict rare blizzards during the morning and evening rush hours, make preparations for snow removal in advance, and thus effectively respond to the characteristic problems of tight time and heavy tasks, improve the urban snow disaster prevention ability, and reduce the impact on traffic and life.

[0032] The big data analysis module further includes a data preprocessing sub-module, a feature extraction sub-module, and a model analysis sub-module. The data preprocessing sub-module is used to preprocess the collected data. The feature extraction sub-module is used to extract key features from the preprocessed data to establish a data model. The model analysis sub-module is used to analyze and predict the impact of subsequent snowfall on urban roads based on the established data model.

[0033] The prediction result display module further includes a visualization sub-module and a real-time update sub-module. The visualization sub-module is used to display the analysis results in an intuitive form such as charts and maps on the interface of the command and control center for decision-makers to refer to. The real-time update sub-module is used to regularly update the analysis results to ensure that decision-makers obtain the latest prediction information.

[0034] The command and control module further includes a road warning generation sub-module and a command suggestion generation sub-module. The road warning generation sub-module is used to judge the road warning timing according to the analysis results and generate corresponding warning information. The command suggestion generation sub-module is used to generate targeted snow removal and traffic control suggestions according to the big data analysis results.

[0035] The operation method of the big data analysis and prediction system based on cloud computing includes the following steps:

[0036] Step S1: Start the data collection module to collect data related to meteorology, traffic flow, and road conditions in real time;

[0037] Step S2: Store the collected data through the cloud computing platform to ensure the security and reliability of the data;

[0038] Step S3: Start the big data analysis module and perform in-depth analysis and calculation on the stored data using distributed computing and machine learning algorithms;

[0039] Step S4: Display the analysis results in the form of maps and charts on the interface of the command and control center;

[0040] Step S5: According to the displayed results, the system automatically generates targeted snow removal and traffic control suggestions, supporting manual intervention;

[0041] Through cloud computing and big data analysis technologies, it realizes the accurate prediction and command and control of rare heavy snow situations in southern cities, improves the city's snow disaster prevention ability, and reduces the impact of snow disasters on traffic and life.

[0042] Step S3 further includes the following steps:

[0043] Step S31: Obtain multi-dimensional data related to real-time collected meteorological data, traffic flow data, and road condition data from the data storage module;

[0044] Step S32: Clean the obtained data, process missing values and outliers to ensure data quality, and then perform normalization and standardization preprocessing on the data for subsequent analysis;

[0045] Step S33: Extract key features from various types of data, where the key features include temperature, snowfall, traffic flow, and road humidity, to establish a data model;

[0046] Step S34: Through task division, the data is divided into multiple small tasks so that each node can process them in parallel, making full use of the computing resources in the cluster;

[0047] Step S35: Start the distributed computing framework and distribute the tasks to each node in the cluster for parallel processing;

[0048] Step S36: Integrate the results obtained by each node to form a comprehensive analysis result.

[0049] Step S35 further includes the following steps:

[0050] Step S351: Use historical data for model training, considering meteorological, traffic, and road condition factors, so that the model can better predict possible future snow disaster situations;

[0051] Step S352: Use the validation dataset to evaluate the trained model to ensure its generalization ability on unknown data;

[0052] Step S353: Apply the trained model to real-time data to generate a real-time prediction of the impact of snow disaster snow accumulation on traffic.

[0053] In Step S353, the key features including temperature, snowfall, traffic flow, and road humidity are respectively marked as feature X1, X2, X3, X4, and then the corresponding weight values W1, W2, W3, W4, and a preset bias parameter c are assigned. The impact index of the current road traffic is calculated through the formula Y = W1X1 + W2X2 + W3X3 + W4X4 + c. In the formula, Y is the impact index of the current road traffic affected by snowfall; this impact index is affected by temperature, snowfall, traffic flow, and road humidity. When the temperature is lower, the larger the feature X1; when the snowfall is larger, the larger the feature X2; when the traffic flow is larger, the greater the probability of traffic accidents caused by snowfall and congestion, and thus the larger the impact index of the current road traffic affected by snowfall. Therefore, the larger the feature value X3. When the road humidity is larger, it is easier to cause road icing, and the larger the feature value X4. Among them, the bias parameter is the compensation value of the impact index after the communication execution cycle time U predicted according to big data. The communication execution cycle is preset according to real-time retrieval and response by humans. When the prediction result comes out, there is a time difference between the predicted impact index of the current road traffic and the impact index during the actual execution period. During the time difference, there are situations such as continuous snowfall and freezing that cause the impact index to increase. By adding the bias parameter c, predictive reference can be made by combining real-time situations and execution control situations, and dual prediction display can be carried out more comprehensively and accurately.

[0054] It should be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device.

[0055] Finally, it should be noted that the above are only preferred embodiments of the present invention and are not used to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A big data analysis and prediction system based on cloud computing, characterized in that: It includes a data acquisition module, a data storage module, a big data analysis module, a prediction result display module, and a command and control module. The data acquisition module is used to collect real-time meteorological, traffic flow, and road condition-related data through sensors and weather stations, and establish a data set. The data storage module is used to achieve efficient storage and management of data using a cloud computing platform, ensuring the security and reliability of the data. The big data analysis module is used to analyze and calculate various types of data to accurately predict the impact of snow disaster and snow accumulation on traffic. The prediction result display module is used to display the analysis results in an intuitive form on the interface of the command and control center for the reference of decision-makers, facilitating the timely adoption of countermeasures. The command and control module is used to automatically provide targeted snow removal and traffic control suggestions based on the results of big data analysis, and also supports manual intervention to ensure the timeliness and accuracy of decision-making.

2. The big data analysis and prediction system based on cloud computing according to claim 1, wherein: The big data analysis module further includes a data preprocessing sub-module, a feature extraction sub-module, and a model analysis sub-module. The data preprocessing sub-module is used to preprocess the collected data. The feature extraction sub-module is used to extract key features from the preprocessed data to establish a data model. The model analysis sub-module is used to analyze and predict the impact of subsequent snowfall on urban roads based on the established data model.

3. The big data analysis and prediction system based on cloud computing according to claim 2, wherein: The prediction result display module further includes a visualization sub-module and a real-time update sub-module. The visualization sub-module is used to display the analysis results in an intuitive form such as charts and maps on the interface of the command and control center for the reference of decision-makers. The real-time update sub-module is used to regularly update the analysis results to ensure that decision-makers obtain the latest prediction information.

4. The big data analysis and prediction system based on cloud computing according to claim 3, wherein: The command and control module further includes a road warning generation sub-module and a command suggestion generation sub-module. The road warning generation sub-module is used to judge the road warning timing based on the analysis results and generate corresponding warning information. The command suggestion generation sub-module is used to generate targeted snow removal and traffic control suggestions based on the results of big data analysis.

5. The big data analysis and prediction system based on cloud computing according to claim 4, wherein: The operation method of the big data analysis and prediction system based on cloud computing includes the following steps: Step S1: Start the data acquisition module to collect real-time meteorological, traffic flow, and road condition-related data; Step S2: Store the collected data through the cloud computing platform to ensure the security and reliability of the data; Step S3: Start the big data analysis module to deeply analyze and calculate the stored data using distributed computing and machine learning algorithms; Step S4: Display the analysis results in the form of maps and charts on the interface of the command and control center; Step S5: According to the displayed results, the system automatically generates targeted snow removal and traffic control suggestions, supporting manual intervention.

6. The big data analysis and prediction system based on cloud computing according to claim 5, wherein: The step S3 further includes the following steps: Step S31: Obtain multi-dimensional data related to real-time collected meteorological data, traffic flow data, and road condition data from the data storage module; Step S32: Clean the obtained data, handle missing values and outliers to ensure data quality, and then perform normalization and standardization preprocessing on the data for subsequent analysis; Step S33: Extract key features from various types of data, where the key features include temperature, snowfall, traffic flow, and road humidity, to establish a data model; Step S34: Through task division, divide the data into multiple small tasks so that each node can process them in parallel, making full use of the computing resources in the cluster; Step S35: Start the distributed computing framework and distribute the tasks to each node in the cluster for parallel processing; Step S36: Integrate the results obtained by each node to form a comprehensive analysis result.

7. The big data analysis and prediction system based on cloud computing according to claim 6, characterized in that: The said Step S35 further includes the following steps: Step S351: Use historical data for model training, considering meteorological, traffic, and road condition factors, so that the model can better predict possible future snow disaster situations; Step S352: Use the validation data set to evaluate the trained model to ensure its generalization ability on unknown data; Step S353: Apply the trained model to real-time data to generate a real-time prediction of the impact of snow disaster snow accumulation on traffic.

8. The big data analysis and prediction system based on cloud computing according to claim 7, characterized in that: In the said Step S353, the key features including temperature, snowfall, traffic flow, and road humidity are respectively marked as feature X1, X2, X3, X4, and then the corresponding weight values W1, W2, W3, W4, and a preset bias parameter c are given. Calculate the impact index of predicting the current road traffic through the formula Y = W1X1 + W2X2 + W3X3 + W4X4 + c. In the formula, Y is the impact index of analyzing and predicting the current road traffic affected by snowfall; this impact index is affected by temperature, snowfall, traffic flow, and road humidity. When the temperature is lower, the greater the feature X1, when the snowfall is greater, the greater the feature X2, when the traffic flow is greater, the greater the probability of traffic accidents caused by snowfall and congestion, and thus the greater the impact index of the current road traffic affected by snowfall. Therefore, the greater the feature value X3, when the road humidity is greater, it is easier to cause road icing, and the greater the feature value X4, where the bias parameter is the compensation value of the impact index after predicting the communication execution cycle time U according to big data.

Images