Internet big data processing system and method based on artificial intelligence
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- XIANYANG NORMAL UNIV
- Filing Date
- 2026-01-15
- Publication Date
- 2026-04-28
Smart Images
Figure CN121935840A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of artificial intelligence and big data processing technology, specifically to an internet big data processing system and method based on artificial intelligence. Background Technology
[0002] With the rapid development of the internet, big data technology has become one of the core technologies of modern information society. Big data processing technology is widely used in finance, e-commerce, healthcare, government, and other fields, providing solutions for processing massive amounts of data through distributed storage and computing architectures such as Hadoop and Spark. Existing technologies extract valuable information from data through batch processing, storage, and analysis. Parallel computing and cluster management improve the efficiency and scale of data processing, meeting the basic needs of industries for big data storage and computing power. The introduction of artificial intelligence technology enhances the intelligence of data processing systems, providing support for decision-making. With the widespread application of big data, the intelligentization, automation, and real-time processing of data have become key research areas.
[0003] However, existing technologies still have significant shortcomings. Relying on static algorithms and batch processing methods makes it difficult to meet the processing demands of real-time data streams. They cannot effectively handle high-concurrency, large-scale data analysis tasks. Existing systems rely on offline data processing and lack efficient analysis of real-time data streams. They cannot achieve efficient, intelligent, and real-time processing of big data in environments with massive and constantly changing data volumes. Summary of the Invention
[0004] To address the shortcomings of existing technologies, this invention provides an Internet big data processing system and method based on artificial intelligence. The technical problem this invention aims to solve is: how to use deep learning models, real-time data analysis, and distributed computing to address the inefficiency of static algorithms and batch processing methods in handling real-time big data streams.
[0005] To achieve the above objectives, the present invention provides the following technical solution: an Internet big data processing system based on artificial intelligence, comprising: Data acquisition module: used to acquire real-time data streams through sensors; Data stream processing module: used to predict the real-time data stream to form trend prediction results, wherein the prediction adopts a deep learning model; Data analysis module: used to perform real-time data analysis on the trend prediction results to form an analysis result dataset, wherein the real-time data analysis adopts an adaptive adjustment mechanism; Parallel processing module: used to perform distributed computation on the analysis result dataset, and generate distributed computation results through the distributed computation; System self-learning module: used to learn the deep learning model online based on the distributed computing results and the real-time data stream, and form closed-loop control through the online learning.
[0006] Preferably, the real-time data stream includes temperature data, humidity data, and pressure data.
[0007] Preferably, the deep learning model recursively calculates the hidden state and cell state on the real-time data stream. The data stream processing module includes a state memory unit that stores the hidden state and cell state. The deep learning model includes a convolutional neural network and a long short-term memory network. The convolutional neural network includes convolutional layers and pooling layers. The convolutional layers extract features from the real-time data stream to generate intermediate feature maps. The pooling layers perform pooling operations on the intermediate feature maps to generate a first part of the prediction result. The pooling operation adopts the average pooling method.
[0008] Preferably, the Long Short-Term Memory network performs a weighted summation of the real-time data stream and the hidden state, generates a gated activation value through the Sigmoid activation function, updates the cell state based on the gated activation value, forms a second part of the prediction result through the update, and then weights and fuses the first part of the prediction result and the second part of the prediction result to form a trend prediction result.
[0009] Preferably, the adaptive adjustment mechanism calculates the absolute error between the real-time data stream and the trend prediction result to obtain the prediction error value, and optimizes and adjusts the trend prediction result based on the prediction error value and the real-time data stream to form an analysis result dataset.
[0010] Preferably, the optimization adjustment includes the following steps: S31. The trend prediction result is weighted and corrected, wherein the weighted correction is performed on the prediction result by weighting the prediction result according to the prediction error value and the real-time data stream to form a corrected prediction result; S32. The mean of the trend prediction results is extracted to form a trend prediction value, and the learning rate of the trend prediction value is adjusted to form an optimized prediction result; S33. The corrected prediction results and the optimized prediction results are fused to form an analysis result dataset, and the fusion is performed using a weighted average method.
[0011] Preferably, the learning rate adjustment sets an error fluctuation threshold based on the volatility of the real-time data stream. When the trend prediction value is greater than the error fluctuation threshold, the learning rate is set to 1.5 times the prediction error value. When the trend prediction value is less than or equal to the error fluctuation threshold, the learning rate is set to 0.5 times the prediction error value.
[0012] Preferably, the distributed computing divides the analysis result dataset into data subsets, distributes the data subsets to computing nodes, and performs parallel summation on the data subsets based on the computing nodes to form the distributed computing result.
[0013] Preferably, the online learning compares the distributed computing results with the real-time data stream using a loss function, generates a quantization error result through the loss function comparison, adjusts the weights of the deep learning model based on the quantization error result, and synchronizes the adjusted deep learning model to the data stream processing module.
[0014] An artificial intelligence-based method for processing big data on the Internet, comprising: S1. Acquire real-time data streams through sensors, the real-time data streams including temperature data, humidity data, and pressure data; S2. The real-time data stream is predicted to form a trend prediction result, wherein the prediction is performed using a deep learning model; S3. Real-time data analysis is performed on the trend prediction results to form an analysis result dataset, wherein the real-time data analysis adopts an adaptive adjustment mechanism; S4. Perform distributed computation on the analysis result dataset to generate distributed computation results; S5. The deep learning model is trained online based on the distributed computing results and the real-time data stream, and closed-loop control is formed through the online learning.
[0015] This invention provides an internet big data processing system and method based on artificial intelligence. It has the following beneficial effects: This AI-based internet big data processing system and method improves accuracy and stability by introducing a deep learning model to predict trends in real-time data streams and optimizing prediction results through an adaptive adjustment mechanism. It combines convolutional neural networks and long short-term memory networks to extract and fuse data features from multiple dimensions, accurately predicting future trends. Distributed computing and parallel processing enhance the system's processing capabilities, supporting efficient operation in large-scale data environments.
[0016] An adaptive adjustment mechanism and online learning function optimize the system's responsiveness to real-time data streams. Real-time error correction and dynamic learning rate adjustment improve prediction accuracy, enhance the system's self-adjustment capabilities, and ensure stable prediction performance under different data environments. The adaptive adjustment mechanism supports closed-loop control based on new data streams, continuously optimizing prediction and data analysis capabilities. Attached Figure Description
[0017] Figure 1 This is a schematic diagram of the system structure of the present invention; Figure 2 This is a flowchart of the data stream processing module of the present invention; Figure 3 The flowchart for the adaptive adjustment of the data analysis module of this invention; Figure 4 This is a flowchart of the distributed computing process of the parallel processing module of the present invention; Figure 5 This is a flowchart of the closed-loop control of the self-learning module of the system of the present invention; Figure 6 This is a flowchart of the overall method of the present invention. Detailed Implementation
[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0019] Example 1 like Figure 1-6 As shown, this embodiment of the invention provides an Internet big data processing system based on artificial intelligence, comprising: Data acquisition module: Used to acquire real-time data streams through sensors. Real-time data streams include temperature data, humidity data, and pressure data.
[0020] The data stream processing module is used to predict trends from real-time data streams using a deep learning model. The deep learning model recursively calculates the hidden states and cell states from the real-time data stream. The module includes a state memory unit that stores the hidden states and cell states. The deep learning model comprises a convolutional neural network (CNN) and a long short-term memory (LSTM) network. The CNN includes convolutional layers and pooling layers. The convolutional layers extract features from the real-time data stream to generate intermediate feature maps. The pooling layers perform pooling operations on the intermediate feature maps to generate the first part of the prediction result, using average pooling. The LSTM network performs a weighted sum of the real-time data stream and the hidden states, and generates gated activation values using the sigmoid activation function. The cell states are updated based on these gated activation values, forming the second part of the prediction result. The first and second parts of the prediction result are then weighted and fused to form the final trend prediction result.
[0021] At each time step, the Long Short-Term Memory (LSTM) network recursively updates the hidden and cell states by combining the current input with the hidden state and cell state from the previous time step. Recursive computation is achieved through a weighted sum of the previous state and the current input; the computation at each time step depends on the information from the previous time step. The LSM network uses a gating mechanism to weightedly update the cell states to obtain the current hidden and cell states.
[0022] The hidden state represents key information from the sequence data at the current time step. At each time step, the Long Short-Term Memory (LSTM) network generates a new hidden state by recursively computing the current input and the hidden state from the previous time step. This recursive process allows the model to pass and update its understanding of the input data features through the sequence data.
[0023] Cell states are used to store long-term dependent information. At each time step, the cell state is recursively updated by combining the current input with the cell state from the previous time step. The Long Short-Term Memory (LSTM) network controls the retention and discarding of information through input gates, forget gates, and output gates to update the cell state and pass long-term dependent information to subsequent time steps.
[0024] The data analysis module performs real-time data analysis on the trend forecast results to generate an analysis result dataset. This real-time data analysis employs an adaptive adjustment mechanism. This mechanism calculates the absolute error between the real-time data stream and the trend forecast results to obtain the prediction error value. Based on this prediction error value and the real-time data stream, the trend forecast results are optimized and adjusted to form the analysis result dataset. The optimization and adjustment include the following steps: S31. The trend prediction results are weighted and corrected. The weighted correction is based on the prediction error value and the real-time data stream to form a corrected prediction result.
[0025] S32. The trend prediction results are averaged to form a trend prediction value, and the learning rate of the trend prediction value is adjusted to form an optimized prediction result. The learning rate adjustment sets an error fluctuation threshold based on the volatility of the real-time data stream. When the trend prediction value is greater than the error fluctuation threshold, the learning rate is set to 1.5 times the prediction error value. When the trend prediction value is less than or equal to the error fluctuation threshold, the learning rate is set to 0.5 times the prediction error value.
[0026] S33. The corrected prediction results and the optimized prediction results are merged to form an analysis result dataset. The fusion is performed using a weighted average method.
[0027] The adaptive adjustment mechanism flexibly adjusts the learning rate based on the volatility of the real-time data stream. In the case of highly volatile data, it can prevent over-adjustment or unnecessary computation, optimize the system's resource utilization efficiency, and improve overall computing performance.
[0028] Parallel processing module: Used for distributed computation on the analysis result dataset, generating distributed computation results. Distributed computation divides the analysis result dataset into data subsets, distributes these subsets to computing nodes, and performs parallel summation on these subsets based on the computing nodes to form the distributed computation results.
[0029] By employing a distributed computing approach through a parallel processing module, the analysis result dataset is divided and distributed across multiple computing nodes for parallel processing, thereby improving data processing speed and computational efficiency. When dealing with large-scale data, this reduces processing latency, ensuring fast and accurate computation and analysis.
[0030] Designed using distributed computing and deep learning models, the system boasts strong scalability and flexibility. As data volume and computational demands increase, the system can adapt to larger-scale computing tasks by adding computing nodes or optimizing model configurations, thus meeting ever-growing needs.
[0031] The system's self-learning module is used to learn the deep learning model online based on distributed computing results and real-time data streams, forming a closed-loop control through online learning. The online learning process compares the distributed computing results and real-time data streams using a loss function, generating a quantization error result. Based on this quantization error result, the weights of the deep learning model are adjusted, and the adjusted deep learning model is synchronized to the data stream processing module.
[0032] The system's self-learning module dynamically adjusts the deep learning model based on distributed computing results and real-time data stream feedback through an online learning mechanism, forming a closed-loop control. Its adaptive capability enables the system to continuously optimize according to constantly changing environmental and data conditions, thereby improving long-term stability and accuracy.
[0033] An artificial intelligence-based method for processing big data on the Internet, comprising: S1. Acquire real-time data streams through sensors, including temperature data, humidity data, and pressure data.
[0034] S2. The real-time data stream is predicted to form trend prediction results. The prediction uses a deep learning model.
[0035] S3. Real-time data analysis of the trend prediction results is performed to form an analysis result dataset. The real-time data analysis adopts an adaptive adjustment mechanism.
[0036] S4. Perform distributed computation on the analysis result dataset to generate distributed computation results.
[0037] S5. Based on the results of distributed computing and real-time data streams, the deep learning model is trained online to form a closed-loop control.
[0038] Example 2 This embodiment describes an artificial intelligence-based internet big data processing system and method. It combines convolutional neural networks and long short-term memory networks to predict real-time data collected by sensors, and optimizes the prediction results through an adaptive adjustment mechanism to improve the prediction accuracy and adaptability of the data stream processing system. The specific implementation is as follows: 1. Data Collection The system collects data through multiple sensors: a temperature sensor collects temperature data in °C, a humidity sensor collects humidity data in %, and a pressure sensor collects pressure data in hPa.
[0039] The following are the actual sensor data collected, with a collection interval of 1 second: Table 1: Sensor Data Acquisition Table.
[0040]
[0041] The aforementioned data stream is input into the data stream processing module for processing and trend prediction.
[0042] 2. Data Stream Processing Convolutional Neural Networks: Convolutional neural networks (CNNs) are used to extract features from real-time data streams. Data at each time point is treated as a feature vector and input into the CNN. For temperature, humidity, and pressure data at five time points, the convolutional layers extract features for each time point.
[0043] Pooling operation: The pooling layer uses average pooling to reduce the dimensionality of the data, with a window size of 3.
[0044] Pooling the temperature data:
[0045]
[0046]
[0047] Pooling operation on humidity data:
[0048]
[0049]
[0050] Pooling operation on stress data:
[0051]
[0052]
[0053] Long Short-Term Memory Networks: Long Short-Term Memory (LSTM) networks are used to analyze long-term dependencies in time series data. At each time step, the LSM network recursively updates the hidden state and cell state by combining the current input with the hidden state and cell state from the previous time step.
[0054] At each time step, the Long Short-Term Memory (LSTM) network performs recursive computation by weighted summation of the previous hidden state and cell state with the current input. Specifically, the LSM network controls the retention and discarding of information through input gates, forget gates, and output gates: The forget gate is responsible for deciding how much information from the cell state at the previous time step should be forgotten, the input gate is responsible for deciding how much information from the current input will be added to the cell state, and the output gate is responsible for deciding how much information from the cell state will be passed to the hidden state at the current time step.
[0055] The hidden state represents key information from the sequence data at the current time step. At each time step, the Long Short-Term Memory network updates the hidden state by recursively calculating a weighted sum of the current input and the hidden state from the previous time step, thus conveying its understanding of the sequence data.
[0056] Cell states are used to store long-term dependent information. At each time step, the cell state is updated by weighting the current input with the cell state at the previous time step. Through a gating mechanism, the Long Short-Term Memory (LSTM) network controls the flow of information, ensuring that only the parts important to long-term dependencies are retained.
[0057] Long Short-Term Memory (LSTM) networks model temperature, humidity, and pressure data, learning from data at the first four time points to predict trends at the fifth time point. The recursive computation of the prediction process includes updates to long-term dependencies on historical data, generating the final prediction result.
[0058] The predicted results are: temperature 23.1℃, humidity 63.5%, and pressure 1017hPa.
[0059] 3. Fusion Prediction Results The prediction results from convolutional neural networks and long short-term memory networks are weighted and fused to form the final prediction result. The weighted fusion formula is: Weight settings: Based on the working principle of the model, CNN is used for local feature extraction, LSTM is used for time series modeling, and the weights are initially set.
[0060] CNNs typically perform well in extracting local features from data, so their weights may be set relatively low, as CNNs are primarily responsible for handling changes over short time spans. LSTMs excel at capturing long-term dependencies in time series data, therefore LSTM weights are usually set higher; LSTMs better understand historical data and long-term trends across consecutive time steps. Therefore, setting... , .
[0061]
[0062]
[0063] .
[0064] 4. Adaptive adjustment mechanism Based on the known data from the fifth time point, the temperature was 23.2℃, the humidity was 64%, and the actual pressure was 1018 hPa. The predicted results are a temperature of 22.928℃, a humidity of 62.5%, and a pressure of 1015.532 hPa.
[0065] Calculation error:
[0066]
[0067]
[0068] Learning rate adjustment: Based on the volatility of the real-time data stream, an error fluctuation threshold is set. When the prediction error exceeds the set threshold, the learning rate is adjusted to 1.5 times the prediction error. When the prediction error is less than or equal to the error fluctuation threshold, the learning rate is adjusted to 0.5 times the prediction error. The set error fluctuation threshold is 0.5℃. Based on the error calculation, the temperature error is 0.272℃, which is less than the error fluctuation threshold. Therefore, the learning rate is adjusted to 0.5 times the prediction error.
[0069] The set error fluctuation threshold is 0.5%. Based on the error calculation, the humidity error is 1.5%, which is greater than the error fluctuation threshold. Therefore, the learning rate is adjusted to 1.5 times the prediction error.
[0070] The set error fluctuation threshold is 0.5%. Based on the error calculation results, the pressure error is 2.468 hPa, which is greater than the error fluctuation threshold. Therefore, the learning rate is adjusted to 1.5 times the prediction error.
[0071] Weighted adjustment: Based on the prediction error, the system uses a weighted correction mechanism to adjust the prediction results. By analyzing historical data and assessing the fluctuation range of the prediction error, a correction coefficient of 1.1 is set. The corrected prediction result is then:
[0072]
[0073]
[0074] Final optimized prediction: The corrected prediction results are then fused with the adjusted learning rate. The final optimized prediction result is calculated as follows: .
[0075]
[0076] .
[0077] Through the steps described above, convolutional neural networks and long short-term memory networks were used to predict real-time data such as temperature, humidity, and pressure, and the prediction results were optimized through weighted fusion. An adaptive adjustment mechanism was employed, dynamically adjusting the system based on real-time data flow and prediction errors, thus improving prediction accuracy. The corrected prediction results were reasonably optimized during the correction process for temperature, humidity, and pressure data, ensuring the system's accurate prediction of future trends and efficient data processing.
[0078] Example 3 This embodiment describes an internet big data processing system and method based on artificial intelligence. Through a parallel processing module and a system self-learning module, it optimizes the accuracy of the prediction model and improves real-time data processing capabilities based on environmental data collected by sensors. The specific implementation is as follows: 1. Data Collection This system deploys multiple environmental monitoring devices in a smart city area to monitor environmental changes. Sensors collect environmental data every second, including temperature, humidity, and air pressure. The collected data is transmitted in real-time via a wireless network to a cloud platform for storage and processing.
[0079] The data acquisition cycle is set to once per second, so the data collected at each point in time accurately reflects the real-time environmental conditions. The following is the data collected during a specific time period: Table 2: Data Collection Table.
[0080]
[0081] The data above was collected in real time by environmental monitoring equipment deployed in smart cities. The sensors, using IoT technology, collect data once per second, which is then wirelessly transmitted to a cloud platform for processing and storage.
[0082] Data such as temperature, humidity, and air pressure are transmitted to the central database via API interface and updated in real time to ensure the timeliness of the data.
[0083] 2. Data partitioning and allocation In the parallel processing module, the system divides the collected raw data into different subsets according to time points, so that they can be distributed to different computing nodes for parallel processing. The data is divided into two subsets, which are processed by computing node 1 and computing node 2 respectively.
[0084] Data Subset 1: Contains data from time points T1 to T3. Computation node 1 will process the data in subset 1.
[0085] Data subset 2: Contains data from time points T4 to T5. Computation node 2 will process the data in subset 2.
[0086] 3. Parallel computing After data allocation, the system assigns each data subset to a corresponding computing node for parallel computation. Each node is responsible for processing the data allocated to it and producing its own result.
[0087] Computation node 1: Responsible for processing data from T1 to T3, and obtaining the following results: temperature:
[0088] humidity:
[0089] pressure:
[0090] Computation node 2: Responsible for processing data from T4 to T5, and obtaining the following results: temperature:
[0091] humidity:
[0092] pressure:
[0093] 4. Weighted average calculation results After parallel data computation, the system will merge the results of each computation node using a weighted average method.
[0094] In this embodiment, data subset 1 is larger or has higher computational complexity, therefore computing node 1 is given a higher weight of 0.6, while computing node 2 processes a smaller data subset, with a weight of 0.4. The results from computing node 1 and computing node 2 are then fused according to a weighted ratio to obtain the following combined result: Temperature-weighted average:
[0095] Humidity-weighted average:
[0096] Stress-weighted average:
[0097] 5. Data Input and Loss Function Calculation In the system's self-learning module, the input data is the result obtained through the parallel processing module. To evaluate the accuracy of the prediction, the system compares the prediction results with the actual sensor data collected.
[0098] The actual data were: temperature 23.5℃, humidity 61%, pressure 1013hPa. The predicted results were provided by the parallel processing module: temperature 23.842℃, humidity 63%, pressure 1015.798hPa.
[0099] The system will calculate the loss function using mean squared error (MSE) to measure the difference between predicted and actual values. The loss function is central to machine learning and deep learning model training, directly impacting the model's optimization process, training effectiveness, and final prediction accuracy. The MSE formula is:
[0100] The calculated temperature error is 0.12, the humidity error is 4, and the pressure error is 7.83.
[0101] The loss function value is used during training to guide the optimization algorithm, adjust the model parameters, and gradually improve the model's predictive ability.
[0102] 6. Weight Adjustment and Learning Rate Calculation By calculating the revised loss function and adjusting the learning rate, we ensured that the model's feedback and optimization were more reasonable. We set the error fluctuation threshold to 4, and the new error value better reflects the range required in real-world applications.
[0103] Model synchronization and feedback adjustment: The system updates the weights of the deep learning model by calculating the error and learning rate. The updated deep learning model is then synchronized to the data stream processing module. After each update, the system applies the new weights to the real-time data stream to improve prediction accuracy.
[0104] Closed-loop control and optimization: After each feedback optimization, the system reprocesses the real-time collected environmental data using the optimized model based on the new learning results. With each update, the system continuously adjusts the model weights, forming an adaptive closed-loop control system. After each update, the system provides more accurate predictions, reduces errors, and improves prediction accuracy.
[0105] In summary, the parallel processing module enables efficient processing of large-scale real-time environmental data, and the system's self-learning module continuously optimizes prediction accuracy by dynamically adjusting model weights. By fusing the results from multiple computing nodes using a weighted average method, the system operates efficiently in big data environments. The learning rate is adjusted according to changes in error magnitude, allowing the system to converge quickly and remain stable during optimization, ultimately improving the accuracy of prediction results and providing reliable data support for intelligent urban environmental monitoring and management.
[0106] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.
Claims
1. An internet big data processing system based on artificial intelligence, characterized in that, include: Data acquisition module: Used to acquire real-time data streams of big data from the Internet through sensors; Data stream processing module: used to predict the real-time data stream to form trend prediction results, wherein the prediction adopts a deep learning model; Data analysis module: used to perform real-time data analysis on the trend prediction results to form an analysis result dataset, wherein the real-time data analysis adopts an adaptive adjustment mechanism; Parallel processing module: used to perform distributed computation on the analysis result dataset, and generate distributed computation results through the distributed computation; System self-learning module: used to learn the deep learning model online based on the distributed computing results and the real-time data stream, and form closed-loop control through the online learning.
2. The Internet big data processing system based on artificial intelligence according to claim 1, characterized in that: The real-time data stream includes temperature data, humidity data, and pressure data.
3. The Internet big data processing system based on artificial intelligence according to claim 1, characterized in that: The deep learning model recursively calculates the hidden state and cell state from the real-time data stream. The data stream processing module includes a state memory unit that stores the hidden state and cell state. The deep learning model includes a convolutional neural network and a long short-term memory network. The convolutional neural network includes convolutional layers and pooling layers. The convolutional layers extract features from the real-time data stream to generate intermediate feature maps. The pooling layers perform pooling operations on the intermediate feature maps to generate a first part of the prediction result. The pooling operation adopts the average pooling method.
4. The Internet big data processing system based on artificial intelligence according to claim 3, characterized in that: The Long Short-Term Memory network performs a weighted summation of the real-time data stream and the hidden state, generates a gated activation value through the Sigmoid activation function, updates the cell state based on the gated activation value, forms a second part of the prediction result through the update, and then weights and fuses the first part of the prediction result and the second part of the prediction result to form a trend prediction result.
5. The Internet big data processing system based on artificial intelligence according to claim 1, characterized in that: The adaptive adjustment mechanism calculates the absolute error between the real-time data stream and the trend prediction results to obtain the prediction error value. Based on the prediction error value and the real-time data stream, the trend prediction results are optimized and adjusted to form an analysis result dataset.
6. The Internet big data processing system based on artificial intelligence according to claim 5, characterized in that: The optimization and adjustment include the following steps: S31. The trend prediction result is weighted and corrected, wherein the weighted correction is performed on the prediction result by weighting the prediction result according to the prediction error value and the real-time data stream to form a corrected prediction result; S32. The trend prediction results are averaged to form a trend prediction value, and the trend prediction value is adjusted by a learning rate to form an optimized prediction result; S33. The corrected prediction results and the optimized prediction results are fused to form an analysis result dataset, and the fusion is performed using a weighted average method.
7. The Internet big data processing system based on artificial intelligence according to claim 6, characterized in that: The learning rate adjustment sets an error fluctuation threshold based on the volatility of the real-time data stream. When the trend prediction value is greater than the error fluctuation threshold, the learning rate is set to 1.5 times the prediction error value. When the trend prediction value is less than or equal to the error fluctuation threshold, the learning rate is set to 0.5 times the prediction error value.
8. The Internet big data processing system based on artificial intelligence according to claim 1, characterized in that: The distributed computing divides the analysis result dataset into data subsets, distributes the data subsets to computing nodes, and performs parallel summation on the data subsets based on the computing nodes to form the distributed computing result.
9. The Internet big data processing system based on artificial intelligence according to claim 1, characterized in that: The online learning process compares the distributed computing results with the real-time data stream using a loss function. The loss function comparison generates a quantization error result, which is then used to adjust the weights of the deep learning model. The adjusted deep learning model is then synchronized to the data stream processing module.
10. A method for processing Internet big data based on artificial intelligence, characterized in that: S1. Acquire real-time data streams through sensors, the real-time data streams including temperature data, humidity data, and pressure data; S2. The real-time data stream is predicted to form a trend prediction result, wherein the prediction is performed using a deep learning model; S3. Real-time data analysis is performed on the trend prediction results to form an analysis result dataset, wherein the real-time data analysis adopts an adaptive adjustment mechanism; S4. Perform distributed computation on the analysis result dataset to generate distributed computation results; S5. The deep learning model is trained online based on the distributed computing results and the real-time data stream, and closed-loop control is formed through the online learning.