An ai artificial intelligence-based big data processing and analysis method and system

Abstract

The application discloses a kind of big data processing analysis method and system based on Ai artificial intelligence, belong to big data processing and artificial intelligence field, including data perception unit, core engine unit and decision execution unit, data perception unit includes data acquisition module and pre-processing module, data acquisition module is responsible for from each equipment sensor, SCADA system and maintenance log collection time series data, time data.The application is continuously optimized feature representation, analysis model and abnormal detection baseline under no intervention by autonomous learning module, cope with environmental change, through planning learning module, can be based on analysis result and long-term goal, autonomously generate and execute optimal action sequence, to optimize traditional autonomous intelligent system, and the multi-step reasoning and long-term return consideration introduced in planning learning, so that decision is no longer the optimal solution for single time point, to improve the depth of analysis and decision quality.

Description

Technical Field

[0001] This invention relates to the fields of big data processing and artificial intelligence technology, and in particular to a big data processing and analysis method and system based on AI artificial intelligence. Background Technology

[0002] With the continuous development of the information society, the amount of data being processed is growing explosively. Big data analytics technology has become the core support for decision-making in various fields. Existing big data analytics systems typically use predefined models and rules for processing, such as relying on manual feature engineering and regular model updates. This makes it difficult to cope with the dynamic changes in data flow, resulting in the degradation of model performance over time. Moreover, the systems remain at the level of descriptive and predictive analysis, lacking the ability to make multi-step inferences and autonomous decisions for complex business objectives. Furthermore, during the analysis and decision-making processes, human intervention is still required to interpret the results and formulate action plans, making it impossible to form a closed loop from perception to analysis and decision-making to action.

[0003] Although big data analytics has made many attempts to introduce machine learning for prediction, it usually limits learning to the static optimization of model parameters, failing to enable the system to acquire advanced artificial intelligence that allows it to evolve autonomously in dynamic environments and achieve long-term goals in a planned manner. Summary of the Invention

[0004] The purpose of this invention is to solve the problems existing in the prior art by proposing a big data processing and analysis method and system based on AI artificial intelligence.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: A big data processing and analysis system based on AI artificial intelligence includes a data sensing unit, a core engine unit, and a decision execution unit. The data sensing unit includes a data acquisition module and a preprocessing module. The data acquisition module is responsible for collecting time-series data and time data from various device sensors, SCADA systems, and maintenance logs. The preprocessing module cleans and aligns the collected time-series data and time data. The core engine unit is connected to the data perception unit and includes a mutually coupled autonomous learning module and a planning learning module. The autonomous learning module includes a dynamic feature engineering model optimization submodule, an anomaly detection pattern discovery submodule, and a knowledge base, which are used to perform dynamic model optimization, anomaly detection, and knowledge accumulation from the data stream. The planning learning module includes a digital twin environment simulator, a planning decision engine, and an action sequence generation submodule, which perform sequence decision deduction based on the environmental state information provided by the autonomous learning module to generate the optimal action sequence. The decision execution unit includes an execution agent module, which receives and executes the action instruction sequence generated by the planning and learning module, and directly applies it to physical devices or business systems. At the same time, it collects new data generated after the action is executed and feeds it back to the data perception unit.

[0006] As a preferred embodiment, the dynamic feature engineering model optimization submodule dynamically learns and optimizes feature representations from the data stream, and automatically monitors, evaluates and updates the fault prediction model to cope with conceptual drift in data distribution and maintain model performance. The anomaly detection mode discovery submodule uses the optimized model and features to perform real-time anomaly detection, and uses clustering algorithms to discover potential new patterns and state patterns in the data. The knowledge base stores and accumulates knowledge of abnormal patterns and new state patterns discovered by the anomaly detection pattern discovery submodule in a structured form, forming the system's experience memory.

[0007] As a preferred embodiment, the digital twin environment simulator constructs a virtual business environment simulation model, integrating physical laws and data-driven models. It adopts a hybrid modeling approach, using equations based on physical laws to model the deterministic degradation process of equipment, and using probability distributions learned from historical data in the knowledge base to model the stochastic process of failure, thereby enabling future prediction and trial and error. In the simulated environment of the digital twin environment simulator, the planning and decision engine uses reinforcement learning algorithms to perform multi-step reasoning and sequential decision-making in order to find the optimal strategy to achieve a given long-term goal. The planning and decision engine takes equipment status characteristics and resource inventory status as input through a policy network and outputs maintenance, scheduling, and parameter adjustment actions. The action sequence generation submodule transforms the optimal strategy output by the planning decision engine into a detailed sequence of action instructions with a timeline.

[0008] As a preferred embodiment, the processing within the autonomous learning module includes unsupervised anomaly detection based on an autoencoder and online feature weight update. In unsupervised anomaly detection, data is acquired from raw time-series signals from the device's sensors, and the acquisition process is as follows: Data slicing: Slicing streaming data into segments using fixed time windows; Feature calculation: Calculate the time-domain and frequency-domain features of the original signal within each window; Standardization: Using the online calculated moving mean and standard deviation, for Z-score standardization is performed to obtain the final input. ; Reconstructing computation: The input is fed into a pre-trained autoencoder network. After one forward propagation, the decoder outputs the reconstructed vector. ; Loss calculation: according to the formula Calculate MSE loss; Data for online feature weight updates is collected from labeled historical data pairs. ,in For feature vectors, The data collection process for the actual labels is as follows: Model prediction: Using the current linear prediction model To predict the sample, where For the Sigmoid function; Calculate the gradient: Calculate the gradient of the loss function resulting from the current sample. For log loss, the gradient formula is: ; Update weights: By substituting the complex update formula for online feature weight calculation, a new weight vector is calculated. .

[0009] As a preferred embodiment, the processing within the planning and learning module includes the design of the reward function for deep reinforcement learning and the optimization of the proximal policy. In the design of the reward function for deep reinforcement learning, variable acquisition includes the acquisition of the environment state and the acquisition of each component of the reward function. The process of acquiring the environment state through multi-source information fusion is as follows: Obtain from the self-learning module: Extract the device health index and real-time anomaly probability output by the self-learning module; Obtain from business systems: Connect to the Enterprise Resource Planning and Manufacturing Execution System via API to read spare parts inventory, available maintenance personnel, and the priority of current production orders; State vector construction: After normalizing the above information, concatenate them into a fixed-dimensional state vector. ; The process of obtaining each component of the reward function through environmental feedback and business records after the action is executed is as follows: direct costs :action Once triggered, the system automatically queries and summarizes data from the standard maintenance man-hour cost database and the spare parts price database; Production stoppage losses According to the action The projected downtime is calculated by multiplying the defined downtime by the standard output value per unit time of the production line. Fault indication : Monitor unplanned shutdown alarm signals from the equipment control system. If in status If a signal is received within a specified time, a huge negative constant is returned; otherwise, zero is returned. Early warning reward : Need to be recorded, if actions For devices marked as having warnings or potential malfunctions by the self-learning module, a positive reward is returned to encourage preventative intervention. In the optimization of the proximal strategy, the process of obtaining variables through a series of states, actions, and reward trajectories is as follows: Value estimation: using a value network It inputs the state. Output an estimate of the long-term reward for that state; Calculating the time difference error: The generalized dominance estimation method is used, and the core is to calculate the time difference error at each step. ,in It is a discount factor; Advantages of smooth estimation: ,in It is a smoothing parameter .

[0010] A big data processing and analysis method based on AI (Artificial Intelligence) includes the following steps: S1: Through the self-learning module, the input data stream is continuously analyzed, the analysis model and feature representation are dynamically optimized, and automatic anomaly detection and knowledge accumulation are achieved; The dynamic optimization analysis model and feature representation include the following steps: Step S101: Use an online learning algorithm to update the feature weights of the input data stream in real time; Step S102: Continuously monitor the performance metrics of the prediction model. When the performance degradation exceeds a predetermined threshold, automatically trigger the hyperparameter search and retraining process of the model, and use a new model with better performance for iteration.

[0011] S2: Through the planning and learning module, high-level goals and constraints are received, and based on the environmental state information provided by the autonomous learning module, multi-step sequence decision-making is performed in the environmental simulation to generate the optimal action sequence; The planning learning module performs multi-step sequential decision inference, which includes the following steps: Step S201: Quantize the high-level objective into a reward function in reinforcement learning; Step S202: In the digital twin environment simulator, starting from the current environmental state, conduct multi-step forward-looking exploration through the interaction between smart devices and the simulated environment, and evaluate the long-term cumulative returns of different action sequences; Step S203: Select and output the action sequence that maximizes long-term cumulative returns as the optimal action sequence.

[0012] S3: The optimal action sequence is executed through the decision execution unit, and the feedback data generated after execution is re-input into the autonomous learning module to form a data analysis closed loop.

[0013] A big data processing and analysis method based on AI (Artificial Intelligence) further includes a collaborative method between an autonomous learning module and a planning learning module, wherein the collaborative method includes the following steps: Step 1: The planning learning module generates and executes a sequence of actions to change the state of the physical or business system. The resulting new data is collected by the data perception unit as feedback and used for the continuous optimization of the self-learning module. Step 2: The knowledge base updated by the self-learning module is used to improve the simulation fidelity of the digital twin environment simulator, thereby enabling the planning and learning module to make more reliable long-term decisions.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention, through its autonomous learning module, can continuously optimize feature representation, analysis model, and anomaly detection baseline without human intervention, and respond to environmental changes. Through its planning and learning module, it can autonomously generate and execute the optimal action sequence based on analysis results and long-term goals, thereby optimizing traditional autonomous intelligent systems.

[0015] 2. The multi-step reasoning and long-term reward considerations introduced in this invention in planning learning make decision-making no longer focused on the optimal solution at a single point in time, thereby improving the depth of analysis and the quality of decision-making.

[0016] 3. This invention integrates analysis, decision-making, execution, and feedback into an automatically operating closed loop, reducing the need for manual monitoring, interpretation, and intervention, improving operational efficiency, and ensuring the real-time and consistent nature of decision responses, thereby forming an efficient closed loop and reducing reliance on human intervention.

[0017] 4. The autonomous learning mechanism of this invention ensures the system's robustness to changes in data distribution, and through planning learning, enables the system to cope with complex scenarios where objectives and constraints are dynamically adjusted. The combination of these two aspects significantly improves the system's portability and adaptability in different application fields, thereby enhancing the system's adaptability. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the workflow of a big data processing and analysis system based on AI proposed in this invention; Figure 2 This is a schematic diagram of the analysis process of a big data processing and analysis method based on AI proposed in this invention. Detailed Implementation

[0019] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0020] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0021] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0022] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0023] Example, refer to Figure 1 A big data processing and analysis system based on AI artificial intelligence includes a data sensing unit, a core engine unit, and a decision execution unit. The data sensing unit includes a data acquisition module and a preprocessing module. The data acquisition module is responsible for collecting time-series data and time data from various device sensors, SCADA systems, and maintenance logs. The preprocessing module cleans and aligns the collected time-series data and time data. The core engine unit is connected to the data perception unit and includes a mutually coupled autonomous learning module and a planning learning module. The autonomous learning module includes a dynamic feature engineering model optimization submodule, an anomaly detection pattern discovery submodule, and a knowledge base, which are used to perform dynamic model optimization, anomaly detection, and knowledge accumulation from the data stream. The planning learning module includes a digital twin environment simulator, a planning decision engine, and an action sequence generation submodule, which perform sequence decision deduction based on the environmental state information provided by the autonomous learning module to generate the optimal action sequence. Furthermore, the dynamic feature engineering model optimization submodule dynamically learns and optimizes feature representations from the data stream, and automatically monitors, evaluates, and updates the fault prediction model to cope with conceptual drift in data distribution and maintain model performance; The anomaly detection mode discovery submodule uses the optimized model and features to perform real-time anomaly detection, and uses clustering algorithms to discover potential new patterns and state patterns in the data. The knowledge base stores and accumulates knowledge of abnormal patterns and new state patterns discovered by the anomaly detection pattern discovery submodule in a structured form, forming the system's experience memory. Furthermore, the digital twin environment simulator constructs a virtual business environment simulation model, integrating physical laws and data-driven models. It adopts a hybrid modeling approach, using equations based on physical laws to model the deterministic degradation process of equipment, and using probability distributions learned from historical data in the knowledge base to model the stochastic process of failure, thereby extrapolating and experimenting for the future. In the simulated environment of the digital twin environment simulator, the planning and decision engine uses reinforcement learning algorithms to perform multi-step reasoning and sequential decision-making in order to find the optimal strategy to achieve a given long-term goal. The planning and decision engine takes equipment status characteristics and resource inventory status as input through a policy network and outputs maintenance, scheduling, and parameter adjustment actions. The action sequence generation submodule transforms the optimal strategy output by the planning decision engine into a detailed sequence of action instructions with a timeline. The decision execution unit includes an execution agent module, which receives and executes the action instruction sequence generated by the planning and learning module, and directly applies it to physical devices or business systems. At the same time, it collects new data generated after the action is executed and feeds it back to the data perception unit.

[0024] The following examples of data processing in the self-learning and planned learning modules illustrate industrial predictive maintenance: The self-learning module's processing includes unsupervised anomaly detection based on an autoencoder and online feature weight update. In unsupervised anomaly detection, data is acquired from raw time-series signals from device sensors, such as vibration waveforms and current readings. The acquisition process is as follows: Data slicing: Slicing streaming data into segments using fixed time windows; Feature calculation: Calculate time-domain and frequency-domain features such as mean square error, peak value, and spectral centroid for the original signal within each window; Standardization: Using the online calculated moving mean and standard deviation, for Perform Z-score standardization to obtain the final input x; Reconstruction computation: Input x into a pre-trained autoencoder network such as an encoder. and decoder In the process, after one forward propagation of the network, the output of the decoder is the reconstructed vector. ; Loss calculation: according to the formula Calculate MSE loss; Data for online feature weight updates is collected from labeled historical data pairs. ,in For feature vectors, The data collection process for the actual labels is as follows: Model prediction: Using the current linear prediction model To predict the sample, where For the Sigmoid function; Calculate the gradient: Calculate the gradient of the loss function resulting from the current sample. For log loss, the gradient formula is: ,here It directly reflects the degree of responsibility of each feature for the prediction error in the current sample; Update weights: By substituting the complex update formula for online feature weight calculation, a new weight vector is calculated. This process is usually done automatically by the optimizer library, but in principle it achieves sparsification of high-frequency features and stable updates of important features.

[0025] The formula for unsupervised anomaly detection based on autoencoders is as follows:

[0026] in, , The input data vector represents the time... The collected equipment status snapshots may include, for example, n sensor readings such as vibration amplitude (mm / s), bearing temperature (°C), motor current (A), and noise level (dB). For encoder function (parameter is) It compresses high-dimensional input data x into a low-dimensional latent space representation, which captures the core feature patterns of the device in its normal state. For decoder function (parameter is) It attempts to reconstruct the original input data from the latent space representation; To reconstruct the output vector, which is the device state data reconstructed by the autoencoder based on the learned normal pattern; The larger the reconstruction loss (error), the greater the difference between the current true state x and the "normal mode" known to the system, and the higher the probability of an anomaly. The system can set a threshold, when... An alert is triggered at any time.

[0027] The formula for dynamically adjusting online feature weights is as follows:

[0028] in,

[0029] This is the feature weight vector at time t, where each weight component corresponds to the importance of a sensor feature, such as vibration or temperature, to the remaining service life prediction model. The weights change dynamically with time / data stream. The gradient of the loss function at time t reflects how much the weight adjustment will affect the model's prediction error under the current data sample. The learning rate is adjusted over time, controlling the step size of weight updates, and typically decays over time; , The regularization coefficient is . Promote weight sparsity, automatically select features, and ignore irrelevant sensors. To prevent excessive weighting and avoid overfitting.

[0030] The processing within the planning and learning module includes the design of the reward function for deep reinforcement learning and the optimization of the proximal policy. In the design of the reward function for deep reinforcement learning, variable acquisition includes acquiring the environment state and the components of the reward function. The environment state is acquired through multi-source information fusion as follows: Obtain from the self-learning module: Extract the device health index and real-time anomaly probability output by the self-learning module, such as the reconstruction loss. Health score normalized to zero-1; Obtain from business systems: Connect to the Enterprise Resource Planning and Manufacturing Execution System via API to read spare parts inventory, available maintenance personnel, and the priority of current production orders; State vector construction: After normalizing the above information, concatenate them into a fixed-dimensional state vector. ; For example: equipment Health score, equipment Health score, bearing inventory, maintenance team status, order priority, etc. , It is the foundation of the digital twin environment The process of obtaining each component of the reward function through environmental feedback and business records after the action is executed is as follows: direct costs :action Once triggered, the system automatically queries and summarizes data from the standard maintenance man-hour cost database and the spare parts price database; Production stoppage losses According to the action The projected downtime is calculated by multiplying the defined downtime by the standard output value per unit time of the production line. Fault indication : Monitor unplanned shutdown alarm signals from the equipment control system. If in status If a signal is received within a specified time, a huge negative constant is returned; otherwise, zero is returned. Early warning reward : Need to be recorded, if actions For devices marked as having warnings or potential malfunctions by the self-learning module, a positive reward is returned to encourage preventative intervention. In the optimization of the proximal strategy, the process of obtaining variables through a series of states, actions, and reward trajectories is as follows: Value estimation: using a value network It inputs the state. The network outputs an estimate of the long-term reward for that state, and is trained together with the policy network. Calculating the time difference error: The Generalized Dominance Estimation (GAE) method is used. The core is calculating the time difference error at each step. ,in It is a discount factor; Advantages of smooth estimation: ,in It is a discount factor; Advantages of smooth estimation: ,in It is a smoothing parameter (0~1), and this process is crucial. It obtains a more stable and less biased advantage estimate than single-step differencing by weighted averaging of multi-step returns, thereby guiding the policy to update more effectively.

[0031] The reward function is calculated using the following formula:

[0032] in, The environmental state at time t is provided by the self-learning module and can be represented as [equipment 1 health, equipment 2 health, ..., spare parts inventory, maintenance team status, production task urgency], etc. For example, the action taken by the agent at time t: =[Perform preventative maintenance on the equipment, or reduce the equipment load by 30%, and request the availability of spare bearing parts]; To perform the action The direct costs incurred include labor costs, spare parts costs, energy consumption, etc. Losses due to downtime caused by maintenance are related to the duration of maintenance and the production value of the equipment. This is a fault indication function; if it is in state... If the device experiences an unplanned shutdown, this function returns a very large penalty value, such as -1000, otherwise it returns zero; The warning reward function is used to give a positive reward when the agent takes preventive action before the device experiences an anomaly warned by the autonomous learning module, thus encouraging early intervention. , , and These are weighting coefficients, set by domain experts, used to balance the importance of the four optimization objectives: cost, downtime, failure, and early warning.

[0033] The objective function of the near-end policy optimization algorithm is calculated as follows:

[0034] in,

[0035] The parameters are the policy neural network π, which takes the current environment state as input. Output all possible maintenance actions The probability distribution on; Under the current policy with parameter θ, in state Choose action The probability of; The probability ratio represents the change in probability of the new policy relative to the old policy when choosing the same action, and is the core basis for the algorithm to update the policy. The dominance function estimate at time t quantifies the state. Select action How much better is it compared to the average action in that state? For example, This means that this maintenance decision brings higher long-term returns, such as lower total cost, compared to the average maintenance decision. This is a safety valve for pruning hyperparameters, which forces that the difference between the new and old strategies cannot be too large, ensuring the stability of the training process and avoiding a collapse due to a single bad update.

[0036] Reference Figure 2 A big data processing and analysis method based on AI (Artificial Intelligence) includes the following steps: S1: Through the self-learning module, the input data stream is continuously analyzed, the analysis model and feature representation are dynamically optimized, and automatic anomaly detection and knowledge accumulation are achieved; Furthermore, the dynamic optimization analysis model and feature representation include the following steps: Step S101: Use an online learning algorithm to update the feature weights of the input data stream in real time; Step S102: Continuously monitor the performance metrics of the prediction model. When the performance degradation exceeds a predetermined threshold, automatically trigger the hyperparameter search and retraining process of the model, and use a new model with better performance for iteration.

[0037] S2: Through the planning and learning module, high-level goals and constraints are received, and based on the environmental state information provided by the autonomous learning module, multi-step sequence decision-making is performed in the environmental simulation to generate the optimal action sequence; Furthermore, the process of multi-step sequential decision deduction performed by the planning learning module includes the following steps: Step S201: Quantize the high-level objective into a reward function in reinforcement learning; Step S202: In the digital twin environment simulator, starting from the current environmental state, conduct multi-step forward-looking exploration through the interaction between smart devices and the simulated environment, and evaluate the long-term cumulative returns of different action sequences; Step S203: Select and output the action sequence that maximizes long-term cumulative returns as the optimal action sequence.

[0038] S3: The optimal action sequence is executed through the decision execution unit, and the feedback data generated after execution is re-input into the autonomous learning module to form a data analysis closed loop; Based on the above, the AI-based big data processing and analysis method also includes a collaborative method between a self-learning module and a planning learning module, which includes the following steps: Step 1: The planning learning module generates and executes a sequence of actions to change the state of the physical or business system. The resulting new data is collected by the data perception unit as feedback and used for the continuous optimization of the self-learning module. Step 2: The knowledge base updated by the self-learning module is used to improve the simulation fidelity of the digital twin environment simulator, thereby enabling the planning and learning module to make more reliable long-term decisions.

[0039] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A big data processing and analysis system based on AI (Artificial Intelligence), comprising a data sensing unit, a core engine unit, and a decision execution unit, characterized in that, The data sensing unit includes a data acquisition module and a preprocessing module. The data acquisition module is responsible for collecting time-series data and time data from various device sensors, SCADA systems, and maintenance logs. The preprocessing module cleans and aligns the collected time-series data and time data. The core engine unit is connected to the data perception unit and includes a mutually coupled autonomous learning module and a planning learning module. The autonomous learning module includes a dynamic feature engineering model optimization submodule, an anomaly detection pattern discovery submodule, and a knowledge base, which are used to perform dynamic model optimization, anomaly detection, and knowledge accumulation from the data stream. The planning learning module includes a digital twin environment simulator, a planning decision engine, and an action sequence generation submodule, which perform sequence decision deduction based on the environmental state information provided by the autonomous learning module to generate the optimal action sequence. The decision execution unit includes an execution agent module, which receives and executes the action instruction sequence generated by the planning and learning module, and directly applies it to physical devices or business systems. At the same time, it collects new data generated after the action is executed and feeds it back to the data perception unit.

2. The big data processing and analysis system based on AI artificial intelligence according to claim 1, characterized in that, The dynamic feature engineering model optimization submodule dynamically learns and optimizes feature representations from the data stream, and automatically monitors, evaluates and updates the fault prediction model to cope with the conceptual drift of data distribution and maintain model performance. The anomaly detection mode discovery submodule uses the optimized model and features to perform real-time anomaly detection, and uses clustering algorithms to discover potential new patterns and state patterns in the data. The knowledge base stores and accumulates knowledge of abnormal patterns and new state patterns discovered by the anomaly detection pattern discovery submodule in a structured form, forming the system's experience memory.

3. The big data processing and analysis system based on AI artificial intelligence according to claim 2, characterized in that, The digital twin environment simulator constructs a virtual business environment simulation model, integrating physical laws and data-driven models. It adopts a hybrid modeling approach, using equations based on physical laws to model the deterministic degradation process of equipment, and using probability distributions learned from historical data in the knowledge base to model the stochastic process of failure, thereby extrapolating and experimenting for the future. In the simulated environment of the digital twin environment simulator, the planning and decision engine uses reinforcement learning algorithms to perform multi-step reasoning and sequential decision-making in order to find the optimal strategy to achieve a given long-term goal. The planning and decision engine takes equipment status characteristics and resource inventory status as input through a policy network and outputs maintenance, scheduling, and parameter adjustment actions. The action sequence generation submodule transforms the optimal strategy output by the planning decision engine into a detailed sequence of action instructions with a timeline.

4. The big data processing and analysis system based on AI artificial intelligence according to claim 3, characterized in that, The processing within the autonomous learning module includes unsupervised anomaly detection based on an autoencoder and online feature weight update. In unsupervised anomaly detection, data is acquired from raw time-series signals from the device's sensors, and the acquisition process is as follows: Data slicing: Slicing streaming data into segments using fixed time windows; Feature calculation: Calculate the time-domain and frequency-domain features of the original signal within each window; Standardization: Using the online calculated moving mean and standard deviation, for Perform Z-score standardization to obtain the final input x; Reconstruction calculation: Input x into the trained autoencoder network, the network propagates forward once, and the output of the decoder is the reconstruction vector. ; Loss calculation: according to the formula Calculate MSE loss; Data for online feature weight updates is collected from labeled historical data pairs. ,in For feature vectors, The data collection process for the actual labels is as follows: Model prediction: Using the current linear prediction model To predict the sample, where For the Sigmoid function; Calculate the gradient: Calculate the gradient of the loss function resulting from the current sample. For log loss, the gradient formula is: ; Update weights: By substituting the complex update formula for online feature weight calculation, a new weight vector is calculated. .

5. The big data processing and analysis system based on AI artificial intelligence according to claim 4, characterized in that, The processing within the planning and learning module includes the design of the reward function for deep reinforcement learning and the optimization of the proximal policy. In the design of the reward function for deep reinforcement learning, variable acquisition includes acquiring the environment state and the components of the reward function. The environment state is acquired through multi-source information fusion as follows: Obtain from the self-learning module: Extract the device health index and real-time anomaly probability output by the self-learning module; Obtain from business systems: Connect to the Enterprise Resource Planning and Manufacturing Execution System via API to read spare parts inventory, available maintenance personnel, and the priority of current production orders; State vector construction: After normalizing the above information, concatenate them into a fixed-dimensional state vector. ; The process of obtaining each component of the reward function through environmental feedback and business records after the action is executed is as follows: direct costs :action Once triggered, the system automatically queries and summarizes data from the standard maintenance man-hour cost database and the spare parts price database; Production stoppage losses According to the action The projected downtime is calculated by multiplying the defined downtime by the standard output value per unit time of the production line. Fault indication : Monitor unplanned shutdown alarm signals from the equipment control system. If in status If a signal is received within a specified time, a huge negative constant is returned; otherwise, zero is returned. Early warning reward : Need to be recorded, if actions For devices marked as having warnings or potential malfunctions by the self-learning module, a positive reward is returned to encourage preventative intervention. In the optimization of the proximal strategy, the process of obtaining variables through a series of states, actions, and reward trajectories is as follows: Value estimation: using a value network It inputs the state. Output an estimate of the long-term reward for that state; Calculating the time difference error: The generalized dominance estimation method is used, and the core is to calculate the time difference error at each step. ,in It is a discount factor; Advantages of smooth estimation: ,in It is a smoothing parameter (0~1).

6. A big data processing and analysis method based on AI (Artificial Intelligence), applied to the big data processing and analysis system based on AI as described in any one of claims 1-5, characterized in that, Includes the following steps: S1: Through the self-learning module, the input data stream is continuously analyzed, the analysis model and feature representation are dynamically optimized, and automatic anomaly detection and knowledge accumulation are achieved; S2: Through the planning and learning module, high-level goals and constraints are received, and based on the environmental state information provided by the autonomous learning module, multi-step sequence decision-making is performed in the environmental simulation to generate the optimal action sequence; S3: The optimal action sequence is executed through the decision execution unit, and the feedback data generated after execution is re-input into the autonomous learning module to form a data analysis closed loop.

7. The big data processing and analysis method based on AI artificial intelligence according to claim 6, characterized in that, The dynamic optimization analysis model and feature representation include the following steps: Step S101: Use an online learning algorithm to update the feature weights of the input data stream in real time; Step S102: Continuously monitor the performance metrics of the prediction model. When the performance degradation exceeds a predetermined threshold, automatically trigger the hyperparameter search and retraining process of the model, and use a new model with better performance for iteration.

8. The big data processing and analysis method based on AI artificial intelligence according to claim 6, characterized in that, The planning learning module performs multi-step sequential decision inference, which includes the following steps: Step S201: Quantize the high-level objective into a reward function in reinforcement learning; Step S202: In the digital twin environment simulator, starting from the current environmental state, conduct multi-step forward-looking exploration through the interaction between smart devices and the simulated environment, and evaluate the long-term cumulative returns of different action sequences; Step S203: Select and output the action sequence that maximizes long-term cumulative returns as the optimal action sequence.

9. A big data processing and analysis method based on AI artificial intelligence according to claim 6, characterized in that, It also includes a collaborative method for the self-directed learning module and the planned learning module, the collaborative method comprising the following steps: Step 1: The planning learning module generates and executes a sequence of actions to change the state of the physical or business system. The resulting new data is collected by the data perception unit as feedback and used for the continuous optimization of the self-learning module. Step 2: The knowledge base updated by the self-learning module is used to improve the simulation fidelity of the digital twin environment simulator, thereby enabling the planning and learning module to make more reliable long-term decisions.

Images