An RPA business processing method and system based on artificial intelligence

Abstract

The application provides an RPA business processing method and system based on artificial intelligence, which comprises the following steps: using an RPA system to collect text data and control camera image data of hydropower plant equipment, using a sensor-based data collection module to collect sensor data, and writing the preprocessed data into a database through the RPA system; feature extraction and feature fusion; constructing a device relationship graph and monitoring and abnormality detection of the state of the hydropower plant equipment, and predicting the state; constructing a cross-site data collaborative analysis model to integrate different site data, generating global optimization suggestions and operation instructions; constructing a detection model to cover abnormal situations, and realizing all-weather automatic monitoring and abnormality processing through the RPA system. The application provides a hydropower plant business processing method and system based on artificial intelligence and RPA, which comprehensively improves the business processing efficiency and intelligent level of the hydropower plant through multi-modal information fusion, graph neural network, federated learning and self-supervised learning technology.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of artificial intelligence, and particularly relates to an RPA business processing method and system based on artificial intelligence. BACKGROUND

[0002] In the operation and maintenance process of modern hydropower plants, the demand for automation and intelligence is increasing. However, the existing technology still has many shortcomings in processing complex operation data, equipment state monitoring, set value management and anomaly detection of hydropower plants. The traditional hydropower plant business processing system mainly relies on manual operation, which is low in efficiency and prone to human error. Especially when dealing with a large amount of multi-modal data (such as text, images, sensor data), the existing technology lacks effective fusion and analysis methods, resulting in a serious information island problem and making it difficult to achieve comprehensive situation assessment.

[0003] The existing equipment state monitoring system is usually based on a single data source, which is difficult to fully capture the complex relationships and dependencies between devices, and cannot effectively predict and prevent equipment failures. The traditional method mainly uses preset rules and simple statistical analysis, which lacks dynamic adaptability and cannot timely reflect the changes of equipment state. At the same time, the existing set value management system mostly relies on manual operation and paper documents, which is low in efficiency and prone to error, and lacks intelligent management means. In addition, cross-site data collaborative analysis has not been widely applied, and the existing technology has obvious shortcomings in data privacy protection and efficient collaborative analysis, making it difficult to realize information sharing and optimization collaboration between sites.

[0004] The anomaly detection and early warning system is usually based on preset thresholds and rules, which cannot adapt to the changing operating environment and is prone to false positives or false negatives. The traditional anomaly detection system lacks self-learning and adaptive ability, and cannot dynamically adjust the detection model to cope with new abnormal patterns, resulting in insufficient reliability and accuracy of the system.

[0005] In summary, the existing technology has obvious shortcomings in multi-modal data processing, equipment state monitoring, set value management, data collaborative analysis and anomaly detection, and an innovative technical solution is urgently needed to overcome these problems. At the same time, the existing hydropower plant business processing system has obvious shortcomings in automation. Although the robot process automation (RPA) technology has been applied in some fields, its application in hydropower plant business processing is still limited. The advantage of RPA technology is that it can simulate manual operation and achieve efficient and accurate process automation, but in terms of complex multi-source data fusion analysis, equipment state monitoring and anomaly detection, the existing RPA system still lacks a comprehensive solution combined with advanced AI technology.

[0006] Therefore, there is a need for a hydropower plant business processing method and system that combines advanced artificial intelligence algorithms and RPA technology to achieve comprehensive automation and intelligentization, overcome the shortcomings of existing technologies, and improve the operational efficiency and safety of hydropower plants. SUMMARY

[0007] The purpose of the present application is to provide an RPA business processing method and system based on artificial intelligence, which comprehensively improves the business processing efficiency and intelligentization level of hydropower plants through multi-modal information fusion, graph neural networks, federated learning, and self-supervised learning techniques.

[0008] To achieve the above-mentioned purpose, the present application provides an RPA business processing method based on artificial intelligence, which comprises:

[0009] S1, using an RPA system to collect text data and control camera image data of hydropower plant equipment, using a sensor-based data collection module to collect sensor data, and preprocessing the text data, image data, and sensor data, and writing the preprocessed text data, image data, and sensor data into a database through the RPA system;

[0010] S2, respectively extracting features from the preprocessed text data, image data, and sensor data and performing feature fusion to obtain a multi-modal feature vector;

[0011] S3, constructing a device relationship graph according to the multi-modal feature vector and an analysis method based on a multi-layer adaptive time sequence model to monitor and detect abnormalities in the state of the monitored hydropower plant equipment, while predicting the state of the hydropower plant equipment;

[0012] S4, constructing a cross-site data collaborative analysis model based on federated learning to integrate different site data, automatically processing the analysis results to generate global optimization suggestions and operation instructions; wherein the objective optimization function of the cross-site data collaborative analysis model is represented as follows:

[0013]

[0014] wherein J represents the total optimization objective function, C i (u i ) represents the operating cost function of site i, g i (u i ) represents the operating constraint condition of site i, u i represents the operating parameter of site i, N represents the total number of sites, and alpha and beta are the weight coefficients of cost and emission, respectively, E i (u i ) represents the emission control function of site i, h i (u it) represents the dynamic constraint condition at time t, and delta represents the uncertainty factor i ;

[0015] S5, using a self-supervised learning technique to construct a detection model, covering abnormal conditions, and realizing all-weather automatic monitoring and abnormal processing through an RPA system;

[0016] The detection model includes a generative contrast learning submodel, an anomaly detection submodel, a multi-modal self-supervised submodel, a time series self-supervised learning submodel, and a frequency domain self-supervised learning submodel.

[0017] The contrast learning submodel is trained by constructing a positive sample and a negative sample pair, and the feature vector is F i , the positive sample is , and the negative sample is The generation formula of the contrast learning model is:

[0018]

[0019] Where augment represents a data augmentation operation, and shuffle represents a random shuffling operation.

[0020] Define the contrast loss function L for training the generative contrast learning model, and the loss function L formula is as follows:

[0021]

[0022] Where d(F i ,F j ) represents the Euclidean distance measurement between feature vectors, m represents the boundary value, represents the Euclidean distance measurement between the feature vector and the positive sample, represents the Euclidean distance measurement between the feature vector and the negative sample.

[0023] The anomaly detection submodel, let the anomaly detection model be d s , and the detection result be y s , as follows:

[0024] y s =d s (F i ),

[0025] Where d s represents the anomaly detection model, y s represents the detection result, and the anomaly detection model d s is trained based on the representation of self-supervised feature learning, combined with support vector machines for classification, as follows:

[0026] y s=sign(w·F i +b),

[0027] Where w represents the weight vector, b represents the bias, and sign represents the sign function, used to return the sign of the input. That is, if the input is greater than 0, it returns 1; if the input is less than 0, it returns -1; if the input is equal to 0, it returns 0.

[0028] The multimodal self-supervised sub-model combines multimodal data and utilizes self-supervised learning for anomaly detection. Let the multimodal feature vector be M. i The self-supervised model is d m The detection result of the multimodal self-supervised model is y. m , means as follows:

[0029]

[0030] Among them, M i This represents a multimodal feature vector containing text features t. i Image features i i and sensor features i ;d m This represents a multimodal self-supervised model;

[0031] The temporal self-supervised learning sub-model employs a temporal self-supervised learning method to achieve anomaly detection by predicting the feature vector of the next time step. Let F be the feature vector of the current time step. i The predicted feature vector for the next time step is The prediction model is d t The loss function L of the time-series self-supervised learning model t For, it is represented as follows:

[0032]

[0033] in, F i+1 The feature vector representing the actual next time step, and the prediction model d. t Based on LSTM, it is represented as follows:

[0034]

[0035] Among them, h i Indicates the current hidden state;

[0036] The frequency domain self-supervised learning sub-model extracts frequency domain features for anomaly detection using Fourier transform. Let the frequency domain feature vector be... The frequency domain self-supervised learning model is d f The detection result of the frequency domain self-supervised learning model is y. f, is represented as follows:

[0037]

[0038] wherein, denotes a Fourier transform operation, denotes a frequency domain feature vector, a frequency domain self-supervised learning model d f Anomaly detection is performed in combination with the frequency domain features and the spatial features.

[0039] Further, the preprocessing includes data cleaning and data standardization.

[0040] Further, the text feature extraction adopts a bag-of-words model and TF-IDF weighting processing to extract text features from the operation reports and the maintenance records to obtain a text feature vector; the image feature adopts a convolutional neural network to perform feature extraction on image data to obtain an image feature vector; and the sensor feature adopts a wavelet transform to perform time series feature extraction on sensor data to obtain a sensor feature vector.

[0041] Further, the text features, the image features, and the sensor features are weighted and fused, specifically including:

[0042] The feature vectors are divided into a plurality of sub-regions, and the correlation between each sub-region is calculated, denoted as t i ,i i ,s i are respectively a text feature vector, an image feature vector, and a sensor feature vector, which are respectively divided into k sub-regions, and a local correlation matrix R ij is calculated, and is represented as follows:

[0043]

[0044] wherein, denotes the correlation between the jth local region of the ith and jth modal, t i,k and i j,k denote the kth sub-region of the text and image feature vectors, respectively, and are respectively the mean of the sub-region;

[0045] Let an initial weight vector be w=[w t ,w i ,w s ], which corresponds to the global weight of the text features, the image features, and the sensor features, and the weight is dynamically adjusted using the local correlation matrix R, and is represented as follows:

[0046]

[0047] wherein, the adjusted local weight satisfies w′ t,k+w′ i,k +w′ s,k =1; Within each sub-region, dynamically adjusted local weights are applied for weighted fusion to obtain the weighted fused feature vector. It is expressed as follows:

[0048]

[0049] in, w′ is the weighted fusion feature vector of the i-th data point in the k-th sub-region. t,k w′ i,k w′ s,k These are the local weights for text features, image features, and sensor features, respectively. These represent the text feature vector, image feature vector, and sensor feature vector within the k-th sub-region, respectively.

[0050] Furthermore, the multimodal feature vectors construct a device relationship graph, represented as follows:

[0051] Based on the physical connections and dependencies between the equipment in the hydropower plant, construct an equipment relationship diagram. The equipment set is ε = {E1, E2, ..., E...} N}, where E i Indicates the i-th device;

[0052] Define the edge set between devices Among them, each edge (E) i E j ) indicates device E i and E j There is a direct relationship between them;

[0053] Constructing a device relationship diagram

[0054] The device relationship graph uses multimodal feature vectors as the initial feature vectors for each device, and a new weighted dynamic feature propagation algorithm is used to propagate features from the device relationship graph, updating the feature representation of each device. The new weighted dynamic feature propagation algorithm defines the feature propagation formula for the l-th layer as follows:

[0055]

[0056] in, This represents the feature vector of the i-th device in the l-th layer. Indicates device E i The neighborhood group, and W represents the connection weight and historical state weight between devices, respectively. (l)W represents the weight matrix of the i-th layer, σ represents the activation function, j represents, W represents the weight matrix of the i-th layer, σ represents the activation function, j represents,

[0057] Further, the analysis method based on the multi-layer adaptive time sequence model specifically comprises:

[0058] The feature vector is input into the encoder part of the variational autoencoder to obtain the hidden variable z t :

[0059] z t = μ + σ ⊙ ∈,

[0060] wherein, μ represents the mean, σ represents the standard deviation, ε represents a random noise variable, f μ and f σ respectively represent neural network functions.

[0061] The encoded hidden variable z t is input into the LSTM network for time sequence modeling, and the update formula of the LSTM network is represented as follows:

[0062]

[0063] wherein, f t , i t , o t respectively represent activation vectors of a forgetting gate, an input gate and an output gate, c t represents a cell state vector, W f , W i , W c , W o represent weight matrices, b f , b i , b c , b o represent bias vectors, σ represents a Sigmoid activation function, ⊙ represents element multiplication, h t represents a time sequence feature vector at time step t.

[0064] The output h t of the LSTM network is used for state prediction, a prediction function f pred is defined, and the calculation is as follows:

[0065]

[0066] wherein, h pred represents a predicted state at time step t, f pred represents a fully connected layer.

[0067] Further, the S4 specifically comprises:

[0068] S41, standardizing and aligning the data of different sites;

[0069] S42, calculating the linear correlation of the data of two sites using Pearson correlation coefficient, and calculating the mutual information between sites to measure the nonlinear correlation of the data of two sites;

[0070] S43, constructing a physical model;

[0071] S44, performing weighted average fusion on the cross-site data, and then performing dimensionality reduction on the fused data using principal component analysis method;

[0072] S45, using the cross-site fusion data for collaborative fault diagnosis to detect potential faults of the equipment, and making optimization scheduling decisions based on the collaborative analysis results to adjust the operation parameters of each site.

[0073] Further, the physical model comprises a hydropower plant energy balance model and a water flow balance model;

[0074] The hydropower plant energy balance model is constructed as follows:

[0075] Let the energy input of site i be E in,i (t), the energy output be E out,i (t), and the energy storage be E s,i (t), and the energy balance equation be represented as follows:

[0076] E in,i (t) = E out,i (t) + ΔE s,i (t),

[0077] Where ΔE s,i (t) represents the change in energy storage at time step t;

[0078] The water flow balance model is constructed as follows:

[0079] Let the water inflow of site i be Q in,i (t), the water outflow be Q out,i (t), and the water storage be Q s,i (t), and the water balance equation be represented as follows:

[0080] Q in,i (t) = Q out,i (t) + ΔQ s,i (t),

[0081] Where ΔQ s,i(t) represents the change in water storage at time step t.

[0082] Further, the detection results of each sub-model are integrated to obtain a final abnormality determination result:

[0083] Let the integrated detection result be D i :

[0084]

[0085] where w k represents the weight of the sub-model, d k (F i ) represents the detection result of the kth sub-model; if D i exceeds a set threshold θ, it is determined to be abnormal:

[0086]

[0087] where A i represents the abnormality determination result, and θ represents the abnormality threshold of the integrated detection.

[0088] In a second aspect of the present application, an RPA business processing system based on artificial intelligence is provided, which comprises:

[0089] A data acquisition module is used to collect and control camera image data using an RPA system for hydropower plant equipment, and sensor data is collected using a sensor-based data acquisition module, and the text data, image data and sensor data are preprocessed, and the preprocessed text data, image data and sensor data are written into a database through an RPA system;

[0090] A feature fusion module is used to extract features from the preprocessed text data, image data and sensor data respectively and perform feature fusion to obtain a multi-modal feature vector;

[0091] A state prediction module is used to construct a device relationship graph according to the multi-modal feature vector and an analysis method based on a multi-layer adaptive time sequence model to monitor and detect abnormalities of the monitored hydropower plant equipment, and to predict the state of the hydropower plant equipment;

[0092] A cross-site data integration module is used to construct a cross-site data collaborative analysis model based on federated learning to integrate different site data, and to automatically process the analysis results to generate global optimization suggestions and operation instructions; wherein the objective optimization function of the cross-site data collaborative analysis model is represented as follows:

[0093]

[0094] where J represents the total optimization objective function, Ci (u i ) represents the operating cost function of site i, g i (u i ) represents the operating constraints of site i, u i represents the operating parameters of site i, N represents the total number of sites, and α and β are the weight coefficients of cost and emission, respectively, E i (u i ) represents the emission control function of site i, h i (u i ,t) represents the dynamic constraints at time t, and Δ represents the range of values of the uncertainty factor δ i .

[0095] The detection model construction module is configured to construct a detection model using a self-supervised learning technique, cover abnormal situations, and realize all-weather automatic monitoring and abnormal processing through an RPA system.

[0096] The detection model includes a generative contrast learning sub-model, an anomaly detection sub-model, a multi-modal self-supervised sub-model, a time series self-supervised learning sub-model, and a frequency domain self-supervised learning sub-model.

[0097] The contrast learning sub-model is trained by constructing a positive sample and a negative sample pair, where the feature vector is F i , the positive sample is , and the negative sample is The generation formula of the contrast learning model is as follows:

[0098]

[0099] augment represents a data augmentation operation, and shuffle represents a random shuffling operation.

[0100] A contrast loss function L is defined for training the generative contrast learning model, and the loss function L formula is as follows:

[0101]

[0102] where d(F i ,F j ) represents the Euclidean distance measurement between feature vectors, m represents the boundary value, represents the Euclidean distance measurement between the feature vector and the positive sample, represents the Euclidean distance measurement between the feature vector and the negative sample.

[0103] The anomaly detection sub-model, where the anomaly detection model is d s , and the detection result is y s , is represented as follows:

[0104] ys = d s (F i ),

[0105] wherein, d s represents an anomaly detection model, y s represents a detection result, the anomaly detection model d s is trained based on a representation of self-supervised feature learning, combined with a support vector machine for classification, represented as follows:

[0106] y s = sign(w·F i +b),

[0107] wherein, w represents a weight vector, b represents a bias, and sign represents a sign function, which returns the sign of the input. That is, if the input is greater than 0, return 1; if the input is less than 0, return -1; if the input is equal to 0, return 0;

[0108] The multi-modal self-supervised sub-model combines multi-modal data and uses self-supervised learning for anomaly detection. Let the multi-modal feature vector be M i , the self-supervised model be d m , and the detection result of the multi-modal self-supervised model be y m , represented as follows:

[0109]

[0110] wherein, M i represents a multi-modal feature vector, including text features t i , image features i i , and sensor features s i ; d m represents a multi-modal self-supervised model;

[0111] The time series self-supervised learning sub-model uses a time series self-supervised learning method to achieve anomaly detection by predicting the feature vector of the next time step. Let the feature vector of the current time step be F i , and the predicted feature vector of the next time step be The prediction model is d t , and the loss function L t of the time series self-supervised learning model is represented as follows:

[0112]

[0113] wherein, F i+1 represents the actual feature vector of the next time step, and the prediction model d t is implemented based on LSTM, represented as follows:

[0114]

[0115] wherein h i represents the current hidden state;

[0116] The frequency domain self-supervised learning sub-model extracts frequency domain features through Fourier transform for anomaly detection, and the frequency domain feature vector is The frequency domain self-supervised learning model is d f , and the detection result of the frequency domain self-supervised learning model is y f , which is represented as follows:

[0117]

[0118] wherein, represents the Fourier transform operation, represents the frequency domain feature vector, and the frequency domain self-supervised learning model d f Combining frequency domain features and spatial features for anomaly detection.

[0119] The beneficial technical effects of the present application are at least as follows:

[0120] (1) The present application utilizes deep learning technology, especially the Transformer model, to fuse and process data from multiple modalities such as text, images, and sensors. Through the RPA system, these data are automatically acquired and processed, realizing comprehensive fusion and comprehensive analysis of information, overcoming the problems of information silos and low processing efficiency in traditional methods.

[0121] (2) The present application adopts graph neural network (GNN) technology to construct a relationship graph between devices and analyze the dependency and influence path between devices. The RPA system automatically generates and executes maintenance strategies and operation plans based on the GNN analysis results, effectively solving the problem of poor dynamic adaptability of traditional single data source monitoring and device state, and improving the comprehensiveness and prediction ability of device state monitoring.

[0122] (3) The present application realizes collaborative analysis of data among multiple hydropower plant sites through federated learning technology, ensuring the privacy and security of data at each site. The RPA system automatically processes the analysis results to generate global optimization suggestions and operation instructions, overcoming the shortcomings of existing technologies in data privacy protection and cross-site collaborative analysis, and realizing efficient information sharing and optimization collaboration.

[0123] (4) The present application utilizes self-supervised learning technology to train an anomaly detection model, covering more abnormal situations, and realizes all-weather automatic monitoring and abnormal processing through the RPA system. Self-supervised learning technology enables the anomaly detection model to adaptively adjust, overcoming the limitations of traditional preset thresholds and rules, and improving the accuracy and reliability of anomaly detection.

[0124] (5) The present application not only solves many deficiencies of the prior art in multi-modal data processing, equipment state monitoring, fixed value management, data collaborative analysis and anomaly detection, but also significantly improves the automation and intelligence level of hydropower plant business processing, and has wide application prospect and significant practical benefits. BRIEF DESCRIPTION OF DRAWINGS

[0125] The application will be further described with the accompanying drawings, but the embodiments in the drawings do not constitute any limitation on the application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of the following drawings.

[0126] Figure 1 A flowchart of the RPA business processing method based on artificial intelligence of the present application.

[0127] Figure 2 A framework diagram of the RPA business processing system based on artificial intelligence of the present application. DETAILED DESCRIPTION

[0128] The embodiments of the present application will be described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary and are only used to explain the present application, and cannot be understood as a limitation on the present application.

[0129] As shown in Figure 1 The RPA business processing method based on artificial intelligence provided by the embodiments of the present application comprises the following steps S1-S5:

[0130] S1, using an RPA system to collect text data and control camera to collect image data of hydropower plant equipment, using a sensor-based data collection module to collect sensor data, and preprocessing the text data, image data and sensor data, and writing the preprocessed text data, image data and sensor data into a database through the RPA system.

[0131] Among them, the text data is extracted from the operation report and the maintenance record. The RPA robot is used to automatically log in to the system, grab the text data, and store it in a unified data warehouse. The text data usually includes equipment operation status, fault report, maintenance record, etc., which records the daily operation information of the hydropower plant.

[0132] An industrial camera and an image sensor are used to take state images of the equipment at regular intervals. The RPA system controls the camera to take pictures and transmits the image data to the data warehouse. The image data can be used to detect the physical state of the equipment, such as whether there is appearance damage or abnormality.

[0133] Temperature, pressure, vibration, and other sensor data are automatically collected using a Data Acquisition (DAQ) module. Through industrial protocols such as Modbus, data is automatically transmitted to a data warehouse. Sensor data reflects the operating parameters of the equipment and can be used to monitor the health status of the equipment.

[0134] Further, noise filtering is performed on the sensor data, and a Moving Average method is used to filter high-frequency noise. The formula is as follows:

[0135]

[0136] where y i is the filtered data, x i is the original data, and n is the window size. The window size n is usually determined according to the data sampling frequency and noise characteristics.

[0137] Missing values in the sensor data are filled using the Linear Interpolation method.

[0138] The formula is as follows:

[0139]

[0140] where x i is the interpolated data, t i is the position of the missing value, t k and t k+1 are the positions of the adjacent known data points.

[0141] At the same time, the text data is normalized, and different formats of text are standardized to a unified format. Regular Expressions are used to extract key information such as device ID, timestamp, status description, etc.

[0142] The data normalization process is as follows:

[0143] The sensor data is standardized using the Z-score method, converting it to a standard normal distribution with a mean of 0 and a standard deviation of 1. The formula is as follows:

[0144]

[0145] where z i is the standardized data, x i is the original data, μ is the data mean, and σ is the data standard deviation.

[0146] Min-Max normalization is performed on the image data to scale the data range to [0, 1]. The formula is as follows:

[0147]

[0148] where x' is the normalized data, x is the original data, x min and x max are the minimum and maximum values of the data, respectively.

[0149] Finally, data storage is performed, and a unified data table structure is designed, including fields such as data source, timestamp, data type, and value, to ensure data accessibility and consistency.

[0150] Database table structure example:

[0151] id: unique identifier

[0152] source: data source (e.g., sensor, image, text)

[0153] timestamp: data collection time

[0154] data_type: data type (e.g., temperature, pressure, image)

[0155] value: data value

[0156] The preprocessed data is automatically written to the database by the RPA system. Preprocessing includes noise filtering, missing value filling, and standardized data, which are stored in the database according to the designed table structure.

[0157] S2, respectively, on the preprocessed text data, image data and sensor data, feature extraction and feature fusion are carried out, and multi-modal feature vectors are obtained.

[0158] In order to effectively fuse different types of data (such as text data, image data and sensor data) together to provide comprehensive equipment state evaluation and anomaly detection, the present application proposes a multi-modal information fusion method. It is specially designed for the characteristics of hydropower plant business data to ensure that various data can complement each other and provide more accurate analysis and decision support.

[0159] Specifically, the Bag-of-Words (BoW) and TF-IDF (Term Frequency-Inverse Document Frequency) weighting processing are used to extract text features from the operation report and maintenance record, and the text feature vector t i is obtained, which is represented as follows:

[0160] t i = [ti1 • tf-idf i1 t i2 • tf-idf i2 ,…,t im • tf-idf im ] (1),

[0161] where t i is the feature vector of the i-th text data, t ij denotes the frequency of the j-th word in the i-th text, tf-idf ij is the TF-IDF weight of the word, and m is the size of the vocabulary.

[0162] The image data is feature-extracted using a convolutional neural network (such as ResNet-50) to obtain an image feature vector i i :

[0163] i i = [i i1 , i i2 , …, i in ] (2),

[0164] where i i is the feature vector of the i-th image, i ij denotes the j-th feature extracted from the i-th image, and n is the dimension of the feature vector.

[0165] The sensor data is time-series feature-extracted using a wavelet transform (WT) to obtain a sensor feature vector s i :

[0166] s i = [s i1 , s i2 , …, s ip ] (3),

[0167] where s i is the feature vector of the i-th sensor record, s ij denotes the j-th time-frequency domain feature of the i-th sensor record, and p is the dimension of the time-frequency domain feature.

[0168] Further, in order to more effectively combine data of different modalities, the present application proposes a feature fusion method based on multi-scale adaptive weighting and local correlation analysis, which not only considers the global weight of each modality, but also dynamically adjusts using the correlation of local regions, thereby optimizing the fusion effect, as follows:

[0169] The feature vectors of different modalities are concatenated to construct a multi-modal feature matrix F i :

[0170] F i =[t i ; i i ; s i ] (4),

[0171] where F i is the multi-modal feature matrix of the i-th data point, t i is the text feature vector, i i is the image feature vector, s i is the sensor feature vector, and denotes the concatenation of feature vectors.

[0172] Then, in order to consider the local correlation between different modalities more carefully, the feature vectors are divided into several sub-regions, and the correlation between each sub-region is calculated. Let t i , i i , and s i be the feature vectors of text, image, and sensor, respectively, the application divides them into k sub-regions and calculates the local correlation matrix R ij . The formula for calculating the local correlation is:

[0173]

[0174] where r denotes the correlation between the k-th local region of the i-th and j-th modalities, t i,k and i j,k are the k-th sub-region of the text and image feature vectors, respectively, and are the mean of the sub-region, respectively.

[0175] At the same time, based on the multi-scale adaptive weighting method, the weight of each modal feature vector is dynamically adjusted, so as to optimize the fusion effect. Let the initial weight vector be w = [w t , w i , w s ], which corresponds to the global weight of the text, image, and sensor features. The local correlation matrix R is used to dynamically adjust the weight:

[0176]

[0177] The adjusted local weight satisfies w′ t,k + w′ i,k + w′ s,k = 1.

[0178] In each sub-region, the dynamically adjusted local weight is applied for weighted fusion to obtain the weighted fused feature vector

[0179]

[0180] where, is the weighted fusion feature vector of the i-th data point in the k-th sub-region, w′ t,k ,w′ i,k ,w′ s,k are the local weights of text, image and sensor features respectively.

[0181] Further, the fused feature vector f i is subjected to Min-Max normalization to ensure that the features are within the same scale range.

[0182]

[0183] where f′ ij is the normalized feature, f ij is the original feature, f min and f max are the minimum and maximum values in the feature vector respectively.

[0184] Then, the final multi-modal feature vector M i is constructed, which is used for subsequent device state monitoring and analysis.

[0185] M i = f′ i (11),

[0186] where M i is the final multi-modal feature vector of the i-th data point, and f′ i is the normalized fused feature vector.

[0187] Through the above detailed steps, the present application can effectively fuse different modal data (text, image and sensor data) together to form a unified multi-modal feature vector M i . This fusion method fully considers the characteristics of hydropower plant data and can provide more comprehensive and accurate device state evaluation, providing a reliable data basis for subsequent device state monitoring and anomaly detection.

[0188] S3, according to the multi-modal feature vector, a device relationship graph is constructed and an analysis method based on a multi-layer adaptive time series model is used to monitor and detect anomalies in the monitored hydropower plant device state, while predicting the state of the hydropower plant device.

[0189] GNN technology can comprehensively analyze the complex relationships and dependencies between devices, automatically generate and execute maintenance strategies and operation plans through the RPA system, and improve the comprehensiveness and prediction ability of device state monitoring.

[0190] To accurately monitor and analyze the equipment status of hydropower plants, this invention proposes a status monitoring and analysis method based on equipment relationship diagrams and dynamic feature propagation. This method combines the complex relationships and dynamic changes among hydropower plant equipment to ensure timely and accurate monitoring of equipment status and to predict and warn of potential anomalies.

[0191] Specifically, an equipment relationship diagram is constructed based on the physical connections and dependencies between the equipment in the hydropower plant. The equipment set is ε = {E1, E2, ..., E...} N}, where E i Indicates the i-th device;

[0192] Define the edge set between devices Each edge (E) i E i ) indicates device E i and E j There is a direct relationship between them;

[0193] Constructing a device relationship diagram

[0194] Then, the feature vector is initialized using the multimodal feature vector M obtained in S2. i As the initial feature vector for each device, let... Let be the initial feature vector of the i-th device.

[0195] Furthermore, a novel weighted dynamic feature propagation algorithm is used to propagate features from the device relationship graph, updating the feature representation of each device. This algorithm combines connection weights between devices and historical state information to ensure the dynamic adaptability of feature propagation. The feature propagation formula for the l-th layer is defined as follows:

[0196]

[0197] in, This represents the feature vector of the i-th device in the l-th layer. For device E i The neighborhood group, and These are the connection weights and historical state weights between devices, respectively. (l) Let σ be the weight matrix of the l-th layer, and σ be the activation function (such as ReLU).

[0198] To more accurately analyze the time-series characteristics of hydropower plant equipment, this invention proposes an analysis method based on a multi-layer adaptive time-series model. This method utilizes the multimodal feature vectors of the equipment and combines a variational autoencoder (VAE) and a long short-term memory network (LSTM) to achieve deep feature extraction and prediction of time-series data.

[0199] Furthermore, for each device's feature vector at time step t Encoding is then performed. First, the feature vector is input into the encoder part of the variational autoencoder (VAE) to obtain the latent variable z. t :

[0200] z t =μ+σ⊙∈ (13),

[0201] in, It is the mean. It is the standard deviation. It is a random noise variable, f μ and f σ These are neural network functions.

[0202] The encoded latent variable z t The data is input into an LSTM network for time series modeling. The update formula for the LSTM is:

[0203]

[0204] Among them, f t i t o t c represents the activation vectors for the forget gate, input gate, and output gate, respectively. t W is the cell state vector. f W i W c W o Let b be the weight matrix. f b i b c b o Let h be the bias vector, σ be the sigmoid activation function, ⊙ be the element-wise product, and h be the bias vector. t Let be the temporal feature vector at time step t.

[0205] Using the output h of LSTM t To perform state prediction, define the prediction function f. pred for:

[0206]

[0207] in, For the predicted state at time step t, f pred It is a fully connected layer.

[0208] Through the above steps, the application uses the device relationship graph and the dynamic feature propagation algorithm to propagate features, combines a multi-layer adaptive time sequence model to analyze time sequence features, and realizes accurate monitoring and abnormal detection of the device state. This method fully considers the complex relationship and dynamic change between devices, can timely and accurately monitor the device state, and predicts and warns potential abnormalities, providing reliable protection for the safe operation of the hydropower plant.

[0209] S4, a cross-site data collaborative analysis model based on federated learning is constructed to integrate different site data, and global optimization suggestions and operation instructions are generated by automatically processing analysis results.

[0210] Specifically, in order to realize data collaborative analysis between stations of a hydropower plant, the application proposes a method based on a physical model and statistical analysis. This method is specially designed for the characteristics of multi-site data of a hydropower plant, ensuring that the data of different stations can be effectively integrated and comprehensively analyzed to improve the safety and reliability of overall operation.

[0211] Specifically, S41 integrates cross-site data:

[0212] The data of different stations is standardized to ensure that the data is analyzed on the same scale. The Z-score standardization method is used:

[0213]

[0214] Where z ij is the standardized data, x ij is the original data, μ j is the mean of the data, σ j is the standard deviation of the data, i is the data sample, and j is the feature dimension.

[0215] Align the data of different stations on the time axis. Let the data of station A and station B be X A (t) and X B (t), respectively, and the aligned data is represented as X(t):

[0216] X(t)={X A (t),X B (t)} (17),

[0217] Where X(t) is the aligned data set, containing the data of station A and station B at time step t.

[0218] S42, calculate the cross-site correlation analysis:

[0219] Correlation between sites is calculated, using Pearson correlation coefficient to measure linear correlation between data of two sites. Let the feature vector of site A and site B be X A and X B , the correlation coefficient r AB is calculated as:

[0220]

[0221] where X A,i and X B,i are the data of site A and site B at time step i, and are the mean of data of each site, and n is the number of data samples.

[0222] Further, mutual information between sites is calculated to measure nonlinear correlation between data of two sites. Let the joint probability distribution of site A and site B be P(A,B), and the marginal probability distribution be P(A) and P(B), the mutual information I(A;B) is calculated as:

[0223]

[0224] where a and b are the values of site A and site B, P(a,b) is the joint probability, and P(a) and P(b) are the marginal probabilities.

[0225] S43, physical model establishment:

[0226] A physical model based on energy balance is established, considering energy input, output and storage of each site. Let the energy input of site i be E in,i (t), the energy output be E out,i (t), and the energy storage be E s,i (t), the energy balance equation is:

[0227] E in,i (t) = E out,i (t) + ΔE s,i (t) (20),

[0228] where ΔE s,i (t) is the change of energy storage at time step t.

[0229] A water flow balance model is established, considering water inflow, outflow and storage of each site. Let the water inflow of site i be Q in,i (t), the water outflow be Q out,i (t), and the water storage be Q s,i (t), the water balance equation is:

[0230] Q in,i (t) = Qout,i (t) + AQ s,i (t) (21),

[0231] where AQ s,i (t) is the change of water storage at time step t.

[0232] S44, cross-site data fusion is performed:

[0233] The cross-site data is fused by weighted average. Let the feature vectors of site A and site B be X A and X B , respectively, and the fused feature vector X AB is calculated as:

[0234] X AB = w A X A + w B X B (22),

[0235] where w A and w B are the weight coefficients of site A and site B, satisfying w A + w B = 1.

[0236] The fused data is reduced in dimension using the principal component analysis method (PCA) to extract the main features. Let the fused data matrix be X, and the calculation formula of PCA is:

[0237] X' = XW (23)

[0238] where X' is the reduced data matrix, and W is the eigenvector matrix representing the direction of the principal component.

[0239] S45, collaborative analysis and decision-making:

[0240] The cross-site fused data is used for collaborative fault diagnosis to detect potential faults of the equipment. Let the diagnostic model be f diag , and the fault indicator be y diag , and the calculation formula is:

[0241] y diag = f diag (X') (24),

[0242] where X' is the reduced feature vector, f diag is the fault diagnosis model, and y diag is the fault indicator.

[0243] To enhance the complexity of the model, the present application further introduces high-order feature interaction and nonlinear transformation to capture complex failure patterns. Let the high-order feature interaction be X", the calculation formula of the fault diagnosis model is extended as:

[0244]

[0245] where φ is a nonlinear feature transformation function, such as a polynomial transformation or a kernel method, and X" is the transformed high-order feature vector.

[0246] Based on the results of collaborative analysis, make optimization scheduling decisions and adjust the operating parameters of each site. Let the operating parameters of site i be u i , the optimization objective function be J, and the mathematical expression of the optimization problem be:

[0247]

[0248] where J is the total optimization objective function, C i (u i ) is the operating cost function of site i, g i (u i ) is the operating constraint condition of site i, u i is the operating parameter of site i, and N is the total number of sites.

[0249] To increase the complexity of the optimization model, consider introducing multi-objective optimization and dynamic constraints. Let the multi-objective optimization function be J, containing two objectives of operating cost and emission control, and the dynamic constraint be h i (u i ,t), then the mathematical expression of the optimization problem is:

[0250]

[0251] where E i (u i ) is the emission control function of site i, α and β are the weight coefficients of cost and emission respectively, and h i (u i ,t) is the dynamic constraint condition at time t.

[0252] In addition, to consider the robustness of the decision, introduce the uncertainty factor δ i , and the optimization problem is further extended to a robust optimization model:

[0253]

[0254] where Δ is the value range of the uncertainty factor δ i .

[0255] Through the above steps, the present invention can realize collaborative data analysis between various sites of a hydropower plant, and use physical models and statistical methods to comprehensively analyze and optimize cross-site data, thereby improving the safety and reliability of the overall operation of the hydropower plant.

[0256] S5. Utilize self-supervised learning technology to build a detection model that covers abnormal situations, and use the RPA system to achieve 24 / 7 automatic monitoring and anomaly handling.

[0257] Specifically, a multidimensional feature vector F is extracted from the collaborative analysis results in step S4. i This is used for anomaly detection. Let F be... i Let be the feature vector at the i-th time step.

[0258] F i =[f i1 ,f i2 ,…,f in (29),

[0259] Among them, f ij Let represent the j-th feature at the i-th time step, and n be the feature dimension.

[0260] Furthermore, a composite algorithm employing multiple detection technologies is used to comprehensively determine whether the equipment status is abnormal. Let the detection result of the composite detection algorithm be D. i Composed of multiple detection sub-models d k Overall decision:

[0261]

[0262] Where K is the number of detection sub-models, w k Let d be the weight of the k-th sub-model. k (F i ) represents the feature vector F of the k-th sub-model. i The test results.

[0263] Specifically, to improve the accuracy and robustness of anomaly detection, a self-supervised learning technique is used to construct the detection model. The self-supervised learning method utilizes the structural information of the data itself to generate labels for feature learning and anomaly detection. The sub-model design is as follows:

[0264] A. Generate a contrastive learning model:

[0265] An anomaly detection model based on generative contrastive learning is trained by constructing pairs of positive and negative samples. Let the feature vector be F. i Positive samples are Negative samples are Its generation formula is:

[0266]

[0267] wherein augment is a data augmentation operation, and shuffle is a random shuffle operation.

[0268] A contrastive loss function (ContrastiveLoss) is defined for training the contrastive learning model. The loss function formula is:

[0269]

[0270] wherein d(F i ,F j ) is the Euclidean distance metric between feature vectors, and m is the margin value. The Euclidean distance metric formula is:

[0271]

[0272] B. Anomaly detection model:

[0273] Based on the feature representation trained by self-supervised learning, an anomaly detection model is constructed. Let the detection model be d s , and the detection result be y s :

[0274] y s =d s (F i ) (34),

[0275] wherein d s is the anomaly detection model, and y s is the detection result. The detection model is trained based on the representation of self-supervised feature learning, and combined with support vector machine (SVM) for classification:

[0276] y s =sign(w·F i +b) (35),

[0277] wherein w is the weight vector, b is the bias, and sign represents the sign function, which returns the sign of the input. That is, if the input is greater than 0, return 1; if the input is less than 0, return -1; if the input is equal to 0, return 0.

[0278] C. Multi-modal self-supervised model:

[0279] Combining multi-modal data, self-supervised learning is used for anomaly detection. Let the multi-modal feature vector be M i , the self-supervised model be d m , and its detection result be y m :

[0280]

[0281] wherein M i is a multi-modal feature vector, containing text feature t i , image feature i i and sensor feature s i , d m is a multi-modal self-supervised model.

[0282] D, a time-series self-supervised learning model:

[0283] The time-series self-supervised learning method is adopted to realize anomaly detection by predicting the feature vector of the next time step. Let the feature vector of the current time step be F i , and the predicted feature vector of the next time step be The prediction model is d t , and the loss function is:

[0284]

[0285] wherein, The prediction model d t is realized based on LSTM (Long Short-Term Memory Network):

[0286]

[0287] wherein h i is the current hidden state.

[0288] E, a frequency domain self-supervised learning model:

[0289] The frequency domain self-supervised learning model based on frequency domain analysis realizes anomaly detection by extracting frequency domain features through Fourier transform. Let the frequency domain feature vector be The self-supervised learning model is d f , and the detection result is y f :

[0290]

[0291] wherein, is a Fourier transform operation, is a frequency domain feature vector, and the detection model d f combines the frequency domain features and spatial features for anomaly detection.

[0292] Through the above self-supervised learning scheme, the application generates labels using the structural information of the data itself, performs feature learning and anomaly detection, and ensures that the anomaly detection model has high robustness and accuracy.

[0293] Further, the detection results of each sub-model are integrated to obtain the final anomaly determination result. Let the integrated detection result be D i :

[0294]

[0295] Among them, w k For the weights of the sub-model, d k (F i ) represents the detection result of the k-th sub-model. If D i If the value exceeds the set threshold θ, it is considered abnormal.

[0296]

[0297] Among them, A i The result represents the anomaly assessment, where θ is the anomaly threshold for comprehensive detection.

[0298] Preferably, an early warning mechanism is triggered based on the detection results, and the early warning level is adaptively adjusted. Let the early warning signal be W. i The warning level is L. i :

[0299] W i =A i ·L i (42),

[0300] Among them, L i The alert level is adaptively adjusted based on historical data and the current situation of the detection results. The alert level can be adjusted according to the frequency and severity of anomaly detection. For example, for frequently occurring anomalies, the alert level can be increased; for rare but serious anomalies, the highest level alert can be triggered directly.

[0301] The warning signal triggers corresponding response measures. Let the set of response measures be {R}. j}, corresponding to different warning levels:

[0302] R j =Response(W i (43),

[0303] Among them, R j For the response measures of Level j warning, W i This is the current early warning signal. Response measures may include notifying maintenance personnel, automatically adjusting equipment operating parameters, and activating backup equipment.

[0304] For a similar AI-based RPA business processing method provided in the above embodiments of the present invention, see [link to related documentation]. Figure 2 This invention also provides a structural block diagram of an AI-based RPA business processing system, which includes:

[0305] The data acquisition module 501 is configured to use an RPA system to collect text data and control camera to collect image data of hydropower plant equipment, use a sensor-based data acquisition module to collect sensor data, and pre-process the text data, the image data and the sensor data, and write the pre-processed text data, the image data and the sensor data into a database through the RPA system;

[0306] The feature fusion module 502 is configured to respectively extract features from the pre-processed text data, the image data and the sensor data and perform feature fusion to obtain a multi-modal feature vector;

[0307] The state prediction module 503 is configured to construct a device relationship graph according to the multi-modal feature vector and perform monitoring and anomaly detection on the monitored hydropower plant equipment state based on a multi-layer adaptive time sequence model analysis method, and predict the state of the hydropower plant equipment;

[0308] The cross-site data integration module 504 is configured to construct a cross-site data collaborative analysis model based on federated learning to integrate different site data, automatically process the analysis results, and generate global optimization suggestions and operation instructions; wherein, the objective optimization function of the cross-site data collaborative analysis model is expressed as follows:

[0309]

[0310] Wherein, J represents the total optimization objective function, C i (u i ) represents the operation cost function of site i, g i (u i ) represents the operation constraint condition of site i, u i represents the operation parameter of site i, N represents the total number of sites, and α and β are weight coefficients of cost and emission, respectively, E i (u i ) represents the emission control function of site i, h i (u i , t) represents the dynamic constraint condition at time t, and Δ represents the value range of the uncertainty factor δ i .

[0311] The detection model construction module 505 is configured to use a self-supervised learning technology to construct a detection model, cover abnormal conditions, and realize all-weather automatic monitoring and abnormal processing through an RPA system;

[0312] Wherein, the detection model includes a generative contrast learning sub-model, an anomaly detection sub-model, a multi-modal self-supervised sub-model, a time sequence self-supervised learning sub-model and a frequency domain self-supervised learning sub-model;

[0313] The contrast learning sub-model is trained by constructing a positive sample and a negative sample pair, and a feature vector is F i , the positive sample is , and the negative sample is The generation formula of the contrast learning model is:

[0314]

[0315] augment represents a data augmentation operation, and shuffle represents a random shuffling operation.

[0316] A contrast loss function L is defined for training the generative contrast learning model, and the loss function L is represented as follows:

[0317]

[0318] where d(F i ,F j ) represents the Euclidean distance measurement between feature vectors, m represents a boundary value, represents the Euclidean distance measurement between the feature vector and the positive sample, represents the Euclidean distance measurement between the feature vector and the negative sample;

[0319] The anomaly detection sub-model, let the anomaly detection model be d s , and the detection result be y s , is represented as follows:

[0320] y s =d s (F i ),

[0321] where d s represents the anomaly detection model, y s represents the detection result, and the anomaly detection model d s is trained based on the representation of self-supervised feature learning and combined with a support vector machine for classification, and is represented as follows:

[0322] y s =sign(w·F i +b),

[0323] where w represents a weight vector, b represents a bias, and sign represents a sign function for returning the sign of the input. That is, if the input is greater than 0, 1 is returned; if the input is less than 0, -1 is returned; and if the input is equal to 0, 0 is returned;

[0324] The multi-modal self-supervised sub-model combines multi-modal data and uses self-supervised learning for anomaly detection, and a multi-modal feature vector is M i , and a self-supervised model is d m, the detection result of the multi-modal self-supervised model is y m , as follows:

[0325]

[0326] wherein M i represents a multi-modal feature vector, containing text features t i , image features i i and sensor features s i ; d m represents a multi-modal self-supervised model;

[0327] The time sequence self-supervised learning sub-model adopts a time sequence self-supervised learning method, and realizes anomaly detection by predicting a feature vector of a next time step, wherein the feature vector of the current time step is F i , and the predicted feature vector of the next time step is The prediction model is d t , and the loss function L t of the time sequence self-supervised learning model is as follows:

[0328]

[0329] wherein, F i+1 represents the actual feature vector of the next time step, and the prediction model d t is realized based on an LSTM, as follows:

[0330]

[0331] wherein h i represents a current hidden state;

[0332] The frequency domain self-supervised learning sub-model realizes anomaly detection by extracting frequency domain features through Fourier transform, wherein the frequency domain feature vector is F The frequency domain self-supervised learning model is d f , and the detection result of the frequency domain self-supervised learning model is y f , as follows:

[0333]

[0334] wherein, represents a Fourier transform operation, represents a frequency domain feature vector, and the frequency domain self-supervised learning model d f combines the frequency domain features and the spatial features to realize anomaly detection.

[0335] To sum up, the application uses a composite detection algorithm and an adaptive early warning mechanism to detect and warn about the abnormal state of the equipment in the hydropower plant, realizes efficient fault detection and early warning management, and ensures the safe operation of the equipment. Specifically, through multi-dimensional feature extraction and the combination of multiple detection technologies, the system can more comprehensively and accurately identify potential equipment failures, and respond in a timely manner through the adaptive early warning mechanism to avoid the expansion of the fault, thereby improving the overall operation reliability and safety of the hydropower plant.

[0336] It should be noted that the above-described workflow is only illustrative and does not limit the scope of protection of the application. In actual application, a person skilled in the art can select part or all of them according to actual needs to achieve the purpose of the embodiment scheme, which is not limited here.

[0337] In addition, technical details not described in detail in this embodiment can be referred to the parameter operation method provided by any embodiment of the application, which will not be described here.

[0338] It should be noted that in this paper, the terms "include", "contain" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or system including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or includes elements inherent to such process, method, article or system. Without more limitations, the element defined by the sentence "includes a" does not exclude the presence of another identical element in the process, method, article or system including the element.

[0339] The above-mentioned embodiment numbers of the application are only for description, not representing the advantages and disadvantages of the embodiments.

[0340] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be realized by means of software and general hardware platform, of course, they can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the application can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory / random access memory, a magnetic disk, an optical disk), and includes a plurality of instructions for making a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) execute the methods described in various embodiments of the application.

[0341] The above is only the preferred embodiment of the application, and does not limit the patent scope of the application. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings, or directly or indirectly applied to other related technical fields, is also included in the patent protection scope of the application.

Claims

1. An RPA business processing method based on artificial intelligence, characterized in that, The method includes: S1. Use the RPA system to collect text data and control the camera to collect image data of the hydropower plant equipment. Use the sensor-based data acquisition module to collect sensor data and preprocess the text data, image data and sensor data. Write the preprocessed text data, image data and sensor data into the database through the RPA system. S2. Extract and fuse features from the preprocessed text data, image data, and sensor data respectively to obtain a multimodal feature vector; S3. Based on the multimodal feature vector, an equipment relationship diagram is constructed, and an analysis method based on a multi-layer adaptive time series model is used to monitor and detect anomalies in the equipment status of the hydropower plant, while also predicting the equipment status of the hydropower plant. S4. Construct a cross-site data collaborative analysis model based on federated learning to integrate data from different sites, and automatically process the analysis results to generate global optimization suggestions and operation instructions; wherein, the objective optimization function of the cross-site data collaborative analysis model is expressed as follows: ， in, Denotes the overall optimization objective function. Indicates site The running cost function, Indicates site The operational constraints, Indicates site Operating parameters Indicates the total number of sites. and These are the weighting coefficients for cost and emissions, respectively. Indicates site The emission control function, Indicates time Dynamic constraints on Indicating uncertainty factors The range of values ​​for ; S5. Utilize self-supervised learning technology to build a detection model that covers abnormal situations, and use the RPA system to achieve 24 / 7 automatic monitoring and anomaly handling. The detection model includes a generative contrastive learning sub-model, an anomaly detection sub-model, a multimodal self-supervised sub-model, a temporal self-supervised learning sub-model, and a frequency domain self-supervised learning sub-model. The contrastive learning sub-model is trained by constructing pairs of positive and negative samples, and receives the feature vector optimized by S4. Positive samples are Negative samples are The formula for generating the contrastive learning model is: ， in, This indicates a data augmentation operation. This indicates a random shuffling operation; Define the contrastive loss function Used to train a generative contrastive learning model, loss function The formula is expressed as follows: ， in, Indicates boundary values, This represents the Euclidean distance metric between the feature vector and the positive sample. Represents the Euclidean distance metric between the feature vector and the negative sample; The anomaly detection sub-model is denoted as follows: The test results are , means as follows: ， in, This represents the anomaly detection sub-model. This represents the detection result; the anomaly detection sub-model. The representation is trained based on self-supervised feature learning and then combined with a support vector machine for classification, as shown below: ， in, Represents the weight vector. This indicates the bias, and sign represents the sign function, which returns the sign of the input. If the input is greater than 0, it returns 1; if the input is less than 0, it returns -1; if the input is equal to 0, it returns 0. The multimodal self-supervised sub-model combines multimodal data and utilizes self-supervised learning for anomaly detection. Let the multimodal feature vector be... The multimodal self-supervised sub-model is The detection results of the multimodal self-supervised sub-model are as follows: , means as follows: ， in, Represents a multimodal feature vector. This is the normalized fused feature vector; Represents a multimodal self-supervised sub-model; The temporal self-supervised learning sub-model employs a temporal self-supervised learning method to achieve anomaly detection by predicting the feature vector of the next time step. Let the feature vector of the current time step be... The predicted feature vector for the next time step is The prediction model is The loss function of the time-series self-supervised learning model For, it is represented as follows: ， in, , The feature vector representing the actual next time step, the prediction model. Based on LSTM, it is represented as follows: ， in, Indicates the current hidden state; The frequency domain self-supervised learning sub-model extracts frequency domain features for anomaly detection using Fourier transform. Let the feature vector at the current time step be... The frequency domain self-supervised learning sub-model is The detection result of the frequency domain self-supervised learning sub-model is , means as follows: ， in, This indicates the Fourier transform operation. This represents the frequency domain feature vector at the current time step, and is a frequency domain self-supervised learning sub-model. Anomaly detection is performed by combining frequency domain features; The device relationship graph constructed based on multimodal feature vectors is represented as follows: Based on the physical connections and dependencies between the equipment in the hydropower plant, an equipment relationship diagram is constructed; the equipment set is... ,in Indicates the first One device; Define the edge set between devices , where each edge Indicates device and There is a direct relationship between them; Constructing a device relationship diagram ; The device relationship graph uses multimodal feature vectors as the initial feature vectors for each device, and a new weighted dynamic feature propagation algorithm is used to propagate features from the device relationship graph, updating the feature representation of each device; wherein, the new weighted dynamic feature propagation algorithm defines the first... The feature propagation formula for a layer is: ， in, Indicates the first The device in the The feature vector of the layer, Indicates device The neighborhood group, and These represent the connection weights and historical state weights between devices, respectively. Indicates the first The weight matrix of the layer, This represents the activation function. Indicates the first The device in the Feature vectors of layer -1; The analysis method based on the multi-layer adaptive time series model specifically includes: eigenvectors The input is fed into the encoder part of the variational autoencoder to obtain the latent variables. : ， in, This represents the mean. Indicates standard deviation, Represents random noise variables. and These represent neural network functions; Encoded latent variables The data is input into an LSTM network for time series modeling. The update formula for the LSTM network is as follows: ， in, , , Let represent the activation vectors of the forget gate, input gate, and output gate, respectively. Represents the cell state vector. , , , Represents the weight matrix. , , , This represents the bias vector. This represents the Sigmoid activation function. Represents element-wise product. Indicates time step The temporal feature vector; Output using LSTM network Perform state prediction and define the prediction function. The calculation is as follows: ， in, Indicates time step The predicted state, Indicates a fully connected layer; S4 specifically includes: S41. Standardize and align data from different sites; S42. Use the Pearson correlation coefficient to calculate the linear correlation between the data of two stations, and at the same time calculate the mutual information between each station to measure the non-linear correlation between the data of two stations. S43. Construct a physical model; the physical model includes a hydropower plant energy balance model and a water flow balance model; S44. Perform weighted average fusion on the cross-site data, and then use principal component analysis to reduce the dimensionality of the fused data; S45. Utilize cross-site fusion data for collaborative fault diagnosis to detect potential equipment faults. Simultaneously, based on the collaborative analysis results, make optimized scheduling decisions and adjust the operating parameters of each site.

2. The RPA business processing method based on artificial intelligence according to claim 1, characterized in that, The preprocessing includes data cleaning and data standardization.

3. The RPA business processing method based on artificial intelligence according to claim 1, characterized in that, Text feature extraction employs a bag-of-words model and TF-IDF weighted processing to extract text features from operation reports and maintenance records, resulting in text feature vectors. Image feature extraction utilizes a convolutional neural network to extract features from image data, resulting in image feature vectors. The sensor features are extracted by wavelet transform to extract time-series features from the sensor data, resulting in a sensor feature vector.

4. The RPA business processing method based on artificial intelligence according to claim 3, characterized in that, Weighted fusion of text features, image features, and sensor features is performed, specifically including: Divide the feature vector into several sub-regions and calculate the correlation between each sub-region. Let... These are text feature vectors, image feature vectors, and sensor feature vectors, respectively divided into... Calculate the local correlation matrix for each sub-region. , means as follows: ， in, This represents the correlation between the k-th local regions of modes i and j. and These represent the k-th sub-regions of the text and image feature vectors, respectively. and These are the mean values ​​of the sub-region; Let the initial weight vector be The global weights corresponding to text features, image features, and sensor features are calculated using a local correlation matrix. Dynamically adjusting the weights is represented as follows: ， ， ， The adjusted local weights satisfy Within each sub-region, dynamically adjusted local weights are applied for weighted fusion to obtain the weighted fused feature vector. , means as follows: ， in, It is the first The data point at the th th Weighted fusion feature vectors within each sub-region These are the local weights for text features, image features, and sensor features, respectively. , , They represent the first Text feature vectors, image feature vectors, and sensor feature vectors within each sub-region.

5. The RPA business processing method based on artificial intelligence according to claim 1, characterized in that, The energy balance model for the hydropower plant is constructed as follows: Set up a site Energy input is Energy output is Energy storage is The energy balance equation is expressed as follows: ， in, Indicates time step Changes in internal energy storage; The water flow balance model is constructed as follows: Set up a site The water inflow is The water output is Water storage capacity is The water balance equation is expressed as follows: ， in, Indicates time step Changes in internal water storage volume.

6. The RPA business processing method based on artificial intelligence according to claim 1, characterized in that, The detection results of each sub-model are combined to obtain the final anomaly determination result: Let the comprehensive test result be : ， in, Indicates the weights of the sub-model. Indicates the first The detection results of each sub-model; if Exceeding the set threshold If so, it is considered abnormal: ， in, This indicates the result of the anomaly assessment. This indicates the abnormal threshold for comprehensive detection.

7. A system for executing the AI-based RPA business processing method as described in claim 1, characterized in that, The system includes: The data acquisition module is used to acquire text data from hydropower plant equipment and control camera image data using the RPA system. It uses a sensor-based data acquisition module to acquire sensor data and preprocesses the text data, image data, and sensor data. The RPA system then writes the preprocessed text data, image data, and sensor data into the database. The feature fusion module is used to extract and fuse features from preprocessed text data, image data, and sensor data to obtain multimodal feature vectors. The status prediction module is used to monitor and detect anomalies in the status of hydropower plant equipment by constructing equipment relationship diagrams based on multimodal feature vectors and using analysis methods based on multi-layer adaptive time series models, while also predicting the status of hydropower plant equipment. The cross-site data integration module is used to construct a cross-site data collaborative analysis model based on federated learning to integrate data from different sites. Through automatic processing of the analysis results, it generates global optimization suggestions and operation instructions. The objective optimization function of the cross-site data collaborative analysis model is expressed as follows: ， in, Denotes the overall optimization objective function. Indicates site The running cost function, Indicates site The operational constraints, Indicates site Operating parameters Indicates the total number of sites. and These are the weighting coefficients for cost and emissions, respectively. Indicates site The emission control function, Indicates time Dynamic constraints on Indicating uncertainty factors The range of values ​​for ; The detection model building module is used to build detection models using self-supervised learning techniques, covering abnormal situations, and to achieve 24 / 7 automatic monitoring and anomaly handling through the RPA system.

Images