A deep learning based robotic process automation method
By using deep learning technology, we constructed multimodal prediction models, identification and classification models, and anomaly detection models, which solved the problems of data processing and model integration in robotic process automation, improved the execution efficiency and intelligence level of business processes, and reduced operating costs.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHENZHEN DAJIABANG COMPUTER CO LTD
- Filing Date
- 2025-02-27
- Publication Date
- 2026-06-09
AI Technical Summary
Existing technologies struggle to construct multimodal prediction models, identification and classification models, and anomaly detection models, and are difficult to integrate into robotic process automation (RPA) programs, making it impossible to effectively evaluate the performance of RPA processes.
By acquiring structured and unstructured data, preprocessing and feature extraction are performed to generate multimodal feature vectors. Deep learning algorithms are then used to construct multimodal prediction models, identification and classification models, and anomaly detection models. These models are then integrated into robotic process automation programs for business process testing and performance evaluation.
It enables the processing of complex data types, improves the intelligence level of automated processes, reduces human error, lowers operating costs, and adapts to changes in different business needs.
Smart Images
Figure CN120145005B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of data processing technology, specifically a robotic process automation method based on deep learning. Background Technology
[0002] Robotic Process Automation (RPA) is a technology that uses software robots or robots to simulate and integrate human interactions in digital systems to execute business processes. These robots can learn and repeatedly perform a series of tasks, thereby improving efficiency, reducing errors, and lowering operating costs. Traditional RPA typically relies on explicit programming rules to process structured data and application interfaces. Deep learning, a subset of machine learning, uses multi-layered neural networks to simulate how the human brain processes information. With the development of deep learning technology, new solutions have been provided for robotic process automation.
[0003] The existing technology has the following problems: it is difficult to build multimodal prediction models, identification and classification models and anomaly detection models to analyze business data; it is also difficult to integrate the various models into the robotic process automation program; and finally, it is difficult to evaluate the performance of the robotic process automation process. Summary of the Invention
[0004] This invention aims to at least solve one of the technical problems existing in the prior art; to this end, this invention proposes a deep learning-based robotic process automation method to solve the above-mentioned problem. Specifically, the first aspect of this invention provides a deep learning-based robotic process automation method, comprising the following steps:
[0005] S1: Obtain the data acquisition request sent by the client and collect business data. The client has a robotic process automation program, and the business data includes structured data and unstructured data.
[0006] S2: Preprocess the collected structured data, including data cleaning, denoising, and standardization; preprocess the collected unstructured data, including text segmentation, stop word removal, image enhancement, grayscale conversion, and denoising.
[0007] S3: Perform feature extraction on the collected and preprocessed structured and unstructured data; generate multimodal feature vectors by feature fusion based on the feature extraction results;
[0008] S4: Construct a multimodal prediction model using deep learning algorithms based on multimodal feature vectors; construct a recognition and classification model using deep learning algorithms; analyze anomalies in business data by constructing an anomaly detection model; integrate the multimodal prediction model, recognition and classification model, and anomaly detection model into the robotic process automation program and perform business process testing.
[0009] S5: Comprehensively evaluate the performance of the robotic process automation process and iteratively update the robotic process automation process based on the evaluation results.
[0010] Preferably, step S1 includes the following steps:
[0011] Use an API interface or web server to receive data retrieval requests sent by clients. The request content includes: the type and source of the business data.
[0012] By utilizing deep learning models, the type and source of business data in the request content can be automatically identified;
[0013] By acquiring a sample set of business data, which includes various business data types and sources; wherein, the business data types include: structured business data and unstructured business data; the structured business data includes: fields and data values in databases or tables; the unstructured business data includes: text, video images, and audio; the business data sources include: databases and web pages;
[0014] Each business data sample is labeled with its data type and source, and the labeled business data sample set is input into a deep learning model for training; based on the trained deep learning model, the request content is input into the deep learning model to automatically identify the type and source of business data in the request content;
[0015] Based on the type and source of business data automatically identified by the deep learning model, the Robotic Process Automation (RPA) program is used to simulate manual operation processes; the RPA program is connected to a database or web page; by executing the RPA script, the data collection operation is automatically completed and the results are output, which are business data collected from the database or web page.
[0016] Preferably, step S3 involves feature extraction processing of the collected and preprocessed structured and unstructured data, including the following steps:
[0017] Feature extraction is performed on the collected and preprocessed structured data, including numerical features and categorical variable features; the numerical features are normalized or standardized; and the categorical variables are converted into a set of binary features using one-hot encoding, where each categorical variable feature represents a category.
[0018] Based on the preprocessed text, word embeddings or sentence embeddings are extracted using natural language processing techniques.
[0019] Based on the preprocessed video images, the video image data is processed using OpenCV computer vision technology to extract video image features, including color features, texture features, and shape features;
[0020] Based on the preprocessed language, the audio signal is segmented into 25-millisecond frames, and language features such as Mel frequency cepstral coefficients, spectral centroid, and spectral bandwidth are extracted from each frame.
[0021] Preferably, step S3, which involves feature fusion based on the feature extraction results to generate a multimodal feature vector, includes the following steps:
[0022] Based on the extracted structured data features, text features, video image features, and speech features, feature alignment is performed using timestamps, and feature dimensionality reduction is performed using principal component analysis.
[0023] The attention mechanism is used to calculate the weight of each feature from the structured data features, text features, video image features, and speech features after feature alignment and dimensionality reduction. The weighted features are then fused to emphasize important features. Joint representation learning is performed through shared network layers or cross-modal loss functions. A multi-layer perceptron network is constructed to input the features after joint representation learning into the multi-layer perceptron network to generate multi-modal feature vectors.
[0024] Preferably, step S4, which involves constructing a multimodal prediction model using a deep learning algorithm based on the multimodal feature vectors, includes the following steps:
[0025] Based on the generated multimodal feature vectors, a multimodal prediction model is constructed using a deep learning model to analyze and predict business data.
[0026] The multimodal feature vectors are input into the multimodal prediction model for training;
[0027] The system collects and preprocesses business data in real time, extracts new multimodal vectors from the real-time collected business data, inputs the new multimodal vectors into the trained multimodal prediction model for analysis and prediction, and outputs the prediction results, which are the predicted trends of business data changes.
[0028] Preferably, step S4, which involves constructing a recognition and classification model using a deep learning algorithm, includes the following steps:
[0029] By using deep learning algorithms to build a recognition and classification model, multimodal feature vectors are input into the recognition and classification model for training. During the training process, supervised learning is performed using labeled data, and the recognition and classification model is optimized using backpropagation algorithm and gradient descent method.
[0030] The real-time collected and preprocessed business data is input into the trained recognition and classification model; the recognition and classification model performs behavioral pattern recognition and classification on the input business data.
[0031] Preferably, step S4 involves analyzing anomalies in business data by constructing an anomaly detection model, including the following steps:
[0032] An anomaly detection model is constructed based on a deep learning algorithm; multimodal feature vectors are input into the anomaly detection model for training; during the training process, iterative optimization is performed to enable the anomaly detection model to learn the distribution characteristics of normal business data;
[0033] The anomaly detection model is trained by inputting real-time collected and preprocessed business data. The model evaluates the input business data based on the learned normal data distribution characteristics and detects whether the real-time business data is abnormal.
[0034] Preferably, step S4, which integrates the multimodal prediction model, the identification and classification model, and the anomaly detection model into the robotic process automation program and performs business process testing, includes the following steps:
[0035] Use the designer of the Robotic Process Automation tool to create a framework for automated processes, including: the start point, end point, intermediate steps, and decision points of the business process;
[0036] The trained multimodal prediction model, recognition and classification model, and anomaly detection model are deployed to the server respectively;
[0037] Design automated business processes in the designer of robotic process automation tools;
[0038] Through API interfaces, calls to multimodal prediction models, recognition and classification models, and anomaly detection models are embedded in the robotic process automation scripts, and an anomaly handling mechanism is designed in the automated business process; when a call fails or returns an unpredictable result in one of the models, an anomaly warning is issued; otherwise, the various models are called for analysis.
[0039] Each component in the business process is tested individually, and the business process is also integrated for testing. By testing the entire robotic process automation business process, it is determined whether all components work together. Based on the test results, the robotic process automation business process is optimized. The tested and optimized robotic process automation business process is deployed to the production environment to begin automated execution of the business process.
[0040] Based on the deployed robotic process automation (RoLA) business processes, key performance indicators, including processing time, error rate, and response time, are collected by monitoring the operational status of the RoLA business processes in real time.
[0041] Preferably, step S5 includes the following steps:
[0042] The trained multimodal prediction model, classification model, and anomaly detection model were evaluated separately, and the key performance indicator F1 score was calculated for each. The model stability coefficient was calculated using the following formula:
[0043] The model stability coefficients M are obtained; where F1 D F1 represents the F1 score of a multimodal prediction model; S This represents the F1 score of the classification model; F1 J These represent the F1 scores of the anomaly detection model;
[0044] The efficiency η of a business process is obtained by calculating the ratio of the processing time of a robotic process automation business process to the processing time of a manual business process.
[0045] By calculating the formula for business process stability:
[0046] The stability of the business process is obtained as C; where Wr represents the error rate of the robotic process automation business process, and T represents the response time of the robotic process automation business process.
[0047] The performance formula for computational robotic process automation is: N = α*M + β*η + γ*C.
[0048] The performance value N of the robotic process automation process is obtained; where α, β and γ represent the model stability coefficient, business process efficiency and business process stability weight coefficients, respectively; the sum of α, β and γ is 1;
[0049] Based on the performance calculation results of the robotic process automation (RPA), it is determined whether to iterate and update the RPA business process. If the real-time calculated RPA performance value is lower than the average historical performance value, the RPA business process is iterated and updated; otherwise, it is not iterated and updated.
[0050] Compared with the prior art, the beneficial effects of the present invention are:
[0051] This invention generates multimodal feature vectors by extracting and fusing features from structured and unstructured data, enabling robotic process automation programs to handle more complex data types. It breaks through the limitation of traditional RPA, which can only handle regular and structured data. Furthermore, by utilizing deep learning algorithms to construct multimodal prediction models, identification and classification models, and anomaly detection models, this invention can analyze business data more accurately and improve the intelligence level of automated processes.
[0052] This invention automates tasks that originally required manual intervention by integrating various constructed models into a robotic process automation (RPA) program, significantly improving the execution efficiency of business processes. The automated process reduces the possibility of human error, lowers operating costs, and improves the execution efficiency of business processes.
[0053] This invention comprehensively evaluates the performance of robotic process automation processes and iteratively updates them to adapt to changes in different business needs. Attached Figure Description
[0054] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0055] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation
[0056] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0057] Please see Figure 1 As shown, a first aspect of the present invention provides a deep learning-based method for automating robotic processes, comprising the following steps:
[0058] S1: Obtain the data acquisition request sent by the client and collect business data. The client has a robotic process automation program, and the business data includes structured data and unstructured data.
[0059] S2: Preprocess the collected structured data, including: data cleaning, denoising, and standardization; preprocess the collected unstructured data, including: text tokenization, stop word removal, image enhancement, grayscale conversion, and denoising;
[0060] S3: Perform feature extraction on the collected and preprocessed structured and unstructured data; generate a multimodal feature vector according to the results of feature extraction;
[0061] S4: Construct a multimodal prediction model using a deep learning algorithm based on the multimodal feature vector; construct an identification and classification model by using a deep learning algorithm; analyze abnormal situations in business data by constructing an anomaly detection model; integrate the multimodal prediction model, the identification and classification model, and the anomaly detection model into the robotic process automation program respectively and conduct business process testing;
[0062] S5: Comprehensively evaluate the performance of the robotic process automation process, and iteratively update the robotic process automation process according to the evaluation results.
[0063] Specifically, ensure that the client has installed and configured the robotic process automation RPA program, and the RPA program should be able to identify and trigger data acquisition requests and automatically start when business data needs to be collected. Automatically capture business data from sources such as databases and APIs through the RPA program. The preprocessing of structured data in business data includes: removing duplicate, missing, or invalid data records, identifying and correcting outliers or noise in the data, and converting the data into a unified format and unit. The preprocessing of text data includes: splitting text data into words or phrases, removing common words that contribute little to the meaning of the text, such as "de" (of), "le" (particle indicating completion), etc.; the preprocessing of video image data includes: enhancing image contrast, converting color images to grayscale images, and using filters to remove noise in the images; the preprocessing of voice data for denoising. Extract features from structured and unstructured data and fuse the extracted structured and unstructured features to generate a multimodal feature vector. Construct a multimodal prediction model, an identification and classification model, and an anomaly detection model and integrate them into the RPA program respectively. Through business process testing, ensure that the RPA program can correctly perform tasks such as data acquisition, preprocessing, feature extraction, model prediction, and anomaly detection. Through comprehensively evaluating the performance of the robotic process automation process, iteratively update the RPA process.
[0064] In this embodiment, step S1 includes the following steps:
[0065] Receive the data acquisition request sent by the client using an API interface or a web server, where the request content includes: the type and source of business data;
[0066] By utilizing deep learning models, the type and source of business data in the request content can be automatically identified;
[0067] By acquiring a sample set of business data, which includes various business data types and sources; wherein, the business data types include: structured business data and unstructured business data; the structured business data includes: fields and data values in databases or tables; the unstructured business data includes: text, video images, and audio; the business data sources include: databases and web pages;
[0068] Each business data sample is labeled with its data type and source, and the labeled business data sample set is input into a deep learning model for training; based on the trained deep learning model, the request content is input into the deep learning model to automatically identify the type and source of business data in the request content;
[0069] Based on the type and source of business data automatically identified by the deep learning model, the Robotic Process Automation (RPA) program is used to simulate manual operation processes; the RPA program is connected to a database or web page; by executing the RPA script, the data collection operation is automatically completed and the results are output, which are business data collected from the database or web page.
[0070] Specifically, an API interface or web server is used as the front-end receiving point to receive data acquisition requests from clients. The request content should clearly indicate the type and source of the required business data. The received request is parsed to extract the type and source information of the business data. A deep learning model is prepared and trained in advance, capable of identifying and classifying the type and source of business data. The parsed request content, i.e., the type and source of the business data, is input into the deep learning model. The deep learning model identifies the input content and outputs the identified business data type and source. A business data sample set is formed by collecting business data samples of various types and sources, and each business data sample is labeled, clearly indicating its data type and source. The labeled business data sample set is input into the deep learning model for training, enabling the model to accurately identify the type and source of business data. Based on the business data type and source identified by the deep learning model, the corresponding RPA program is configured. The RPA program should be able to simulate manual operation processes and connect to the corresponding database or web page. The RPA script is executed to automatically complete the data collection operation. During the collection process, the RPA program extracts the required business data from the database or web page according to predefined rules and logic. The RPA program takes the collected business data as output and provides it to the client or subsequent processing flow.
[0071] In this embodiment, step S3 involves feature extraction processing of the collected and preprocessed structured and unstructured data, including the following steps:
[0072] Feature extraction is performed on the collected and preprocessed structured data, including numerical features and categorical variable features; the numerical features are normalized or standardized; and the categorical variables are converted into a set of binary features using one-hot encoding, where each categorical variable feature represents a category.
[0073] Based on the preprocessed text, word embeddings or sentence embeddings are extracted using natural language processing techniques.
[0074] Based on the preprocessed video images, the video image data is processed using OpenCV computer vision technology to extract video image features, including color features, texture features, and shape features;
[0075] Based on the preprocessed language, the audio signal is segmented into 25-millisecond frames, and language features such as Mel frequency cepstral coefficients, spectral centroid, and spectral bandwidth are extracted from each frame.
[0076] Specifically, feature extraction for structured data involves numerical features, which typically represent quantifiable metrics or attributes. Normalization scales the numerical features to the [0,1] interval using their maximum and minimum values. For each column of features, a min-max function is used for scaling. Standardization scales the features to a standard normal distribution using their mean and standard deviation, resulting in a mean of 0 and a variance of 1. Standardization can be used even if the data does not follow a normal distribution. Categorical variables represent data with a finite number of categories. One-hot encoding converts categorical variables into binary features, with each feature representing a category. For a categorical variable with N categories, one-hot encoding converts it into N binary features, each corresponding to a category, with only one feature being 1 and the rest 0. Feature extraction for unstructured data, particularly text data, includes word embeddings or sentence embeddings. Pre-trained models such as Word2Vec, GloVe, or BERT are used to convert words or sentences in the text into low-dimensional vector representations. Feature extraction of video image data includes: extracting color features from the image, including RGB values and HSV values; extracting texture features, including LBP local binary pattern features and Gabor filter features; and extracting shape features from the image, including edge features and contour features. Feature extraction of speech data includes: segmenting the audio signal into frames of fixed length, which is 25 milliseconds in this embodiment, and dynamically adjusting according to actual application conditions; and extracting language features, including Mel-frequency cepstral coefficients, spectral centroid, and spectral bandwidth. Based on the above, structured data features include: one-hot encoding of normalized or standardized numerical feature values and categorical variable features; text features include: word embedding feature vectors or sentence embedding feature vectors; video image features include: RGB values and HSV values, LBP local binary pattern features and Gabor filter features, edge intensity, and contour edge points; and speech features include: Mel-frequency cepstral coefficients, spectral centroid, and spectral bandwidth.
[0077] In this embodiment, step S3, which involves generating a multimodal feature vector through feature fusion based on the feature extraction results, includes the following steps:
[0078] Based on the extracted structured data features, text features, video image features, and speech features, feature alignment is performed using timestamps, and feature dimensionality reduction is performed using principal component analysis.
[0079] The attention mechanism is used to calculate the weight of each feature from the structured data features, text features, video image features, and speech features after feature alignment and dimensionality reduction. The weighted features are then fused to emphasize important features. Joint representation learning is performed through shared network layers or cross-modal loss functions. A multi-layer perceptron network is constructed to input the features after joint representation learning into the multi-layer perceptron network to generate multi-modal feature vectors.
[0080] Specifically, when processing multimodal data, since different sensors or data sources may generate data at different times, it is necessary to use timestamps to align these data onto the same time axis. The timestamp information of each modality's data is extracted, and appropriate alignment methods such as interpolation and resampling are used to align the modality data onto the same time axis. The aligned modality data is then standardized to eliminate dimensional differences. The covariance matrix of each modality's data is calculated, and eigenvalue decomposition is performed on the covariance matrix to obtain eigenvalues and eigenvectors. The top k principal components are selected based on the magnitude of the eigenvalues to construct a dimensionality-reduced feature space. Each modality's data is projected into this dimensionality-reduced feature space to obtain the dimensionality-reduced feature representation. An attention mechanism is calculated for each dimensionality-reduced modality feature to obtain the weight of each feature. The dimensionality-reduced modality features are then weighted and fused according to their weights to obtain the fused feature representation. A shared network layer or a cross-modal loss function is designed and constructed to achieve joint representation learning. The modality features after joint representation learning are used as input to an MLP network. Construct an MLP network, consisting of an input layer, multiple hidden layers, and an output layer. The hidden layers can be transformed using non-linear activation functions such as ReLU. Train and optimize the MLP network to obtain the final multimodal feature vectors.
[0081] In this embodiment, step S4, which involves constructing a multimodal prediction model using a deep learning algorithm based on multimodal feature vectors, includes the following steps:
[0082] Based on the generated multimodal feature vectors, a multimodal prediction model is constructed using a deep learning model to analyze and predict business data.
[0083] The multimodal feature vectors are input into the multimodal prediction model for training;
[0084] The system collects and preprocesses business data in real time, extracts new multimodal vectors from the real-time collected business data, inputs the new multimodal vectors into the trained multimodal prediction model for analysis and prediction, and outputs the prediction results, which are the predicted trends of business data changes.
[0085] Specifically, based on the characteristics of the business data and the prediction requirements, a suitable deep learning model is selected to construct a multimodal prediction model. For time-series data, recurrent neural networks or their variants, such as Long Short-Term Memory (LSTM) or Gated Recurrent Units (GRU), can be chosen; for image data, convolutional neural networks can be selected; and for text data, word embedding models or Transformers can be selected. The architecture of the multimodal prediction model is designed to ensure it can receive and process multimodal feature vectors. The model architecture can include an input layer, multiple hidden layers such as convolutional layers, recurrent layers, and fully connected layers, and an output layer. A suitable loss function is selected based on the requirements of the prediction task; for regression tasks, mean squared error or mean absolute error can be chosen; for classification tasks, cross-entropy loss can be chosen. The generated multimodal feature vectors are used as input data and paired with corresponding business data labels, such as historical business data trends. The dataset is divided into training, validation, and test sets for model training, validation, and testing. The training set is input into the multimodal prediction model for training. During training, the model parameters are adjusted to minimize the loss function and improve the model's prediction accuracy. The model's performance is evaluated using a validation set to select the optimal model parameters and architecture. Real-time business data is collected and preprocessed, including data cleaning and feature extraction. This preprocessed data is then converted into new multimodal vectors and input into the trained multimodal prediction model for predictive analysis. Based on the input multimodal vectors, the multimodal prediction model outputs prediction results, i.e., the changing trends of the real-time business data.
[0086] In this embodiment, step S4 involves constructing a recognition and classification model using a deep learning algorithm, including the following steps:
[0087] By using deep learning algorithms to build a recognition and classification model, multimodal feature vectors are input into the recognition and classification model for training. During the training process, supervised learning is performed using labeled data, and the recognition and classification model is optimized using backpropagation algorithm and gradient descent method.
[0088] The real-time collected and preprocessed business data is input into the trained recognition and classification model; the recognition and classification model performs behavioral pattern recognition and classification on the input business data.
[0089] Specifically, a recognition and classification model is built using deep learning algorithms. A suitable deep learning framework, such as TensorFlow or PyTorch, is chosen, as these frameworks provide all the tools and functions needed to build and train neural networks. The architecture of the recognition and classification model is designed based on the characteristics of the multimodal feature vectors and the requirements of the classification task. The model may include an input layer, a feature extraction layer, a fully connected layer, and an output layer. The output layer typically uses the softmax function for multi-class classification or the sigmoid function for binary classification. A loss function and optimizer are selected to update the model parameters using the backpropagation algorithm to minimize the loss function. A labeled multimodal feature vector dataset is collected and organized, where the labels represent the behavioral pattern category corresponding to each feature vector. The dataset is preprocessed, including data cleaning, normalization, and standardization, to ensure data quality and improve model training efficiency. The preprocessed multimodal feature vectors and their corresponding labels are input into the recognition and classification model. The gradient of the loss function with respect to the model parameters is calculated using the backpropagation algorithm. The model parameters are updated using the optimizer to minimize the loss function. The above steps are repeated until the model's performance on the validation set stabilizes or the predetermined number of training epochs is completed. Real-time business data is collected and preprocessed, such as through data cleaning and feature extraction, to generate new multimodal feature vectors. These preprocessed real-time business data, i.e., the new multimodal feature vectors, are then input into the trained recognition and classification model. The model performs forward propagation on the input data, calculates the predicted probability of each behavior pattern category, and selects the behavior pattern category with the highest predicted probability as the classification result.
[0090] In this embodiment, step S4 involves constructing an anomaly detection model to analyze anomalies in business data, including the following steps:
[0091] An anomaly detection model is constructed based on a deep learning algorithm; multimodal feature vectors are input into the anomaly detection model for training; during the training process, iterative optimization is performed to enable the anomaly detection model to learn the distribution characteristics of normal business data;
[0092] The anomaly detection model is trained by inputting real-time collected and preprocessed business data. The model evaluates the input business data based on the learned normal data distribution characteristics and detects whether the real-time business data is abnormal.
[0093] Specifically, select deep learning frameworks, including TensorFlow and PyTorch, to build and train the anomaly detection model. Based on the characteristics of the multimodal feature vectors and the requirements of the anomaly detection task, design the architecture of the anomaly detection model. Common architectures include autoencoders, variants of generative adversarial networks (GANs), and density-based networks. These models can learn the distribution characteristics of normal data. For autoencoders, reconstruction error is typically used as the loss function, i.e., the difference between the input data and the data reconstructed by the model. For variants of GANs, it may be necessary to define loss functions for the generator and discriminator, as well as possible additional loss functions. Select a suitable optimizer to update the model parameters through iterative optimization. Collect and organize the multimodal feature vectors of normal business data for training the anomaly detection model. Preprocess the normal business data, including data cleaning, normalization, and standardization, to ensure data quality. Input the preprocessed normal business data into the anomaly detection model. Through iterative optimization, enable the model to learn the distribution characteristics of normal data. This typically involves minimizing the loss function so that the model can accurately reconstruct the input data. Validate the model's performance using a portion of normal business data not used in training to ensure that the model can accurately reconstruct or generate normal data. Real-time acquisition of business data and preprocessing to generate new multimodal feature vectors. The preprocessed real-time business data is then input into a trained anomaly detection model. The anomaly detection model evaluates the input data based on learned normal data distribution characteristics. For autoencoders, the reconstruction error between the input data and the model-reconstructed data is calculated; if the reconstruction error exceeds a preset threshold, the input data is considered anomaly. For variants of generative adversarial networks, it may be necessary to determine whether the input data is anomaly based on the difference between the generator-generated data and the input data, the discriminator's output, or other metrics.
[0094] In this embodiment, step S4, which integrates the multimodal prediction model, the identification and classification model, and the anomaly detection model into the robotic process automation program and performs business process testing, includes the following steps:
[0095] Use the designer of the Robotic Process Automation tool to create a framework for automated processes, including: the start point, end point, intermediate steps, and decision points of the business process;
[0096] The trained multimodal prediction model, recognition and classification model, and anomaly detection model are deployed to the server respectively;
[0097] Design automated business processes in the designer of robotic process automation tools;
[0098] Through API interfaces, calls to multimodal prediction models, recognition and classification models, and anomaly detection models are embedded in the robotic process automation scripts, and an anomaly handling mechanism is designed in the automated business process; when a call fails or returns an unpredictable result in one of the models, an anomaly warning is issued; otherwise, the various models are called for analysis.
[0099] Each component in the business process is tested individually, and the business process is also integrated for testing. By testing the entire robotic process automation business process, it is determined whether all components work together. Based on the test results, the robotic process automation business process is optimized. The tested and optimized robotic process automation business process is deployed to the production environment to begin automated execution of the business process.
[0100] Based on the deployed robotic process automation (RoLA) business processes, key performance indicators, including processing time, error rate, and response time, are collected by monitoring the operational status of the RoLA business processes in real time.
[0101] Specifically, the designer of a Robotic Process Automation (RPA) tool first constructs the basic framework of the automated process, including: clearly defining the start point, end point, intermediate steps, and decision points of the business process. The trained multimodal prediction model, classification model, and anomaly detection model are deployed to the server. These models are the core of the automated process, responsible for processing and analyzing data and providing decision support. In the RPA tool's designer, specific automated business processes are designed according to business needs, including: defining the operations of each step, data flow, and the order of model calls. Through API interfaces, calls to the multimodal prediction model, classification model, and anomaly detection model are embedded in the RPA script; this means that in the automated process, when these models are needed for analysis or prediction, the RPA system can automatically call the corresponding model and obtain the results. Simultaneously, an anomaly handling mechanism is designed; when one of the model calls fails or returns unpredictable results, an anomaly warning can be issued, and corresponding measures can be taken, such as retrying, skipping the step, or triggering other backup processes. Each component in the business process is tested individually to ensure it functions correctly and meets expected requirements. Based on component testing, integration testing is performed on the business process, including testing the interfaces and interactions between components to ensure the entire process works collaboratively and achieves the intended results. Based on the test results, the robotic process automation (RoPA) business process is optimized, including adjusting the order of model calls, optimizing data processing flows, and improving exception handling mechanisms. The tested and optimized RoPA business process is then deployed to the production environment to begin automated execution. The operational status of the RoPA business process is monitored in real time, and key performance indicators are collected, including processing time, error rate, and response time.
[0102] In this embodiment, step S5 includes the following steps:
[0103] The trained multimodal prediction model, classification model, and anomaly detection model were evaluated separately, and the key performance indicator F1 score was calculated for each. The model stability coefficient was calculated using the following formula:
[0104] The model stability coefficients M are obtained; where F1 D F1 represents the F1 score of a multimodal prediction model; S This represents the F1 score of the classification model; F1 J These represent the F1 scores of the anomaly detection model;
[0105] The efficiency η of a business process is obtained by calculating the ratio of the processing time of a robotic process automation business process to the processing time of a manual business process.
[0106] By calculating the formula for business process stability:
[0107] The stability of the business process is obtained as C; where Wr represents the error rate of the robotic process automation business process, and T represents the response time of the robotic process automation business process.
[0108] The performance formula for computational robotic process automation is: N = α*M + β*η + γ*C.
[0109] The performance value N of the robotic process automation process is obtained; where α, β and γ represent the model stability coefficient, business process efficiency and business process stability weight coefficients, respectively; the sum of α, β and γ is 1;
[0110] Based on the performance calculation results of the robotic process automation (RPA), it is determined whether to iterate and update the RPA business process. If the real-time calculated RPA performance value is lower than the average historical performance value, the RPA business process is iterated and updated; otherwise, it is not iterated and updated.
[0111] Specifically, the model stability coefficient, business process efficiency, and business process stability are calculated using formulas. Then, based on weighting coefficients of 0.4, 0.3, and 0.3 for these coefficients, a comprehensive calculation is performed to obtain the Robotic Process Automation (RPA) performance value. The weighting coefficients are dynamically adjusted based on actual conditions and historical data. The average historical RPA performance value is calculated, and the real-time RPA performance value is compared with this historical average. If the real-time performance value is lower than the historical average, it indicates that the RPA process has room for improvement and requires iterative updates; otherwise, the current process continues unchanged.
[0112] The above embodiments are merely illustrative of the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A deep learning-based method for automating robotic processes, characterized in that, Includes the following steps: S1: Obtain the data acquisition request sent by the client and collect business data. The client has a robotic process automation program, and the business data includes structured data and unstructured data. S2: Preprocess the collected structured data, including data cleaning, denoising, and standardization; preprocess the collected unstructured data, including text segmentation, stop word removal, image enhancement, grayscale conversion, and denoising. S3: Perform feature extraction on the collected and preprocessed structured and unstructured data; generate multimodal feature vectors by feature fusion based on the feature extraction results; S4: Construct a multimodal prediction model using deep learning algorithms based on multimodal feature vectors; construct a recognition and classification model using deep learning algorithms; analyze anomalies in business data by constructing an anomaly detection model; integrate the multimodal prediction model, recognition and classification model, and anomaly detection model into the robotic process automation program and perform business process testing. S5: Comprehensively evaluate the performance of the robotic process automation (RPA) process and iteratively update the RPA process based on the evaluation results; Step S5 includes the following steps: The trained multimodal prediction model, classification model, and anomaly detection model were evaluated separately, and the key performance indicator F1 score was calculated for each. The model stability coefficient was calculated using the following formula: The model stability coefficients M are obtained; where, This represents the F1 score of a multimodal prediction model. This represents the F1 score of the classification model. These represent the F1 scores of the anomaly detection model; Business process efficiency is obtained by calculating the ratio of the processing time of automated business processes to that of manual business processes. ; By calculating the formula for business process stability: Achieve business process stability ;in, The error rate of the robotic process automation (RPA) is represented by T, and the response time of the RPA is represented by T. The performance formula for computational robotic process automation is comprehensively evaluated: Obtain the performance value of the robotic process automation process. ;in, , and These represent the weighting coefficients for model stability, business process efficiency, and business process stability, respectively. , , The sum between them is 1; Based on the performance calculation results of the robotic process automation (RPA), it is determined whether to iterate and update the RPA business process. If the real-time calculated RPA performance value is lower than the average historical performance value, the RPA business process is iterated and updated; otherwise, it is not iterated and updated.
2. The deep learning-based robotic process automation method according to claim 1, characterized in that, Step S1 includes the following steps: Use an API interface or web server to receive data retrieval requests sent by clients. The request content includes: the type and source of the business data. By utilizing deep learning models, the type and source of business data in the request content can be automatically identified; By acquiring a sample set of business data, which includes various business data types and sources; wherein, the business data types include: structured business data and unstructured business data; the structured business data includes: fields and data values in databases or tables; the unstructured business data includes: text, video images, and audio; the business data sources include: databases and web pages; Each business data sample is labeled with its data type and source, and the labeled business data sample set is input into a deep learning model for training; based on the trained deep learning model, the request content is input into the deep learning model to automatically identify the type and source of business data in the request content; Based on the type and source of business data automatically identified by the deep learning model, the Robotic Process Automation (RPA) program is used to simulate manual operation processes; the RPA program is connected to a database or web page; by executing the RPA script, the data collection operation is automatically completed and the results are output, which are business data collected from the database or web page.
3. The deep learning-based robotic process automation method according to claim 1, characterized in that, Step S3 involves feature extraction of the collected and preprocessed structured and unstructured data, including the following steps: Feature extraction is performed on the collected and preprocessed structured data, including numerical features and categorical variable features; the numerical features are normalized or standardized; and the categorical variables are converted into a set of binary features using one-hot encoding, where each categorical variable feature represents a category. Based on the preprocessed text, word embeddings or sentence embeddings are extracted using natural language processing techniques. Based on the preprocessed video images, the video image data is processed using OpenCV computer vision technology to extract video image features, including color features, texture features, and shape features; Based on the preprocessed language, the audio signal is segmented into 25-millisecond frames, and language features such as Mel frequency cepstral coefficients, spectral centroid, and spectral bandwidth are extracted from each frame.
4. The deep learning-based robotic process automation method according to claim 3, characterized in that, Step S3, which involves feature fusion based on the feature extraction results to generate a multimodal feature vector, includes the following steps: Based on the extracted structured data features, text features, video image features, and speech features, feature alignment is performed using timestamps, and feature dimensionality reduction is performed using principal component analysis. The attention mechanism is used to calculate the weight of each feature from the structured data features, text features, video image features, and speech features after feature alignment and dimensionality reduction. The weighted features are then fused to emphasize important features. Joint representation learning is performed through shared network layers or cross-modal loss functions. A multi-layer perceptron network is constructed to input the features after joint representation learning into the multi-layer perceptron network to generate multi-modal feature vectors.
5. The deep learning-based robotic process automation method according to claim 1, characterized in that, Step S4, which involves constructing a multimodal prediction model using a deep learning algorithm based on the multimodal feature vectors, includes the following steps: Based on the generated multimodal feature vectors, a multimodal prediction model is constructed using a deep learning model to analyze and predict business data. The multimodal feature vectors are input into the multimodal prediction model for training; The system collects and preprocesses business data in real time, extracts new multimodal vectors from the real-time collected business data, inputs the new multimodal vectors into the trained multimodal prediction model for analysis and prediction, and outputs the prediction results, which are the predicted trends of business data changes.
6. The deep learning-based robotic process automation method according to claim 1, characterized in that, Step S4 involves constructing a recognition and classification model using deep learning algorithms, including the following steps: By using deep learning algorithms to build a recognition and classification model, multimodal feature vectors are input into the recognition and classification model for training. During the training process, supervised learning is performed using labeled data, and the recognition and classification model is optimized using backpropagation algorithm and gradient descent method. The real-time collected and preprocessed business data is input into the trained recognition and classification model; the recognition and classification model performs behavioral pattern recognition and classification on the input business data.
7. The deep learning-based robotic process automation method according to claim 1, characterized in that, Step S4 involves analyzing anomalies in business data by constructing an anomaly detection model, including the following steps: An anomaly detection model is constructed based on a deep learning algorithm; multimodal feature vectors are input into the anomaly detection model for training; during the training process, iterative optimization is performed to enable the anomaly detection model to learn the distribution characteristics of normal business data; The anomaly detection model is trained by inputting real-time collected and preprocessed business data. The model evaluates the input business data based on the learned normal data distribution characteristics and detects whether the real-time business data is abnormal.
8. The deep learning-based robotic process automation method according to claim 1, characterized in that, Step S4 involves integrating the multimodal prediction model, the identification and classification model, and the anomaly detection model into the robotic process automation program and conducting business process testing, including the following steps: Use the designer of the Robotic Process Automation tool to create a framework for automated processes, including: the start point, end point, intermediate steps, and decision points of the business process; The trained multimodal prediction model, recognition and classification model, and anomaly detection model are deployed to the server respectively; Design automated business processes in the designer of robotic process automation tools; Through API interfaces, calls to multimodal prediction models, recognition and classification models, and anomaly detection models are embedded in the robotic process automation scripts, and an anomaly handling mechanism is designed in the automated business process; when a call fails or returns an unpredictable result in one of the models, an anomaly warning is issued; otherwise, the various models are called for analysis. Each component in the business process is tested individually, and the business process is also integrated for testing. By testing the entire robotic process automation business process, it is determined whether all components work together. Based on the test results, the robotic process automation business process is optimized. The tested and optimized robotic process automation business process is deployed to the production environment to begin automated execution of the business process. Based on the deployed robotic process automation (RoLA) business processes, key performance indicators, including processing time, error rate, and response time, are collected by monitoring the operational status of the RoLA business processes in real time.