Method and system for realizing automatic downloading of incremental data based on RPA

Abstract

The invention discloses an automatic incremental data downloading method and system based on RPA, and relates to the technical field of computers.The method comprises the steps that parameters are configured in a configuration interface, and the configured parameters are stored in a configuration file through the encryption technology; reading the encrypted parameters from the configuration file by applying a robot process automation technology, and writing an automatic script according to the service architecture of each website to execute incremental data query; converting the incremental data into a data file based on the query result, uploading the data file to a server, and recording an uploading log; and downloading and storing the data file, obtaining a monitoring result in combination with an uploaded log, and sending a notification. According to the invention, by simulating and copying manual operation steps, the RPA can rapidly superpose an automation function on the existing system, so that the project period is greatly shortened, and the speed of responding to business requirements is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method and system for automatically downloading incremental data based on RPA. Background Art

[0002] With the continued prosperity of the financial industry and the increasing complexity of business models, bank branches, as the forefront of financial services, are facing increasingly severe operational risk challenges during their operations. These risks not only affect the security of funds and customer trust, but also directly affect the competitiveness and market position of banks. In this context, traditional operational risk monitoring and management methods seem to be inadequate, and they are often highly dependent on manual operations. This process is not only inefficient and costly, but also prone to frequent errors due to human factors, thereby weakening the accuracy and timeliness of risk prevention and control.

[0003] At present, the existing technical solutions mostly focus on manual or semi-automatic data processing modes, such as the widespread use of Excel and other electronic spreadsheet tools for data collection, aggregation and analysis. Although this method has improved the standardization of data processing to a certain extent, its limitations are becoming increasingly prominent in the face of massive and dynamically changing financial outlets filing information: First, it is inefficient. A large amount of repetitive and mechanical data entry and verification work occupies a large amount of human resources, making it difficult to improve overall operational efficiency; second, the error rate is high. Long-term and high-intensity manual operations are very likely to cause fatigue, thereby increasing the probability of data entry errors, omissions and other problems; third, the real-time performance is poor. Limited by the response speed of manual operations, the system is difficult to achieve real-time monitoring and abnormal warning of filing information, which to a certain extent weakens the bank's ability to respond quickly to potential risks.

[0004] Currently, no effective solution has been proposed for the problems in the related technologies. Summary of the invention

[0005] In response to the problems in the related technology, the present invention proposes a method and system for automatically downloading incremental data based on RPA to overcome the above-mentioned technical problems existing in the existing related technology.

[0006] To this end, the specific technical solution adopted by the present invention is as follows:

[0007] According to one aspect of the present invention, a method for automatically downloading incremental data based on RPA is provided, and the method comprises the following steps:

[0008] S1. Configure parameters in the configuration interface and store the configured parameters in the configuration file through encryption technology;

[0009] S2. Use robotic process automation technology to read encrypted parameters from configuration files and write automated scripts to perform incremental data queries based on the business architecture of each branch;

[0010] S3. Convert the incremental data into data files based on the query results, upload the data files to the server, and record the upload log;

[0011] S4. Download and save the data file, obtain the monitoring results in combination with the uploaded log and send a notification.

[0012] Furthermore, configuring parameters in the configuration interface and storing the configured parameters in the configuration file through encryption technology includes the following steps:

[0013] S11. Design the configuration interface according to the monitoring network requirements;

[0014] S12, input the parameters into the configuration page and execute the corresponding configuration;

[0015] S13, retrieving the symmetric key from the key management tool, and encrypting the configured parameters using a symmetric encryption algorithm;

[0016] S14. Store the encrypted parameters in a configuration file.

[0017] Furthermore, the parameters include a list of outlets, outlet information, and login credentials.

[0018] Furthermore, the encrypted parameters are read from the configuration file using the robotic process automation technology, and the automated script is written according to the business architecture of each branch to perform incremental data query, including the following steps:

[0019] S21. Use robotic process automation technology to read the encrypted parameters from the configuration file, retrieve the symmetric key from the key management tool, and recover the parameters through the decryption algorithm;

[0020] S22, extracting a list of outlets from the restored parameters, and opening a login interface of the financial management system based on the list of outlets;

[0021] S23, read the login credentials in the recovery parameters, and write an automation script based on the business architecture of each branch;

[0022] S24. Based on the automated script, fill in and submit the login form to enter the incremental data query interface;

[0023] S25. In combination with the incremental data query interface and the preset query conditions, an operation of querying the incremental data of each outlet is executed.

[0024] Furthermore, the business architecture includes business processes and page structures.

[0025] Furthermore, in combination with the incremental data query interface and the preset query conditions, the operation of querying the incremental data of each outlet includes the following steps:

[0026] S251. Navigate to the incremental data query interface according to the robotic process automation technology and the preset path language;

[0027] S252. Based on the incremental data query interface, the preset query conditions are applied in conjunction with the asynchronous processing technology to execute the query operation, retrieve the incremental data of each outlet, and identify the unprocessed data.

[0028] Further, based on the query result, the incremental data is converted into a data file, and the data file is uploaded to the server. Recording the upload log includes the following steps:

[0029] S31, using the data analysis operation library to read the incremental data in the query result;

[0030] S32, mapping the incremental data in the query result to a preset data structure model to obtain a structured format;

[0031] S33. Export the structured format as a data file using a standard library module according to configuration requirements;

[0032] S34. Read the login credentials in the configuration file, upload the data file to the server through the communication interface provided by the object storage service, and record the upload log.

[0033] Furthermore, the data file is a dat file.

[0034] Further, downloading and saving the data file, obtaining the monitoring results in combination with the uploaded log and sending a notification include the following steps:

[0035] S41, downloading and saving the uploaded data file from the server;

[0036] S42, based on the saved data file, searching for the corresponding upload log to obtain the monitoring result;

[0037] S43, using idempotent operations to update the monitoring results to corresponding monitoring records;

[0038] S44. Send a notification based on the updated monitoring record.

[0039] According to another aspect of the present invention, a system for automatically downloading incremental data based on RPA is also provided, the system comprising a parameter configuration and encryption storage module, an incremental data query module, a data file conversion and upload module, and a monitoring result acquisition and notification sending module;

[0040] The parameter configuration and encryption storage module is used to configure the parameters in the configuration interface and store the configured parameters in the configuration file through encryption technology;

[0041] The incremental data query module is used to read encrypted parameters from the configuration file using robotic process automation technology, and to write automated scripts to perform incremental data queries based on the business architecture of each branch;

[0042] The data file conversion and upload module is used to convert incremental data into data files based on query results, upload the data files to the server, and record the upload log;

[0043] The monitoring result acquisition and notification sending module is used to download and save data files, obtain monitoring results in combination with uploaded logs and send notifications.

[0044] The beneficial effects of the present invention are:

[0045] 1. Through the application of RPA technology, the present invention can focus on the automation of business processes. By simulating and replicating manual operation steps, RPA can quickly add automation functions to existing systems. This process greatly shortens the project cycle and significantly improves the speed of responding to business needs. At the same time, the deployment cost of RPA is also greatly reduced, making the implementation of automated processes more economical and efficient.

[0046] 2. The present invention uses RPA technology to simulate manual operations to interact seamlessly with existing systems. There is no need to make any structural changes or customized development to the original information system, ensuring that the implementation of RPA will not infringe on or affect the normal operation of the core system, thereby protecting the stability and security of the existing system to the greatest extent, avoiding complex integration processes, significantly reducing integration risks, and making the deployment of automated processes smoother and more reliable.

[0047] 3. The present invention uses RPA technology to quickly reconstruct processes and expand functions. By conveniently modifying and re-recording automation scripts, RPA can easily adapt to new needs without investing a lot of development and testing resources. This flexibility makes RPA an ideal choice for coping with ever-changing business environments.

[0048] 4. The present invention can accurately replicate the processes and details of manual operations through RPA, greatly reducing the problems caused by human errors. This not only improves work efficiency, but also helps reduce business risks caused by manual operation errors.

[0049] 5. The present invention allows real-time tracking and monitoring of the execution status of automated tasks through a graphical configuration interface, automatically records detailed log information for each operation step, provides a reliable reference for subsequent audits and troubleshooting, and significantly enhances the transparency and reliability of the automated system. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0051] Figure 1 is a flowchart of a method for automatically downloading incremental data based on RPA according to an embodiment of the present invention;

[0052] Figure 2 is a workflow diagram of a method for automatically downloading incremental data based on RPA according to an embodiment of the present invention;

[0053] Figure 3 is a flow chart of an embodiment of a method for automatically downloading incremental data based on RPA according to an embodiment of the present invention;

[0054] Figure 4 It is a principle block diagram of a system for automatically downloading incremental data based on RPA according to an embodiment of the present invention.

[0055] In the figure:

[0056] 1. Parameter configuration and encryption storage module; 2. Incremental data query module; 3. Data file conversion and upload module; 4. Monitoring result acquisition and notification sending module. DETAILED DESCRIPTION

[0057] To further illustrate each embodiment, the present invention provides drawings, which are part of the disclosure of the present invention and are mainly used to illustrate the embodiments. They can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these contents, ordinary technicians in this field should be able to understand other possible implementation methods and advantages of the present invention.

[0058] According to an embodiment of the present invention, a method and system for automatically downloading incremental data based on RPA are provided.

[0059] The present invention will now be further described with reference to the accompanying drawings and specific embodiments. Figure 1-Figure 3 As shown, according to an embodiment of the present invention, a method for automatically downloading incremental data based on RPA includes the following steps:

[0060] S1. Configure parameters in the configuration interface and store the configured parameters in the configuration file through encryption technology.

[0061] Specifically, configuring parameters in the configuration interface and storing the configured parameters in the configuration file through encryption technology includes the following steps:

[0062] S11. Design the configuration interface according to the requirements of monitoring points.

[0063] S12. Input the parameters into the configuration page and execute the corresponding configuration.

[0064] It should be noted that the corresponding configuration includes: configuring the list of outlets that need to be monitored, configuring the name and login account password of each outlet, configuring the account management system to be used and its login method, and configuring the specific content of monitoring, such as late filing, incorrect filing account and account name, etc.

[0065] S13. Retrieve the symmetric key from the key management tool, and encrypt the configured parameters using a symmetric encryption algorithm.

[0066] It should be noted that the login account passwords of each branch and the login account passwords of related system servers are encrypted and stored.

[0067] S14. Store the encrypted parameters in a configuration file.

[0068] Specifically, the parameters include the branch list, branch information and login credentials.

[0069] It should be noted that configuration files or databases are used to manage the information of outlets and login credentials that need to be monitored; encryption storage technology (such as AES, RSA, etc.) is used to protect the security of sensitive information; and a graphical configuration interface is provided to facilitate the administrator to set and maintain parameters. When the administrator configures sensitive information such as account passwords through the configuration interface, the data will be encrypted using the following encryption method and saved in the database, where the key is saved in the key tool client, and the Robotic Process Automation (RPA) calls the decryption algorithm to fill in the system input box.

[0070] from Crypto.Cipher import AES

[0071] from Crypto.Random import get_random_bytes

[0072] from Crypto.Util.Padding import pad

[0073] from binascii import unhexlify

[0074] def aes_gcm_encrypt(plain_text,key):

[0075] #Convert key into a byte string

[0076] key=key.encode()if isinstance(key,str)else key

[0077] #Generate a random nonce, usually 12 bytes long

[0078] nonce = get_random_bytes(12)

[0079] #Create an AES GCM mode encryptor

[0080] cipher=AES.new(key,AES.MODE_GCM,nonce=nonce)

[0081] #Padding the plain text

[0082] padded_data=pad(plain_text.encode(),AES.block_size)

[0083] #Encrypted data

[0084] ciphertext,tag=cipher.encrypt_and_digest(padded_data)

[0085] #Return nonce, ciphertext and tag, because they are needed for decryption

[0086] return nonce,ciphertext,tag

[0087] def aes_gcm_decrypt(nonce,ciphertext,tag,key):

[0088] #Convert key into a byte string

[0089] key=key.encode()if isinstance(key,str)else key

[0090] #Create an AES GCM mode decryptor

[0091] cipher=AES.new(key,AES.MODE_GCM,nonce=nonce)

[0092] #Decrypt data

[0093] try:

[0094] decrypted_padded=cipher.decrypt_and_verify(ciphertext,tag)

[0095] except(ValueError,KeyError):

[0096] raise ValueError("Incorrect decryption")

[0097] #Remove padding and restore plaintext

[0098] plain_text=decrypted_padded.decode()

[0099] return plain_text

[0100] In addition, encryption algorithms such as AES and RSA are used to protect sensitive information, such as account passwords, etc.; security management mechanisms such as regular key updates and audit logs are implemented.

[0101] S2. Use robotic process automation technology to read encrypted parameters from configuration files and write automated scripts to perform incremental data queries based on the business architecture of each branch.

[0102] Specifically, using robotic process automation technology to read encrypted parameters from the configuration file and writing an automated script to perform incremental data query according to the business architecture of each branch includes the following steps:

[0103] S21. Use robotic process automation technology to read the encrypted parameters from the configuration file, retrieve the symmetric key from the key management tool, and restore the parameters through the decryption algorithm.

[0104] S22. Extract the branch list from the restored parameters, and open the login interface of the financial management system based on the branch list.

[0105] S23. Read the login credentials in the recovery parameters, and write an automation script based on the business architecture of each branch.

[0106] Specifically, the business architecture includes business processes and page structures.

[0107] S24. Based on the automated script, fill in and submit the login form to enter the incremental data query interface.

[0108] It should be noted that RPA reads the list of outlets to be processed; according to the list, it opens the account management system website through the browser and enters the login interface; reads the login account number and password of the outlet that currently needs to be processed; and automatically fills in the account number and password to log in to the account management system website. The specific implementation method is:

[0109] Use RPA technology to achieve automatic login to the account management system. First, RPA will read the list of outlets that need to be logged in stored in the configuration file; then, RPA will simulate manual operation and open the browser to enter the login page of the account management system; next, RPA will read the login account and password stored in the configuration file, and automatically fill in and submit the login form based on the automation (JavaScript) script method.

[0110] In addition, to improve the success rate of login, RPA will set up a retry mechanism; if the first login fails, RPA will try to log in again several times until it succeeds or reaches the maximum number of retries. The reasons for login failure may include: network interruption, system maintenance, account and password errors, etc. If the automatic login fails, RPA will record the error log and send an alarm to notify relevant personnel to handle it.

[0111] S25. In combination with the incremental data query interface and the preset query conditions, an operation of querying the incremental data of each outlet is executed.

[0112] Specifically, in combination with the incremental data query interface and the preset query conditions, the operation of querying the incremental data of each outlet includes the following steps:

[0113] S251. Navigate to the incremental data query interface according to the robotic process automation technology and the preset path language;

[0114] S252. Based on the incremental data query interface, the preset query conditions are applied in conjunction with the asynchronous processing technology to execute the query operation, retrieve the incremental data of each outlet, and identify the unprocessed data.

[0115] It should be noted that RPA enters the incremental data query interface in the corresponding way of each outlet according to the different situations of each outlet; RPA queries all incremental data on the day or within the specified date range; RPA filters unprocessed data from all incremental data. The specific implementation method is as follows:

[0116] RPA will design targeted automated scripts to complete the query of incremental data based on the business processes and page structures of different outlets. First, RPA will locate the entry page for incremental data query through Xpath and automatically fill in the query conditions, such as the query time range.

[0117] To improve query efficiency, RPA uses asynchronous processing or multi-threading to query incremental data from multiple outlets simultaneously. During the query process, RPA monitors page loading in real time, automatically detecting and handling possible abnormal situations such as network interruptions and system response timeouts.

[0118] After the query is completed, RPA will filter out the unprocessed data from the returned incremental data to prepare for subsequent data processing; in the event of a query failure, RPA will record the error message and retry several times until the query is successful or the maximum number of retries is reached.

[0119] S3. Convert the incremental data into data files based on the query results, upload the data files to the server, and record the upload log.

[0120] Specifically, converting incremental data into data files based on query results, uploading the data files to the server, and recording the upload logs include the following steps:

[0121] S31. Use the data analysis operation library to read the incremental data in the query results.

[0122] S32: Map the incremental data in the query result to a preset data structure model to obtain a structured format.

[0123] It should be noted that the data structure model is any data structure designed according to business requirements; for example, the JSON data structure of the record:

[0124] {

[0125] "branch_code":"BR001",

[0126] "branch_name":"ABC Bank Branch",

[0127] "filing_status":"pending",

[0128] "account_details":{

[0129] "account_number":"1234567890",

[0130] "account_name":"Zhang San"

[0131] },

[0132] "errors":[

[0133] "Mismatch in account holder name",

[0134] "Missing filing documents" ]

[0136] }

[0137] This JSON data structure model contains the following fields:

[0138] branch_code: branch code; branch_name: branch name; filing_status: filing status, possible values ​​include "pending", "approved", "rejected", etc.; account_details: account details; account_number: account number; account_name: account name; errors: error list, used to record error information in branch filing, such as account name mismatch, missing filing documents, etc.

[0139] Among them, RPA reads the content in the incremental data (such as branch name, registered account number, account name, etc.); RPA converts the read incremental data content into a standardized structured text. The specific implementation method is as follows:

[0140] RPA uses tools such as Python's data analysis library (Pandas) to read the content in the incremental data, such as branch name, registered account number, account name, etc.; then, RPA will design a unified data structure model to map the read data into a standardized structured format, such as Excel.

[0141] During the data mapping process, RPA will perform a series of data cleaning and format conversion operations to ensure data quality and consistency. For example, for fields such as the registered account number and account name, RPA will perform format verification to remove extra spaces or special characters; for data with errors or missing data, RPA will mark them for subsequent manual review and correction.

[0142] import pandas as pd

[0143] #Read data

[0144] data=pd.read_csv('data.csv')

[0145] #Format validation and data cleaning

[0146] def clean_data(row):

[0147] #Add your data cleaning logic here, such as removing extra spaces or special characters

[0148] cleaned_row = row.copy()

[0149] cleaned_row['Record Account'] = cleaned_row['Record Account'].strip() #Remove spaces

[0150] cleaned_row['Account Name'] = cleaned_row['Account Name'].strip() # remove spaces

[0151] #Add other required data cleaning operations

[0152] return cleaned_row

[0153] data_cleaned=data.apply(clean_data,axis=1)

[0154] #Write the results to Excel and mark errors or missing data

[0155] def mark_errors(row):

[0156] #Add your error marker logic here

[0157] if pd.isnull(row['Account'])or pd.isnull(row['Account name'])or'Special case'inrow['Other fields']:

[0158] return 'error'

[0159] else:

[0160] return 'normal'

[0161] data_cleaned['data status'] = data_cleaned.apply(mark_errors,axis=1)

[0162] #Write the results to Excel

[0163] data_cleaned.to_excel('cleaned_data.xlsx',index=False)

[0164] Through this standardized data processing method, subsequent data conversion and uploading operations will be more efficient and reliable.

[0165] In addition, a retry mechanism is implemented for key operations (such as file uploading) to improve the success rate; exception capture and fault-tolerant processing are used to ensure the stability of system operation; and a monitoring and alarm function is implemented to promptly discover and resolve problems in system operation.

[0166] S33. According to the configuration requirements, use the standard library module to export the structured format as a data file.

[0167] Specifically, the data file is a dat file.

[0168] It should be noted that RPA converts incremental data from Excel format to DAT format according to configuration requirements; RPA automatically generates DAT files and ensures the accuracy of file format and content. The specific implementation method is as follows:

[0169] After completing the standardization of incremental data, RPA will automatically convert the data into a file of a specified format according to the system configuration requirements. For example, converting data from Excel format to DAT format.

[0170] import pandas as pd

[0171] #Read Excel file

[0172] excel_file = 'example.xlsx' #Please replace with your Excel file path

[0173] df=pd.read_excel(excel_file)

[0174] #Save the data as a .dat file

[0175] dat_file = 'output.dat' # Output .dat file path

[0176] df.to_csv(dat_file,sep='\t',index=False)

[0177] RPA will use tools such as Python's standard library module (csv module) to export standardized structured data into dat files. In the process of generating dat files, RPA will strictly follow the format specifications of dat files to ensure the accuracy of the file content. At the same time, RPA will also support flexible file naming rule configuration to meet the requirements of different systems.

[0178] To improve the reliability of file generation, RPA will perform format and content verification on the generated dat file. For example, it will check whether the format of the file header and tail is correct and whether the values ​​of each field meet expectations. If any abnormality is found, RPA will record the error information and notify relevant personnel for manual intervention.

[0179] S34. Read the login credentials in the configuration file, upload the data file to the server through the communication interface provided by the object storage service, and record the upload log.

[0180] It should be noted that RPA reads the login information of the configured related system server, and uploads the generated dat file to the related system server through OSS (object storage service). RPA ensures that the file is uploaded successfully and records the upload log. The specific implementation method is as follows:

[0181] After generating the dat file, RPA will automatically upload the file to the relevant system server. RPA will first read the server login information stored in the configuration file, including IP address, account, password, etc.; then, RPA will use the software development tool set (SDK, Software Development Kit) or communication interface (API interface) provided by the object storage service (such as Alibaba Cloud OSS, AWS S3, etc.) to automatically upload the file. The object storage service can provide secure and reliable file upload capabilities, and supports breakpoint resume, concurrent upload and other functions to improve the efficiency and stability of file upload.

[0182] During the file upload process, RPA will monitor the upload status in real time to ensure that the file is uploaded successfully. If the upload fails due to network interruption or server abnormality, RPA will automatically retry several times until the file is successfully uploaded. If the automatic upload fails, RPA will record the error log and send an alarm to notify relevant personnel to handle it.

[0183] At the same time, RPA will also record detailed log information for each file upload, including upload time, file name, upload status, etc.; this log information can be used for subsequent monitoring and auditing.

[0184] S4. Download and save the data file, obtain the monitoring results in combination with the uploaded log and send a notification.

[0185] Specifically, downloading and saving data files, obtaining monitoring results in combination with uploaded logs and sending notifications include the following steps:

[0186] S41, downloading and saving the uploaded data file from the server;

[0187] S42, based on the saved data file, searching for the corresponding upload log to obtain the monitoring result;

[0188] S43, using idempotent operations to update the monitoring results to corresponding monitoring records;

[0189] S44. Send a notification based on the updated monitoring record.

[0190] It should be noted that RPA downloads and saves the uploaded confirmation file; RPA finds the corresponding monitoring record in the relevant system; RPA uploads the monitoring results to the monitoring record and sends a notification to the relevant person in charge. The specific implementation method is as follows:

[0191] After the file is uploaded, RPA will perform the feedback and notification operations of the monitoring results. First, RPA will download and save the uploaded confirmation file for subsequent monitoring result query and comparison.

[0192] import requests

[0193] url='http: / / www.example.com / confirmation_file'

[0194] response = requests.get(url)

[0195] if response.status_code==200:

[0196] with open('confirmation_file.txt','wb')as file:

[0197] file.write(response.content)

[0198] print("Confirm that the file was downloaded successfully.")

[0199] else:

[0200] print("Confirmation file download failed.")

[0201] Then, RPA will find the corresponding monitoring records in the relevant system; based on the content of the confirmation file, RPA will automatically update the monitoring results to the corresponding monitoring records; for the update of monitoring results, RPA will take idempotent operations to ensure that each update is safe and reliable.

[0202] import pandas as pd

[0203] #Read confirmation file

[0204] confirmation_data=pd.read_csv('confirmation_file.txt')

[0205] #Connect to the database or API to update monitoring records

[0206] for index,row in confirmation_data.iterrows():

[0207] monitoring_id = row['monitoring record ID']

[0208] monitoring_result = row['monitoring result']

[0209] #Implement the logic of updating monitoring records here, which can be a database update operation or API request

[0210] #Example: update_monitoring_record(monitoring_id,monitoring_result)

[0211] print("Monitoring record update completed.")

[0212] Idempotent operations mean that no matter whether an operation is performed once or multiple times, the results produced are the same. In the scenario of monitoring result updates, this means that even if the update request is sent repeatedly due to network failure, system error or other unexpected circumstances, there will be no inconsistent or erroneous changes to the monitoring results. The specific implementation steps are as follows:

[0213] Unique identifier: A unique identifier is assigned to each monitoring result update operation. This identifier can be generated based on the monitoring task number, timestamp, and hash value of related data to ensure its uniqueness. For example, for a task that monitors website visits every hour, "website name_hours_current timestamp_data hash value" can be used as a unique identifier.

[0214] Request verification: When RPA receives an update request, it first verifies whether the unique identifier in the request already exists in the system. For example, if it finds that the same unique identifier has been processed, it can directly ignore subsequent duplicate requests.

[0215] Data backup: Before executing the update operation, back up the original monitoring result data. If an accident occurs during the update process, the backup data can be used for recovery to ensure the integrity and accuracy of the data.

[0216] Finally, RPA will send a notification to the relevant person in charge to inform them of the updated monitoring results. The notification method can be SMS, email, or message queue, etc., which is determined by the settings in the configuration file; the notification content includes a summary of the monitoring results and a link to the relevant monitoring records, which is convenient for the person in charge to view and handle in time.

[0217] The above automated processing flow greatly improves the efficiency of feedback and notification of monitoring results, avoiding omissions and errors that may occur in manual operations; at the same time, it also provides reliable basic data for subsequent review and analysis of monitoring results.

[0218] import smtplib

[0219] from email.mime.text import MIMEText

[0220] #Set notification content

[0221] notification_content="""

[0222] The monitoring results have been updated successfully.

[0223] Summary Information:...

[0224] Monitoring record link:...

[0225] """

[0226] #Send email notification

[0227] def send_email_notification():

[0228] smtp_server='smtp.example.com'

[0229] sender_email = 'sender@example.com'

[0230] receiver_email = 'receiver@example.com'

[0231] msg=MIMEText(notification_content)

[0232] msg['Subject'] = 'Monitoring result update notification'

[0233] msg['From'] = sender_email

[0234] msg['To'] = receiver_email

[0235] with smtplib.SMTP(smtp_server)as server:

[0236] server.sendmail(sender_email,receiver_email,msg.as_string())

[0237] print("Email notification sent successfully.")

[0238] #Decide which notification method to use based on the settings in the configuration file

[0239] notification_method = 'email' # can be 'sms', 'email', 'message_queue', etc.

[0240] if notification_method=='email':

[0241] send_email_notification()

[0242] elif notification_method=='sms':

[0243] #Send SMS notification

[0244] pass

[0245] elif notification_method=='message_queue':

[0246] #Send message queue notification

[0247] pass

[0248] In addition, a modular design is used to divide system functions into independent components. Through configuration files or plug-in mechanisms, new account management system access and function expansion are supported. Both horizontal and vertical expansion are considered to meet the needs of future business growth.

[0249] Adopt standardized coding style and comment specifications to improve code readability; design a complete unit test and integration test mechanism to ensure the correctness of the functions; provide detailed user manuals and deployment documents to facilitate subsequent maintenance and iteration.

[0250] like Figure 4 As shown, according to another embodiment of the present invention, a system for automatically downloading incremental data based on RPA is also provided, the system comprising a parameter configuration and encryption storage module 1, an incremental data query module 2, a data file conversion and upload module 3, and a monitoring result acquisition and notification sending module 4;

[0251] The parameter configuration and encryption storage module 1 is used to configure the parameters in the configuration interface and store the configured parameters in the configuration file through encryption technology;

[0252] Incremental data query module 2, used to read encrypted parameters from configuration files using robotic process automation technology, and write automated scripts to perform incremental data queries based on the business architecture of each branch;

[0253] The data file conversion and upload module 3 is used to convert the incremental data into data files based on the query results, upload the data files to the server, and record the upload log;

[0254] The monitoring result acquisition and notification sending module 4 is used to download and save data files, obtain monitoring results in combination with uploaded logs and send notifications.

[0255] In summary, with the help of the above technical solution of the present invention, through the application of RPA technology, it is possible to focus on the automation of business processes. By simulating and copying manual operation steps, RPA can quickly superimpose automation functions on existing systems. This process greatly shortens the project cycle and significantly improves the speed of responding to business needs; at the same time, the deployment cost of RPA is also greatly reduced, making the implementation of automated processes more economical and efficient. By simulating manual operations through RPA technology, seamless interaction with existing systems is achieved without any structural changes or customized development of the original information system, ensuring that the implementation of RPA will not infringe or affect the normal operation of the core system, thereby protecting the stability and security of the existing system to the greatest extent, avoiding complex integration processes, significantly reducing integration risks, and making the deployment of automated processes more stable and reliable. Through RPA technology, processes can be quickly reconstructed and functionally expanded. By conveniently modifying and re-recording automated scripts, RPA can easily adapt to new needs without investing a lot of development and testing resources. This flexibility makes RPA an ideal choice for coping with the ever-changing business environment. RPA can accurately replicate the processes and details of manual operations, greatly reducing problems caused by human errors. This not only improves work efficiency, but also helps reduce business risks caused by manual operation errors. Through the graphical configuration interface, it allows real-time tracking and monitoring of the execution status of automated tasks, automatically records detailed log information for each operation step, and provides a reliable reference for subsequent audits and troubleshooting, significantly enhancing the transparency and reliability of the automation system.

[0256] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for automatically downloading incremental data based on RPA, characterized in that: The method comprises the following steps: S1. Configure parameters in the configuration interface and store the configured parameters in the configuration file through encryption technology; S2. Use robotic process automation technology to read encrypted parameters from configuration files and write automated scripts to perform incremental data queries based on the business architecture of each branch; S3. Convert the incremental data into data files based on the query results, upload the data files to the server, and record the upload log; S4. Download and save the data file, obtain the monitoring results in combination with the uploaded log and send a notification.

2. The method for automatically downloading incremental data based on RPA according to claim 1 is characterized in that: The step of configuring parameters in the configuration interface and storing the configured parameters in the configuration file by encryption technology comprises the following steps: S11. Design the configuration interface according to the monitoring network requirements; S12, input the parameters into the configuration page and execute the corresponding configuration; S13, retrieving the symmetric key from the key management tool, and encrypting the configured parameters using a symmetric encryption algorithm; S14. Store the encrypted parameters in a configuration file.

3. The method for automatically downloading incremental data based on RPA according to claim 2 is characterized in that: The parameters include a list of outlets, outlet information and login credentials.

4. The method for automatically downloading incremental data based on RPA according to claim 3 is characterized in that: The method of using the robotic process automation technology to read the encrypted parameters from the configuration file and writing an automated script to perform incremental data query according to the business architecture of each branch includes the following steps: S21. Use robotic process automation technology to read the encrypted parameters from the configuration file, retrieve the symmetric key from the key management tool, and recover the parameters through the decryption algorithm; S22, extracting a list of outlets from the restored parameters, and opening a login interface of the financial management system based on the list of outlets; S23, read the login credentials in the recovery parameters, and write an automation script based on the business architecture of each branch; S24. Based on the automated script, fill in and submit the login form to enter the incremental data query interface; S25. In combination with the incremental data query interface and the preset query conditions, an operation of querying the incremental data of each outlet is executed.

5. The method for automatically downloading incremental data based on RPA according to claim 4 is characterized in that: The business architecture includes business processes and page structures.

6. The method for automatically downloading incremental data based on RPA according to claim 4 is characterized in that: The operation of querying the incremental data of each network point by combining the incremental data query interface with the preset query conditions includes the following steps: S251. Navigate to the incremental data query interface according to the robotic process automation technology and the preset path language; S252. Based on the incremental data query interface, the preset query conditions are applied in conjunction with the asynchronous processing technology to execute the query operation, retrieve the incremental data of each outlet, and identify the unprocessed data.

7. The method for automatically downloading incremental data based on RPA according to claim 1 is characterized in that: The converting of incremental data into data files based on the query results, uploading the data files to the server, and recording the upload logs include the following steps: S31, using the data analysis operation library to read the incremental data in the query result; S32, mapping the incremental data in the query result to a preset data structure model to obtain a structured format; S33. Export the structured format as a data file using a standard library module according to configuration requirements; S34. Read the login credentials in the configuration file, upload the data file to the server through the communication interface provided by the object storage service, and record the upload log.

8. The method for automatically downloading incremental data based on RPA according to claim 7 is characterized in that: The data file is a dat file.

9. The method for automatically downloading incremental data based on RPA according to claim 1 is characterized in that: The downloading and saving of data files, obtaining monitoring results in combination with uploading logs and sending notifications include the following steps: S41, downloading and saving the uploaded data file from the server; S42, based on the saved data file, searching for the corresponding upload log to obtain the monitoring result; S43, using idempotent operations to update the monitoring results to corresponding monitoring records; S44. Send a notification based on the updated monitoring record.

10. A system for automatically downloading incremental data based on RPA, used to implement the method for automatically downloading incremental data based on RPA according to any one of claims 1 to 9, characterized in that: The system includes a parameter configuration and encryption storage module, an incremental data query module, a data file conversion and upload module, and a monitoring result acquisition and notification sending module; The parameter configuration and encryption storage module is used to configure parameters in the configuration interface and store the configured parameters in the configuration file through encryption technology; The incremental data query module is used to read the encrypted parameters from the configuration file using the robotic process automation technology, and to write an automated script to perform incremental data query according to the business architecture of each branch; The data file conversion and upload module is used to convert the incremental data into data files based on the query results, upload the data files to the server, and record the upload log; The monitoring result acquisition and notification sending module is used to download and save data files, obtain monitoring results in combination with uploaded logs and send notifications.

Images