Method for automatically submitting electronic archive attachments
By combining data stream processing, machine learning, natural language processing, blockchain and cloud computing technologies, the full process of electronic file attachments is realized, and the process of excessive manual intervention, poor data accuracy and processing in the submission of traditional electronic file attachments is solved. The slow speed problem improves data processing efficiency and user experience, and enhances data security and system flexibility.
Patent Information
- Application Number
- CN202410978827.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-22
- Publication Date
- 2024-11-29
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
There are a lot of problems in the submission of existing electronic archive attachments, such as poor data accuracy and consistency, slow processing speed, poor user experience and insufficient large-scale data processing capabilities. The existing automation methods are used in real-time processing, intelligent preprocessing and Classification, multi-format support, upload and archive processes are not automated, and the processing capabilities are insufficient during peak periods.
Adopt data stream processing technology, machine learning algorithms, natural language processing technology, blockchain technology and cloud computing technology to realize the full process automation of real-time collection, preprocessing, classification, format conversion, uploading and archiving of electronic archive attachments, and support Multiple file formats, enhance data security, provide real-time notifications and multi-user collaboration functions, utilize AWS cloud computing to expand processing capabilities, and provide customized configuration interfaces.
Significantly reduce manual intervention, improve data accuracy and consistency, improve processing speed, enhance data security, improve user experience, meet large-scale data processing needs, and improve system flexibility and adaptability.
Smart Images

Figure CN119046237A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of electronic archive management, and in particular to a method for automatically submitting electronic archive attachments. Background Art
[0002] In modern management systems, the management and submission of electronic archives is an important component. The traditional electronic archive attachment submission process usually requires a lot of manual intervention, including file collection, organization, format conversion and uploading, which is not only time-consuming but also prone to errors. At the same time, with the increasing number of electronic archives, manual processing methods are difficult to cope with the submission and management needs of large-scale data.
[0003] In the prior art, the submission of electronic file attachments mostly relies on manual operations. This method has the following main defects:
[0004] 1. Too much manual intervention: The traditional electronic file attachment submission process requires manual collection and organization of files, and format conversion and uploading are also done manually. This not only consumes a lot of human resources, but is also prone to human errors, resulting in data loss or submission failure.
[0005] 2. Poor data accuracy and consistency: Data errors and inconsistencies often occur during manual processing. Different operators may have different processing methods for the same file, resulting in the inability to ensure data accuracy and consistency. This poses a great challenge to subsequent data management and retrieval.
[0006] 3. Insufficient large-scale data processing capabilities: With the continuous increase in the number of electronic archives, manual processing methods are difficult to cope with the submission and management needs of large-scale data. The existing manual processing methods are inefficient and cannot meet the needs of modern archive management systems for big data processing.
[0007] 4. Slow processing speed: Manual processing results in a slow submission speed for electronic file attachments, which cannot meet the business needs of rapid response. Especially during business peak periods, the bottleneck of manual processing is more obvious, which seriously affects the normal operation of the business.
[0008] 5. Poor user experience: The traditional manual processing method is complicated and users need to complete multiple steps to submit an electronic file attachment. For users who are not familiar with system operation, the manual processing method increases the learning cost and provides poor user experience.
[0009] In order to overcome the above defects, some automated electronic file attachment submission methods have emerged in recent years. These methods use computer technology and automation tools to try to reduce manual intervention and improve submission efficiency and data accuracy. For example, some systems use batch processing tools to upload and process multiple files at one time. However, these methods still have some shortcomings in practical applications:
[0010] 1. Limited real-time data processing capabilities: Many existing automation methods can only process static data and cannot achieve real-time collection and processing of electronic file attachments. Data delay problems still exist, affecting the timeliness and efficiency of submission.
[0011] 2. Insufficient intelligent preprocessing and classification: Most existing automated methods remain at the basic preprocessing stage and are unable to conduct in-depth analysis and intelligent classification of data. Preprocessing work such as data cleaning, formatting and deduplication still requires manual intervention, and intelligent classification and labeling functions are lacking.
[0012] 3. Imperfect multi-format support and automatic conversion: Although some systems support the processing of multiple file formats, the format conversion function is not perfect, and conversion errors or incompatibility are prone to occur. This limits the application scope of the automation method to a certain extent.
[0013] 4. Lack of automation in the uploading and archiving process: Although automated methods can improve the upload speed, the degree of automation in the archiving process is still not high. Functions such as data verification, version control, and permission management mostly rely on manual operations, affecting overall efficiency and security.
[0014] 5. Large-scale data processing capabilities need to be improved: Existing automation methods still have bottlenecks when processing large-scale data and cannot fully utilize cloud computing and distributed processing technologies. The system scalability and elasticity are insufficient and cannot adapt to peak data processing needs.
[0015] 6. Lack of user feedback and notification mechanism: Many automated methods cannot provide timely feedback to users on submission results after the submission process is completed. Users can only obtain submission status through manual query, which affects user experience and operational convenience.
[0016] Therefore, how to provide a method for automatically submitting electronic file attachments is a problem that those skilled in the art urgently need to solve. Summary of the invention
[0017] One object of the present invention is to propose a method for automatically submitting electronic archive attachments. The present invention makes full use of data stream processing technology, machine learning algorithms, natural language processing technology, blockchain technology and cloud computing technology to realize the full-process automated processing from real-time collection, preprocessing, classification, format conversion to uploading and archiving of electronic archive attachments. It has the advantages of reducing manual intervention, improving data accuracy and consistency, supporting multiple file formats, enhancing data security, increasing processing speed, improving user experience and having large-scale data processing capabilities.
[0018] A method for automatically submitting electronic file attachments according to an embodiment of the present invention includes the following steps:
[0019] S1. Use data stream processing technology to collect attachment data of electronic files in real time;
[0020] S2. Preprocess the collected attachment data, including data cleaning, formatting and deduplication, to ensure the accuracy and consistency of the data;
[0021] S3. Intelligently identify and classify the types of electronic file attachments based on the preprocessed data, and mark them;
[0022] S4. Supports multiple file formats and automatically converts attachment data into the target format. It integrates BERT natural language processing technology to automatically extract and analyze attachment content and generate key summaries and tags.
[0023] S5. Automatically upload and archive attachment data through API, and use blockchain technology to track and verify electronic file attachments to ensure data immutability and security;
[0024] S6. Perform data verification on the uploaded files to ensure the integrity and correctness of the files;
[0025] S7, automatically generate submission reports, and feedback submission results through the system, provide real-time notification function, to ensure that users know the submission status and results in time;
[0026] S8. Provide multi-user collaboration function, support multiple users to process and submit electronic file attachments at the same time, and provide version control and permission management;
[0027] S9. Use AWS cloud computing technology to expand the system's processing capacity to ensure efficient processing and submission speed even during peak periods;
[0028] S10. Provide a custom configuration interface to allow users to adjust data processing and submission processes according to specific needs, thereby improving the flexibility and adaptability of the system.
[0029] Optionally, the S3 specifically includes:
[0030] S31. Extract the initial features of the electronic file attachments through the preprocessed data:
[0031] F={f1,f2,...,f n};
[0032] Among them, f i represents a single feature of each attachment, and n is the total number of features;
[0033] S32. Use an intelligent recognition algorithm based on machine learning to analyze the initial feature set F and generate a feature vector for each attachment:
[0034] V i =[v i1 ,v i2 ,...,v im ];
[0035] Among them, V i represents the feature vector of the i-th attachment, and m is the dimension of the feature vector;
[0036] S33, for the feature vector V i Perform standardization and calculate the standardized eigenvector:
[0037] V′ i =[v′ i1 ,v′ i2 ,...,v′ im ];
[0038] The normalization formula is:
[0039]
[0040] Among them, v′ ij is the standardized eigenvalue, μ j is the average value of feature j, σ j is the three-dimensional standard deviation of feature j;
[0041] S34, using the support vector machine algorithm to normalize the feature vector V′ i Classify and generate classification label L i , L i represents the classification result of the i-th attachment;
[0042] S35, the classification result L i Verification and verification are performed to ensure classification accuracy and consistency, and the classification accuracy A is calculated, and the formula is:
[0043]
[0044] Where N is the total number of attachments, is the true label of the i-th attachment, δ(x,y) is an exponential function, when x=y, δ(x,y)=1, otherwise δ(x,y)=0;
[0045] S36, according to the classification result L i , mark the electronic file attachment and generate the file attachment D containing the marked information i =(A i ,L i ), where D i is the i-th attachment, A i is the content of the ith attachment, L i is the classification label of the i-th attachment;
[0046] S37. Use the graph neural network algorithm to further optimize the classified and annotated electronic file attachments, analyze the association relationship between the attachments, and generate optimized annotation information L′ i ;
[0047] S38, including optimization information L' i The archive attachments are stored in the database to form a collection of classified and annotated electronic archive attachments:
[0048] {D′1,D′2,...,D′ N}.
[0049] Optionally, the S4 specifically includes:
[0050] S41, receiving the classified and annotated electronic archive attachment set {D′1, D′2, ..., D′ N}, identify the format of the data file according to the annotation results;
[0051] S42, based on the identified file format and classification and annotation results, calling the corresponding format conversion module to convert the attachment data into a target format, where the target format is a preset standard format to ensure unified processing;
[0052] S43. During the format conversion process, BERT natural language processing technology is used to analyze the text attachments, including content extraction, sentiment analysis, topic modeling, and tagging;
[0053] S44, calling the image processing module to convert the image attachment data into a format, and standardizing the image data to meet the requirements of the electronic archive system;
[0054] S45. After the format conversion is completed, the data verification module is called to verify the integrity and correctness of the converted data to ensure that the data is not damaged or lost during the conversion process;
[0055] S46, storing the attachment data after format conversion, content extraction, sentiment analysis, topic modeling and tagging in a temporary database, preparing for subsequent automatic uploading and archiving processing.
[0056] Optionally, the BERT natural language processing technology specifically includes:
[0057] Use the BERT model to segment the text content of the attachment and convert the text content into word vector representation. Suppose the text content is T, the segmentation result is:
[0058] T={t1,t2,…,t n};
[0059] Among them, t i represents the i-th word;
[0060] The bidirectional encoder of the BERT model is used to understand the context of the word vector and generate the context representation of each word:
[0061] C={c1,c2,…,c n};
[0062] Among them, c i Represents the context vector representation of the i-th word;
[0063] Perform multi-head attention on the context representation and calculate the multi-head attention weight for each word
[0064]
[0065] in, It is the word t i The attention score at the h-th head is defined as:
[0066]
[0067] in, and b h is the trainable parameter of the h-th head, h i is the hidden state represented by the context vector;
[0068] Calculate the multi-head weighted context representation u h And concatenate:
[0069] u=concat(u 1 ,u 2 ,…,uH );
[0070] Where H is the number of heads, each u h The calculation formula is:
[0071]
[0072] Extract the key summary S from the context representation u and select representative word vectors through clustering or sorting algorithms:
[0073] S={s1,s2,…,s m};
[0074] Where m is the length of the summary;
[0075] Generate a summary text of the attachment content based on the extracted key summary, and add tags to the text using the preset annotation rules:
[0076] L={l1,l2,…,l p};
[0077] Among them, l i represents the i-th label;
[0078] Using sentiment analysis and topic modeling techniques, the extracted text summary S is further analyzed to generate sentiment score E and topic distribution T d .
[0079] Optionally, the S5 specifically includes:
[0080] S51. Automatically upload electronic file attachment data by using the API interface. First, define the endpoint URL and related parameters of the API, including the URL, HTTP method, request header, request body, and verification information;
[0081] S52, configuring an automated script in the system, calling the API interface to upload the attachment data, the script is responsible for reading the file to be uploaded, converting it into a suitable format, such as JSON or XML format, and sending it to the specified server through the POST method;
[0082] S53. During the upload process, the attachment data is verified using a hash algorithm, the hash value of the file is calculated, and the calculation result is compared with the hash value returned by the server to ensure the integrity and correctness of the uploaded file;
[0083] S54. Use blockchain technology to track and verify electronic file attachments, write the metadata of each uploaded attachment file into the blockchain, use smart contracts to automatically record each upload operation, and generate a unique transaction ID. Through the consensus mechanism of the blockchain network, ensure that each node can verify and record the authenticity and immutability of the upload operation;
[0084] S55. In the blockchain, the Merkle tree structure is used to organize and manage nearby data, and the root hash value of the Merkle tree is used to verify the integrity of the entire attachment data set.
[0085] Optionally, the S9 specifically includes:
[0086] S91. Configure and deploy AWS EC2 instances, select appropriate instance types and quantities based on the load requirements of the electronic file attachment submission system to ensure the system's processing capacity during peak periods;
[0087] S92. Use AWS Auto Scaling service to automatically adjust the number of EC2 instances according to changes in system load to dynamically use peak periods and underestimate load changes to ensure efficient use of resources;
[0088] S93. Configure and use AWS S3 to store electronic archive attachment data to ensure high availability, durability, and security of data; use S3's version control function to manage different versions of attachment data;
[0089] S94. Use AWS Lambda serverless computing services to handle data preprocessing, format conversion, intelligent identification and classification tasks, automatically execute related functions through trigger mechanisms, reduce dependence on fixed server resources, and improve system flexibility and response speed;
[0090] S95. Configure AWS RDS to manage and store metadata and processing logs of electronic archive attachments, ensure data consistency and efficient query, and use multi-availability zone deployment to improve database availability and fault recovery capabilities;
[0091] S96. Use AWS CloudWatch for system monitoring and log management, monitor the operating status and performance indicators of each service in real time, set alarm rules, promptly detect and handle system abnormalities, and ensure stable operation of the system;
[0092] S97. Configure AWS IAM to manage user and service access rights to ensure system security and data protection; define detailed permission policies to prevent unauthorized access and operations;
[0093] S98. Use AWS VPC to create an isolated network environment, configure subnets, routers, and network ACLs to ensure system network security and data transmission security;
[0094] S99, AWS Direct Connect enables secure connection between local data centers and AWS cloud environments, ensuring fast and secure data transmission and supporting hybrid cloud architecture;
[0095] S910, configure AWS Backup for regular data backup to ensure rapid recovery in the event of data loss or disaster; use AWS Glacier to store long-term backup data to reduce storage costs.
[0096] The beneficial effects of the present invention are:
[0097] (1) Reduce manual intervention: Utilize data stream processing technology to achieve real-time collection and processing of electronic file attachments. Combined with automated scripts and APIs, this significantly reduces the number of manual operation steps, improves overall processing efficiency, and reduces the possibility of human error.
[0098] (2) Improve data accuracy and consistency: Ensure data accuracy and consistency through data cleaning, formatting, deduplication, and in-depth analysis using the random forest algorithm. Intelligent classification and labeling algorithms further improve the accuracy of data management and avoid data bias and errors in manual operations.
[0099] (3) Support for multiple file formats and automatic conversion: The system supports multiple file formats and integrates BERT natural language processing technology, which can automatically extract and analyze attachment content, generate key summaries and tags, and ensure that electronic file attachments in different formats can be correctly processed and submitted, reducing the manual operation burden on users.
[0100] (4) Enhanced data security: Blockchain technology is used to track and verify electronic file attachments to ensure data immutability and security. The data verification function during the upload process further ensures the integrity and correctness of the data.
[0101] (5) Improve processing speed: Use AWS cloud computing technology to expand the system's processing capabilities. By configuring and deploying EC2 instances, using Auto Scaling services, and integrating serverless computing services (such as AWS Lambda), ensure that the system can maintain efficient processing and submission speeds during peak periods to meet business needs for rapid response.
[0102] (6) Improved user experience: The system provides automatic submission report generation and real-time notification functions to ensure that users can understand the submission status and results in a timely manner. Multi-user collaboration functions and version control, permission management and other mechanisms make system operation easier and significantly improve user experience.
[0103] (7) Large-scale data processing capabilities: The system is designed to handle large-scale electronic file attachments. Cloud computing technology is used to dynamically allocate and expand resources, ensuring that large amounts of data can be processed efficiently even during peak periods, thus meeting the needs of modern archive management systems for big data processing.
[0104] (8) Flexibility and adaptability: A custom configuration interface is provided to allow users to adjust the data processing and submission process according to specific needs. The flexibility and adaptability of the system are significantly improved, which can better meet the needs of different users and business scenarios. BRIEF DESCRIPTION OF THE DRAWINGS
[0105] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:
[0106] Figure 1 A flowchart of a method for automatically submitting electronic file attachments proposed by the present invention;
[0107] Figure 2 A schematic diagram of the multi-format support and automatic conversion process of a method for automatic submission of electronic archive attachments proposed by the present invention;
[0108] Figure 3 A schematic diagram of system expansion and resource management of a method for automatically submitting electronic archive attachments proposed by the present invention. DETAILED DESCRIPTION
[0109] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.
[0110] refer to Figure 1-3 , a method for automatically submitting electronic file attachments, comprising the following steps:
[0111] S1. Use data stream processing technology to collect attachment data of electronic files in real time;
[0112] S2. Preprocess the collected attachment data, including data cleaning, formatting and deduplication, to ensure the accuracy and consistency of the data;
[0113] S3. Intelligently identify and classify the types of electronic file attachments based on the preprocessed data, and mark them;
[0114] S4. Supports multiple file formats and automatically converts attachment data into the target format. It integrates BERT natural language processing technology to automatically extract and analyze attachment content and generate key summaries and tags.
[0115] S5. Automatically upload and archive attachment data through API, and use blockchain technology to track and verify electronic file attachments to ensure data immutability and security;
[0116] S6. Perform data verification on the uploaded files to ensure the integrity and correctness of the files;
[0117] S7, automatically generate submission reports, and feedback submission results through the system, provide real-time notification function, to ensure that users know the submission status and results in time;
[0118] S8. Provide multi-user collaboration function, support multiple users to process and submit electronic file attachments at the same time, and provide version control and permission management;
[0119] S9. Use AWS cloud computing technology to expand the system's processing capacity to ensure efficient processing and submission speed even during peak periods;
[0120] S10. Provide a custom configuration interface to allow users to adjust data processing and submission processes according to specific needs, thereby improving the flexibility and adaptability of the system.
[0121] In this implementation, S3 specifically includes:
[0122] S31. Extract the initial features of the electronic file attachments through the preprocessed data:
[0123] F={f1,f2,...,f n};
[0124] Among them, f i represents a single feature of each attachment, and n is the total number of features;
[0125] S32. Use an intelligent recognition algorithm based on machine learning to analyze the initial feature set F and generate a feature vector for each attachment:
[0126] V i =[v i1 ,v i2 ,...,v im ];
[0127] Among them, V irepresents the feature vector of the i-th attachment, and m is the dimension of the feature vector;
[0128] S33, for the feature vector V i Perform standardization and calculate the standardized eigenvector:
[0129] V′ i =[v′ i1 ,v′ i2 ,...,v′ im ];
[0130] The normalization formula is:
[0131]
[0132] Among them, v′ ij is the standardized eigenvalue, μ j is the average value of feature j, σ j is the three-dimensional standard deviation of feature j;
[0133] S34, using the support vector machine algorithm to normalize the feature vector V′ i Classify and generate classification label L i , L i represents the classification result of the i-th attachment;
[0134] S35, the classification result L i Verification and verification are performed to ensure classification accuracy and consistency, and the classification accuracy A is calculated, and the formula is:
[0135]
[0136] Where N is the total number of attachments, is the true label of the i-th attachment, δ(x,y) is an exponential function, when x=y, δ(x,y)=1, otherwise δ(x,y)=0;
[0137] S36, according to the classification result L i , mark the electronic file attachment and generate the file attachment D containing the marked information i =(A i ,L i ), where D i is the i-th attachment, A i is the content of the ith attachment, L i is the classification label of the i-th attachment;
[0138] S37. Use the graph neural network algorithm to further optimize the classified and annotated electronic file attachments, analyze the association relationship between the attachments, and generate optimized annotation information L′ i ;
[0139] S38, including optimization information L' i The archive attachments are stored in the database to form a collection of classified and annotated electronic archive attachments:
[0140] {D′1,D′2,...,D′ N}.
[0141] In this implementation, S4 specifically includes:
[0142] S41, receiving the classified and annotated electronic archive attachment set {D′1, D′2, ..., D′ N}, identify the format of the data file according to the annotation results;
[0143] S42, based on the identified file format and classification and annotation results, calling the corresponding format conversion module to convert the attachment data into a target format, where the target format is a preset standard format to ensure unified processing;
[0144] S43. During the format conversion process, BERT natural language processing technology is used to analyze the text attachments, including content extraction, sentiment analysis, topic modeling, and tagging;
[0145] S44, calling the image processing module to convert the image attachment data into a format, and standardizing the image data to meet the requirements of the electronic archive system;
[0146] S45. After the format conversion is completed, the data verification module is called to verify the integrity and correctness of the converted data to ensure that the data is not damaged or lost during the conversion process;
[0147] S46, storing the attachment data after format conversion, content extraction, sentiment analysis, topic modeling and tagging in a temporary database, preparing for subsequent automatic uploading and archiving processing.
[0148] In this implementation, the BERT natural language processing technology specifically includes:
[0149] Use the BERT model to segment the text content of the attachment and convert the text content into word vector representation. Suppose the text content is T, the segmentation result is:
[0150] T={t1,t2,…,t n};
[0151] Among them, t i represents the i-th word;
[0152] The bidirectional encoder of the BERT model is used to understand the context of the word vector and generate the context representation of each word:
[0153] C={c1,c2,…,c n};
[0154] Among them, c i Represents the context vector representation of the i-th word;
[0155] Perform multi-head attention on the context representation and calculate the multi-head attention weight for each word
[0156]
[0157] in, It is the word t i The attention score at the h-th head is defined as:
[0158]
[0159] in, and b h is the trainable parameter of the h-th head, h i is the hidden state represented by the context vector;
[0160] Calculate the multi-head weighted context representation u h And concatenate:
[0161] u=concat(u 1 ,u 2 , ..., u H );
[0162] Where H is the number of heads, each u h The calculation formula is:
[0163]
[0164] Extract the key summary S from the context representation u and select representative word vectors through clustering or sorting algorithms:
[0165] S={s1,s2,…,s m};
[0166] Where m is the length of the summary;
[0167] Generate a summary text of the attachment content based on the extracted key summary, and add tags to the text using the preset annotation rules:
[0168] L = {l1, l2, ..., l p};
[0169] Among them, l i represents the i-th label;
[0170] Using sentiment analysis and topic modeling techniques, the extracted text summary S is further analyzed to generate sentiment score E and topic distribution T d .
[0171] In this implementation manner, S5 specifically includes:
[0172] S51. Automatically upload electronic file attachment data by using the API interface. First, define the endpoint URL and related parameters of the API, including the URL, HTTP method, request header, request body, and verification information;
[0173] S52, configuring an automated script in the system, calling the API interface to upload the attachment data, the script is responsible for reading the file to be uploaded, converting it into a suitable format, such as JSON or XML format, and sending it to the specified server through the POST method;
[0174] S53. During the upload process, the attachment data is verified using a hash algorithm, the hash value of the file is calculated, and the calculation result is compared with the hash value returned by the server to ensure the integrity and correctness of the uploaded file;
[0175] S54. Use blockchain technology to track and verify electronic file attachments, write the metadata of each uploaded attachment file into the blockchain, use smart contracts to automatically record each upload operation, and generate a unique transaction ID. Through the consensus mechanism of the blockchain network, ensure that each node can verify and record the authenticity and immutability of the upload operation;
[0176] S55. In the blockchain, the Merkle tree structure is used to organize and manage nearby data, and the root hash value of the Merkle tree is used to verify the integrity of the entire attachment data set.
[0177] In this implementation manner, the S9 specifically includes:
[0178] S91. Configure and deploy AWS EC2 instances, select appropriate instance types and quantities based on the load requirements of the electronic file attachment submission system to ensure the system's processing capacity during peak periods;
[0179] S92. Use AWS Auto Scaling service to automatically adjust the number of EC2 instances according to changes in system load to dynamically use peak periods and underestimate load changes to ensure efficient use of resources;
[0180] S93. Configure and use AWS S3 to store electronic archive attachment data to ensure high availability, durability, and security of data; use S3's version control function to manage different versions of attachment data;
[0181] S94. Use AWS Lambda serverless computing services to handle data preprocessing, format conversion, intelligent identification and classification tasks, automatically execute related functions through trigger mechanisms, reduce dependence on fixed server resources, and improve system flexibility and response speed;
[0182] S95. Configure AWS RDS to manage and store metadata and processing logs of electronic archive attachments, ensure data consistency and efficient query, and use multi-availability zone deployment to improve database availability and fault recovery capabilities;
[0183] S96. Use AWS CloudWatch for system monitoring and log management, monitor the operating status and performance indicators of each service in real time, set alarm rules, promptly detect and handle system abnormalities, and ensure stable operation of the system;
[0184] S97. Configure AWS IAM to manage user and service access rights to ensure system security and data protection; define detailed permission policies to prevent unauthorized access and operations;
[0185] S98. Use AWS VPC to create an isolated network environment, configure subnets, routers, and network ACLs to ensure system network security and data transmission security;
[0186] S99, AWS Direct Connect enables secure connection between local data centers and AWS cloud environments, ensuring fast and secure data transmission and supporting hybrid cloud architecture;
[0187] S910, configure AWS Backup for regular data backup to ensure rapid recovery in the event of data loss or disaster; use AWS Glacier to store long-term backup data to reduce storage costs.
[0188] Embodiment 1:
[0189] In order to verify the feasibility of the present invention in implementation, the present invention is applied to the archive management department of a large enterprise. The department needs to process a large number of electronic archive attachments every day. These attachments include various types such as contracts, invoices, reports, etc., with different formats, such as PDF, DOCX, XLSX, etc. The traditional manual processing method requires manual collection, sorting, format conversion and uploading, which not only consumes a lot of time and human resources, but is also prone to errors, affecting the accuracy and consistency of data. In order to improve work efficiency and data quality, the enterprise decided to introduce the electronic archive attachment automatic submission method of the present invention.
[0190] First, through data stream processing technology, the electronic file attachment data uploaded by each department is collected in real time. The system is equipped with an efficient data stream processing framework to receive and process attachment data from different departments in real time. These data include PDF files of contracts, XLSX files of invoices, and DOCX files of various reports.
[0191] The collected attachment data first enters the pre-processing stage. The system uses data cleaning, formatting and deduplication technologies to pre-process the data to ensure data integrity and consistency. For example, for uploaded contract PDF files, the system will automatically remove duplicate files, correct format errors, and perform data cleaning to ensure that all files meet the predetermined standards.
[0192] The pre-processed data enters the intelligent identification and classification stage. The system uses an intelligent classification algorithm based on machine learning, which can automatically identify and classify different types of file attachments. For each uploaded file, the system automatically identifies its type (such as contract, invoice or report) and marks it accordingly.
[0193] In terms of multi-format support and conversion, the system supports the processing of multiple file formats and can automatically convert attachment data to the target format. For example, the system can automatically convert uploaded DOCX reports into PDF format to meet the requirements of the archive management system. The integrated BERT natural language processing technology enables the system to automatically extract and analyze the content of attachments and generate key summaries and tags. For example, for a contract document, the system can automatically extract the main terms and key information of the contract and generate corresponding tags for subsequent retrieval and management.
[0194] In the process of data uploading and archiving, the system realizes automated operation through API. All processed attachment data will be automatically uploaded to the enterprise's archive management system, and tracked and verified using blockchain technology to ensure the data is tamper-proof and secure.
[0195] In order to ensure the integrity and correctness of the uploaded files, the system will perform data verification on the files before uploading. For example, by verifying the hash value of the file, the system can ensure that the uploaded file is completely consistent with the original file, avoiding the risk of file damage or tampering.
[0196] The system also has the function of automatically generating submission reports and real-time notifications. Whenever a file is successfully uploaded, the system will automatically generate a submission report and provide feedback to the user via email or system notification.
[0197] The multi-user collaboration function enables multiple users to process and submit electronic file attachments at the same time. The system provides version control and permission management functions. For example, users from different departments can upload files at the same time. The system will automatically manage the file version and perform corresponding operations according to user permissions to ensure data security and consistency.
[0198] In order to cope with the needs of large-scale data processing, the system uses AWS cloud computing technology to expand processing capabilities. For example, during business peaks, the system automatically adjusts the number of computing resources through AWS EC2 instances and Auto Scaling services to ensure that the system can maintain efficient processing speed during peak periods. Through AWS Lambda serverless computing services, the system can flexibly handle tasks such as data preprocessing, format conversion, and intelligent classification, further improving the system's response speed and flexibility.
[0199] Table 1 Performance index comparison table of electronic archive attachment automatic submission system
[0200] index Traditional manual method Automation System Processing speed 5 minutes / portion 30 seconds / copy Data verification accuracy 95% 99.99% Manual intervention rate 30% <1% Peak processing speed Low 10 copies / second Customer satisfaction 70% 95%
[0201] It can be concluded from the table above that during the system trial operation phase in March 2024, the system processed about 5,000 electronic file attachments per day, and the processing speed was reduced from an average of 5 minutes per file in the traditional manual method to 30 seconds, and the overall efficiency was increased by 10 times. The accuracy of data verification reached 99.99%, and the manual intervention rate dropped from the original 30% to less than 1%. During the peak business period in March 2024, the system maintained a processing speed of 10 files per second during the peak period through the processing capacity expanded by AWS cloud computing technology, without any system crash or performance degradation. User feedback shows that the system's real-time notification function and multi-user collaboration function have significantly improved the user experience, and user satisfaction has reached more than 95%.
[0202] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.
Claims
1. A method for automatically submitting electronic file attachments, characterized in that: The steps include: S1. Use data stream processing technology to collect attachment data of electronic files in real time; S2. Preprocess the collected attachment data, including data cleaning, formatting and deduplication; S3. Intelligently identify and classify the types of electronic file attachments based on the preprocessed data, and mark them; S4. Supports multiple file formats and automatically converts attachment data into the target format. It integrates BERT natural language processing technology to automatically extract and analyze attachment content and generate key summaries and tags. S5. Automatically upload and archive attachment data through API, and use blockchain technology to track and verify electronic file attachments; S6. Perform data verification on the uploaded file; S7, automatically generate submission reports, and provide real-time notification function through system feedback of submission results; S8. Provide multi-user collaboration function, support multiple users to process and submit electronic file attachments at the same time, and provide version control and permission management; S9. Use AWS cloud computing technology to expand the processing power of the system; S10. Provide a custom configuration interface to allow users to adjust data processing and submission processes according to specific needs.
2. A method for automatically submitting electronic file attachments according to claim 1, characterized in that: The S3 specifically includes: S31. Extract the initial features of the electronic file attachments through the preprocessed data: F={f1,f2,...,f n }; Among them, f i represents a single feature of each attachment, and n is the total number of features; S32. Use an intelligent recognition algorithm based on machine learning to analyze the initial feature set F and generate a feature vector for each attachment: V i =[v i1 ,v i2 ,...,v im ]; Among them, V i represents the feature vector of the i-th attachment, and m is the dimension of the feature vector; S33, for the eigenvector V i Perform standardization and calculate the standardized eigenvector: V′ i =[v′ i1 ,v′ i2 ,…,v′ im ]; The normalization formula is: Among them, v′ ij is the standardized eigenvalue, μ j is the average value of feature j, σ j is the three-dimensional standard deviation of feature j; S34, using the support vector machine algorithm to normalize the feature vector V′ i Classify and generate classification label L i , L i represents the classification result of the i-th attachment; S35, the classification result L i Verify and check, and calculate the classification accuracy A, the formula is: Where N is the total number of attachments, is the true label of the i-th attachment, δ(x,y) is an exponential function, when x=y, δ(x,y)=1, otherwise δ(x,y)=0; S36, according to the classification result L i , mark the electronic file attachment and generate the file attachment D containing the marked information i =(A i ,L i ), where D i is the i-th attachment, A i is the content of the ith attachment, L i is the classification label of the i-th attachment; S37. Use the graph neural network algorithm to further optimize the classified and annotated electronic file attachments, analyze the association relationship between the attachments, and generate optimized annotation information L′ i ; S38, including optimization information L' i The archive attachments are stored in the database to form a collection of classified and annotated electronic archive attachments: {D′1,D′2,...,D′ N }。 3. The method for automatically submitting electronic file attachments according to claim 1, characterized in that: The S4 specifically includes: S41, receiving the classified and annotated electronic archive attachment set {D′1, D′2, ..., D′ N }, identify the format of the data file according to the annotation results; S42, based on the identified file format and classification and annotation results, calling the corresponding format conversion module to convert the attachment data into a target format; S43. During the format conversion process, BERT natural language processing technology is used to analyze the text attachments, including content extraction, sentiment analysis, topic modeling, and tagging; S44, calling the image processing module to convert the format of the image attachment data and perform standardization processing on the image data; S45. After the format conversion is completed, the data verification module is called to verify the integrity and correctness of the converted data; S46, storing the attachment data after format conversion, content extraction, sentiment analysis, topic modeling and tagging in a temporary database, preparing for subsequent automatic uploading and archiving processing.
4. A method for automatically submitting electronic file attachments according to claim 3, characterized in that: The BERT natural language processing technology specifically includes: Use the BERT model to segment the text content of the attachment and convert the text content into word vector representation. Suppose the text content is T, the segmentation result is: T={t1,t2,…,t n }; Among them, t i represents the i-th word; The bidirectional encoder of the BERT model is used to understand the context of the word vector and generate the context representation of each word: C={c1,c2,…,c n }; Among them, c i Represents the context vector representation of the i-th word; Perform multi-head attention on the context representation and calculate the multi-head attention weight for each word in, It is the word t i The attention score at the h-th head is defined as: in, and b h is the trainable parameter of the h-th head, h i is the hidden state represented by the context vector; Calculate the multi-head weighted context representation u h And concatenate: u=concat(u 1 ,u 2 ,...,u H ); Where H is the number of heads, each u h The calculation formula is: Extract the key summary S from the context representation u and select representative word vectors through clustering or sorting algorithms: S={s1,s2,…,s m }; Where m is the length of the summary; Generate a summary text of the attachment content based on the extracted key summary, and add tags to the text using the preset annotation rules: L={l1,l2,…l p}; Among them, l i represents the i-th label; Using sentiment analysis and topic modeling techniques, the extracted text summary S is further analyzed to generate sentiment score E and topic distribution T d .
5. The method for automatically submitting electronic file attachments according to claim 1, characterized in that: The S5 specifically includes: S51. Automatically upload electronic file attachment data by using the API interface. First, define the endpoint URL and related parameters of the API, including the URL, HTTP method, request header, request body, and verification information; S52, configuring an automated script in the system, calling the API interface to upload the attachment data, the script is responsible for reading the file to be uploaded, converting it into a suitable format, and sending it to the specified server through the POST method; S53, during the upload process, the attachment data is verified using a hash algorithm, the hash value of the file is calculated, and the calculation result is compared with the hash value returned by the server; S54. Use blockchain technology to track and verify electronic file attachments, write the metadata of each uploaded attachment file into the blockchain, use smart contracts to automatically record each upload operation, and generate a unique transaction ID. Through the consensus mechanism of the blockchain network, ensure that each node can verify and record the authenticity and immutability of the upload operation; S55. In the blockchain, the Merkle tree structure is used to organize and manage nearby data.
6. The method for automatically submitting electronic file attachments according to claim 1, characterized in that: The S9 specifically includes: S91. Configure and deploy AWS EC2 instances, and select the appropriate instance type and quantity based on the load requirements of the electronic file attachment submission system; S92. Use AWS Auto Scaling service to automatically adjust the number of EC2 instances according to changes in system load to dynamically use peak periods and underestimate load changes; S93. Configure and use AWS S3 to store electronic archive attachment data, and use S3's version control function to manage different versions of attachment data; S94. Use AWS Lambda serverless computing services to handle data preprocessing, format conversion, intelligent identification and classification tasks, automatically execute related functions through trigger mechanisms, and reduce dependence on fixed server resources; S95. Configure AWS RDS to manage and store metadata and processing logs of electronic archive attachments, and use multi-availability zone deployment to improve database availability and fault recovery capabilities; S96. Use AWS CloudWatch for system monitoring and log management, monitor the operating status and performance indicators of each service in real time, and set alarm rules; S97. Configure AWS IAM to manage user and service access rights to ensure system security and data protection; define detailed permission policies; S98. Use AWS VPC to create an isolated network environment and configure subnets, routers, and network ACLs. S99, securely connect local data centers to AWS cloud environments through AWS Direct Connect; S910. Configure AWS Backup for regular data backup and use AWS Glacier to store long-term backup data.
Citation Information
Cited By
Electronic accounting file paperless processing and filing method based on BERT model
CN121434462A
Electronic accounting file paperless processing and archiving method based on BERT model
CN121434462B
