A general method of scanning for transmissions to an application

Abstract

This invention discloses a general method for scanning and sending data to applications, relating to the field of data processing technology. The method includes the following steps: scanning device sending emails, parsing and filtering emails, data processing, file processing and storage, file sending and encapsulation, process control, and log management. In this invention, the SMTP email receiving service module is used to parse the email structure, filter valid scanned data, and block spam; the data processing and parsing module is used to extract user identity information and scan metadata, completing identity matching and metadata standardization; the file processing module is used to manage temporary file storage, realize intelligent conversion of scanned file formats, and ensure compliance with application requirements; the file sending service module is used to encapsulate files and metadata, call application interfaces for transmission, and determine the sending status; the process control module is used to monitor the process, perform dynamic retries, and clean up temporary files; and the log management unit records the entire process to ensure traceability.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a general method for scanning and sending data to an application. Background Technology

[0002] Scanning and sending to the application refers to the process of converting physical documents into digital files using scanning equipment and then sending them to the target application system via a transmission link. Its functions are: first, to support paperless office work, reduce the cost of storing and managing paper documents, and reduce the difficulty of information retrieval; second, to improve the efficiency of cross-departmental and cross-system collaboration and avoid the risks of delays and loss due to manual transmission of physical documents; and third, to ensure the standardized management of digital files, providing a foundation for subsequent data statistics, access control, archiving and traceability.

[0003] In the existing technology, the entire process of sending scanned files to the application has shortcomings such as reliance on manual screening for receiving and parsing scanned data, fragmented management of user identity and metadata, lack of universality in file storage and format conversion, insufficient adaptation and reliability of file sending interfaces, and lack of unified monitoring and traceability throughout the process. These shortcomings result in low processing efficiency, difficulty in cross-system adaptation, and poor process maintainability.

[0004] Therefore, a general method for sending scans to applications is proposed to solve the above problems. Summary of the Invention

[0005] The main objective of this invention is to provide a general method for sending scans to applications, in order to solve the problems mentioned in the background above.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: a general method for scanning and sending data to an application, comprising the following steps:

[0007] S1: The scanning device attaches the scanned document to an email containing the body text and sends it to the system via the SMTP protocol;

[0008] S2: The SMTP email receiving service module parses emails, filters valid attachments according to the set of valid attachments, blocks spam emails, and extracts recipient addresses and body text.

[0009] S3: The data processing and parsing module extracts the email address from the recipient's address, obtains the username U through the local user mapping table or the unified identity authentication interface and verifies its validity, and at the same time parses and generates a set of metadata.

[0010] S4: The file processing module creates a temporary folder to store valid attachments, queries the target format of the application system, and performs format conversion and validity verification through the attachment format conversion function.

[0011] S5: The file sending service module encapsulates the file and metadata according to the data sending format, calls the application system interface to send the file, and determines the sending status.

[0012] S6: If the process control module detects that the sending status is failed, it will retry according to the formula of the kth retry interval; if it still fails after reaching the maximum number of retries, it will clean up the temporary file; if the sending is successful, it will clean up the temporary file after the log management unit completes the full process log recording and archives the logs.

[0013] S7: The entire process is recorded by the log management unit, forming a traceable log chain. Logs that exceed the log storage time are automatically archived.

[0014] Preferably, the SMTP email receiving service module is used to receive emails containing scanned documents sent by the scanning device, parse the email structure, and filter valid scanned data;

[0015] The data processing and parsing module is used to extract user identity information and scan relevant metadata from emails, and to complete user identity matching and metadata standardization.

[0016] The file processing module is used to manage temporary file storage and scan file format conversion to ensure that the files meet the requirements of the application system.

[0017] The file sending service module is used to encapsulate the processed file and metadata, and call the application system interface to complete the data transmission.

[0018] The process control module is used to monitor the execution status of the entire process, execute the retry mechanism, and record operation logs to ensure process traceability.

[0019] Preferably, the SMTP email receiving service module includes an email structure parsing unit, a valid data filtering unit, and a spam interception unit;

[0020] The email structure parsing unit is used to separate the email header, attachment data stream and body text, parse the encoding format and convert it into a unified encoding;

[0021] The effective data filtering unit is used to filter attachments and text that meet the scanning characteristics. The set of effective attachments is defined by the following formula:

[0022] ,

[0023] in, This is in attachment format. This is a set of preset scanning formats. For attachment size, Maximum capacity for a single attachment. The creation time of the attachment. This refers to the effective time threshold for emails.

[0024] The spam blocking unit is used to block invalid emails based on sender IP whitelists and subject keywords. The effectiveness of the blocking must meet the following requirements: for suspected spam emails identified by the rules, the blocking success rate must be [percentage missing]. , The number of spam emails blocked / the number of suspected spam emails. Suspected spam emails refer to emails that trigger the blocking rules.

[0025] Preferably, the data processing and parsing module includes a user identity extraction unit, which is used to extract a set of email addresses from the email recipient field. ,when Simultaneous generation of parallel processing tasks;

[0026] For a single Address, obtain the application system username through a priority strategy:

[0027] (1) First, match the local user mapping table. If a record exists, output it directly. ,in For local user mapping tables, Email address;

[0028] (2) When there is no local mapping, the unified identity authentication interface is called through the function. Obtain, among which For authentication interface functions, the return value must meet the following requirements. , Register a set of users for the application system;

[0029] (3) If authentication fails three times in a row, the user will be marked as invalid, triggering an alarm and terminating the file processing flow corresponding to the email address.

[0030] Preferably, the data processing and parsing module further includes a metadata parsing unit, which is used to extract key metadata from the email body and attachment attributes:

[0031] (1) Basic metadata: Scanning device identifier Scanning time ;

[0032] (2) Content metadata: Content tags are generated through keyword weighting algorithms, and the tag set is... ,in The weights are:

[0033] ,

[0034] Only keep Tags;

[0035] (3) Output metadata set:

[0036] , used for auxiliary information verification when sending files;

[0037] If an email is identified as spam, its metadata is also recorded in the spam retrieval log for post-event analysis and strategy optimization.

[0038] Preferably, the file processing module includes a temporary storage management unit, which manages temporary files according to the following rules:

[0039] (1) Folder naming:

[0040] ,

[0041] Hash() is an irreversible hash function to avoid storing sensitive information in plaintext, and T_stamp is the received timestamp.

[0042] (2) Storage monitoring: Real-time monitoring of the total capacity of the temporary directory, and when the usage... At that time, files that have not been processed for more than 3 days are cleaned up according to the principle of least recent use. Preset total capacity;

[0043] (3) Automatic cleanup: The temporary folder is deleted within 1 hour after processing. In special scenarios, the retention time can be manually extended through the interface;

[0044] Spam Removal: If an email is identified as spam, its corresponding temporary files will be marked and prioritized for removal immediately after interception, and the relevant storage records will be updated to the spam removal log.

[0045] Preferably, the file processing module further includes a format intelligent conversion unit, which is used to achieve format adaptation between the scanned file and the application system.

[0046] (1) Target format acquisition: Query the target format through the application system interface metadata interface;

[0047] (2) Conversion rules: When the attachment format is not equal to the target format, the conversion engine is called to perform the conversion. The conversion function is: , This is the format of the converted attachment. Let be the objective function. The quality level is indicated by the grade; the higher the grade, the clearer the converted file, but the larger the file size.

[0048] (3) Conversion verification: Compare the key features before and after conversion to determine the validity of the conversion. The validity V(a,a_conv)=1 if and only if the feature deviation rate is ≤10%, otherwise V=0 and the conversion is repeated; among them, the resolution is the key feature for image files and the number of pages is the key feature for document files.

[0049] Spam content blocking: If the content of an attachment is detected to be highly matched with a known spam signature database during the format conversion process, the conversion process will be immediately stopped, and the attachment and the entire email will be transferred to the spam recycling unit for processing.

[0050] Preferably, the file sending service module includes an interface adaptation unit, which is used to pre-configure interface parameters for multiple application systems, and the parameter set includes:

[0051] (1) Basic parameters: interface URL, request method, timeout T_timeout;

[0052] (2) Security parameters: authentication method, data encryption algorithm;

[0053] (3) Adaptation rules: Adjust the data encapsulation format according to the application system type, and support both form submission and JSON data modes.

[0054] Preferably, the file sending service module further includes a data encapsulation and sending unit, which is used for:

[0055] Data encapsulation: Encapsulate and send data according to the application system interface requirements. The general format is:

[0056] ,

[0057] Sign is a data signature used by the interface to verify data integrity.

[0058] Send execution: Send data to the interface in the specified format, and determine the status based on the returned result.

[0059] If the sending status is successful, the return code is ∈ {200, 201} and the response time is less than or equal to the timeout time;

[0060] If the sending status is failed, and the return code is ∈{4xx,5xx} or the response times out, the retry mechanism of the process control module is triggered.

[0061] Spam delivery blocking: If the email-related task is found to be marked as spam during the preparation phase or log backtesting, the delivery task will be immediately canceled, the spam recycling process will be invoked to handle the relevant data, and the blocking event will be recorded in the log management unit.

[0062] Preferably, the process control module includes a retry control unit and a log management unit;

[0063] The retry control unit adopts a dynamic backoff strategy: a preset maximum number of retries N_max, and a retry interval for the kth retry. ,in Based on the interval, Use a random value between 0 and 0.1 to avoid retry storms; if the task still fails after k=maximum number of retries, mark it as pending manual processing and push an alert to the administrator;

[0064] The log management unit records the entire process log in the following format:

[0065] ,

[0066] Log storage duration T_log=90 days, supports multi-dimensional query by email address, username U, status, and critical operation logs need to be backed up to offline storage simultaneously;

[0067] The log management unit establishes an independent category for spam processing logs, recording the entire process from interception and recycling to final disposal. It supports specific queries and report generation based on sender, interception reason, and time range, providing data support for optimizing spam defense strategies.

[0068] The present invention has the following beneficial effects:

[0069] 1. In this invention, the entire process of scanning files and sending them to the application is automated. The SMTP email receiving service module automatically parses the email structure, filters valid scanned data, and blocks spam. The data processing and parsing module automatically extracts user identity information and standardizes scan metadata. The file processing module automatically manages temporary file storage and performs intelligent format conversion. The file sending service module automatically encapsulates data and calls the application interface to send it. The process control module automatically performs retries and cleans up temporary files. Each module is interconnected, and no manual intervention is required throughout the entire process, completely solving the inefficiency problems of manual screening, manual conversion, and system-by-system adaptation in traditional processes.

[0070] 2. In this invention, the intelligent format conversion unit of the file processing module can adapt to the format requirements of different application systems; the interface adaptation unit of the file sending service module is pre-configured with interface parameters of multiple application systems, supporting multiple data encapsulation modes such as forms and JSON; the user identity matching and metadata standardization of the data processing and parsing module ensure cross-system identity recognition and data compatibility, and the collaboration of each module enables the system to quickly connect to various applications such as OA, ERP, and document management, without the need to develop customized logic for each system, significantly reducing integration time and cost.

[0071] 3. In this invention, the reliability of the process and the traceability of the entire chain are guaranteed. The dynamic backoff and retry mechanism of the process control module ensures the high reliability of file sending and can automatically recover even if temporary network or interface anomalies occur. The log management unit works in conjunction with each module to record the entire process operation from email reception and data processing to file sending. It supports multi-dimensional query and offline backup, making problem troubleshooting efficient and convenient. At the same time, the automatic cleanup mechanism of temporary storage management makes reasonable use of storage resources, further improving the stability and maintainability of the system. Attached Figure Description

[0072] Figure 1 This is a flowchart of a general method for sending scans to an application according to the present invention;

[0073] Figure 2 This is a system framework diagram of a general method for sending scans to an application according to the present invention. Detailed Implementation

[0074] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0075] Please see Figure 1-2 This invention provides a technical solution: a general method for sending scanned data to an application, comprising the following steps:

[0076] S1: The scanning device attaches the scanned document to an email containing the body text and sends it to the system via the SMTP protocol;

[0077] S2: The SMTP email receiving service module parses emails, filters valid attachments according to the set of valid attachments, blocks spam emails, and extracts recipient addresses and body text.

[0078] S3: The data processing and parsing module extracts the email address from the recipient's address, obtains the username U through the local user mapping table or the unified identity authentication interface and verifies its validity, and at the same time parses and generates a set of metadata.

[0079] S4: The file processing module creates a temporary folder to store valid attachments, queries the target format of the application system, and performs format conversion and validity verification through the attachment format conversion function.

[0080] S5: The file sending service module encapsulates the file and metadata according to the data sending format, calls the application system interface to send the file, and determines the sending status.

[0081] S6: If the process control module detects that the sending status is failed, it will retry according to the formula of the kth retry interval; if it still fails after reaching the maximum number of retries, it will clean up the temporary file; if the sending is successful, it will clean up the temporary file after the log management unit completes the full process log recording and archives the logs.

[0082] S7: The entire process is recorded by the log management unit, forming a traceable log chain. Logs that exceed the log storage time are automatically archived.

[0083] The SMTP email receiving service module is used to receive emails containing scanned documents sent by scanning devices, parse the email structure, and filter valid scanned data.

[0084] The data processing and parsing module is used to extract user identity information and scan relevant metadata from emails, and to complete user identity matching and metadata standardization.

[0085] The file processing module is used to manage temporary file storage and scan and convert file formats to ensure that files meet the requirements of the application system.

[0086] The file sending service module is used to encapsulate the processed file and metadata, and call the application system interface to complete the data transmission.

[0087] The process control module is used to monitor the execution status of the entire process, implement the retry mechanism, and record operation logs to ensure process traceability.

[0088] The SMTP email receiving service module includes an email structure parsing unit, a valid data filtering unit, and a spam interception unit, enabling accurate email parsing and effective data filtering.

[0089] 1. Email Structure Parsing Unit

[0090] Used to separate email headers, attachment data streams, and body text; supports parsing and unified conversion of encoding formats such as Base64 and Quoted-Printable.

[0091] (1) Use the Python email library to parse emails: extract the recipient address using msg.get_all('To'), and use msg.walk() to traverse the email and separate the attachment data stream from the body text;

[0092] (2) Encoding conversion: Based on the Content-Transfer-Encoding attribute of the email, use the base64 or quopri library to decode the body text into UTF-8 plain text.

[0093] 2. Effective Data Filtering Unit

[0094] This is used to filter attachments and text that match the scanning characteristics. The set of valid attachments is defined by the following formula, where, This is in attachment format. This is a set of preset scanning formats. For attachment size, Maximum capacity for a single attachment. The creation time of the attachment. Email validity period threshold:

[0095]

[0096] (1) The format can be determined by the file extension or MIME type;

[0097] (2) The size of the attachment is checked using the `len(payload)` parameter;

[0098] (3) The timestamp is set to the 24 hours prior to email receipt. The attachment creation time is extracted from the X-Attachment-Creation-Time header and its timeliness is verified.

[0099] 3. Spam Filtering Unit

[0100] Spam blocking is based on sender IP whitelists and subject keywords; the blocking rate must meet certain requirements. ,

[0101] (1) Maintain the IP whitelist database and verify the sender's IP using socket.gethostbyname();

[0102] (2) Check if the email subject contains keywords such as scan or scan. If it does not meet the requirements, it is judged as spam and discarded.

[0103] The data processing and parsing module includes a user identity extraction unit;

[0104] 1. Address set extraction and parallel task triggering

[0105] The data processing and parsing module includes a user identity extraction unit, which is used to extract a set of email addresses from the email recipient field. ;

[0106] when Simultaneous generation of parallel processing tasks:

[0107] The Python `concurrent.futures.ThreadPoolExecutor` framework is used to allocate a separate thread for each email address, simultaneously executing the identity matching logic. To avoid resource overload, the maximum number of threads in the thread pool is limited to `min(n, 10)`, i.e. .

[0108] 2. Single Address priority identity matching strategy

[0109] For a single Addresses are matched in the following order of priority:

[0110] (1) Local user mapping table matching

[0111] Local user mappings are stored in the MySQL database table `user_mapping(email VARCHAR(255), username VARCHAR(100))`, and a Redis caching layer is introduced to optimize query performance. The function is called... ,in For local user mapping tables, For email addresses, the internal logic is as follows:

[0112] Prioritize querying the Redis cache (cache expiration time CacheTTL = 3600 seconds);

[0113] Execute an SQL query when the cache is not hit;

[0114] If the query returns results, simply output the username U.

[0115] (2) Unified identity authentication interface call

[0116] If the local mapping fails, the unified identity authentication interface function will be called. This interface is based on the OAuth 2.0 protocol and sends a POST request to the authentication server, carrying... The username U returned by the interface must meet the following requirements: ,in To register a set of users for the application system and to prevent interface errors from affecting the overall process, a circuit breaker mechanism is introduced: if the interface times out 5 times consecutively, authentication will be directly determined to have failed within the next 30 seconds of CircuitBreakTime.

[0117] (3) Authentication failure handling mechanism

[0118] Maintain authentication failure counter Each time the unified identity authentication interface is called Failure ,when hour:

[0119] mark "Invalid user";

[0120] Trigger the WeChat Work alarm API and send a message containing Alarms regarding the number of failures and their causes;

[0121] Terminate The corresponding file processing flow releases system resources.

[0122] The data processing and parsing module also includes a metadata parsing unit, which is used to extract key metadata from the attributes of the email body and attachments to form a standardized Meta set, providing data support for file sending, system statistics and access control;

[0123] 1. Extraction of basic metadata

[0124] (1) Scanning equipment identification Extraction: Device identifiers are extracted using a multi-source priority strategy.

[0125] Priority 1: Read directly from the X-Scanner-ID field in the email header;

[0126] Priority 2: Extract from attachment filenames using the regular expression r'ScanDev(\w+?)_';

[0127] Priority 3: If none of the above results are found, mark it as an unknown device and log it for subsequent device identifier mapping completion.

[0128] (2) Scanning time Extraction: Device identifiers are extracted using a multi-source priority strategy.

[0129] Priority 1: Extract from the attachment creation time field X-Attachment-Creation-Time;

[0130] Priority 2: Match the email body using the regular expression r's scan time: (\d{4}-\d{2}-\d{2}\d{2}:\d{2}:\d{2})';

[0131] Priority 3: If no results are found, use the email receipt timestamp as the default value to ensure the integrity of time information.

[0132] 2. Extraction of content metadata

[0133] (1) Word segmentation and stop word filtering: The email body is segmented using the jieba word segmentation library, and a stop word list is loaded to filter out meaningless words, thus obtaining a set of effective words. ;

[0134] (2) Term Frequency (TF) Calculation: For each valid word The formula for calculating word frequency is:

[0135] ,in It is a word Number of times it appears in the main text This is the total word count of the main text.

[0136] (3) Inverse Document Frequency (IDF) Calculation: Inverse document frequency reflects the scarcity of words, and the calculation formula is: Where N is the total number of emails currently being processed. It is a word containing The number of emails.

[0137] (4) Keyword weight calculation and selection: The final weight of each keyword is:

[0138] ,

[0139] Only keep The words form a set of content tags. .

[0140] 3. Meta-information Set Output: Integrates basic metadata and content metadata to output a metadata set:

[0141] , used for auxiliary information verification when sending files;

[0142] If an email is identified as spam, all metadata generated by this unit will no longer be passed to subsequent file processing modules. Instead, it will be synchronously encapsulated and recorded in the spam collection log, and stored in the isolation area along with the original data of the email. This metadata will be used for subsequent in-depth analysis, spam feature mining, and optimization and iteration of interception strategies.

[0143] The file processing module includes a temporary storage management unit, which manages temporary files according to the following rules:

[0144] 1. Temporary folder naming rules: Temporary folder names are generated using the following formula:

[0145] ,

[0146] Hash() is an irreversible hash function that uses the SHA-256 algorithm. It hashes Email_addr and takes the first 16 characters to avoid storing sensitive information in plaintext. T_stamp is the received timestamp.

[0147] 2. Storage monitoring rules: Monitor the total capacity of the temporary directory in real time, and trigger cleanup when the usage meets the following conditions: ,in , This refers to all folders under the temporary directory. To allocate space for the folder; Set the preset total temporary storage capacity.

[0148] Cleanup strategy: Clean up folders that have not been processed within the last 3 days according to the Least Recently Used (LRU) principle. Specific implementation:

[0149] (1) Traverse all folders in the temporary directory and get the last access time (LastAccessTime) of each folder;

[0150] (2) Screening out A folder containing (current timestamp minus 3 days and seconds);

[0151] (3) Sort by access time from earliest to latest, and delete sequentially until... (Reserve 10% buffer space).

[0152] 3. Automatic Cleanup Rules

[0153] (1) Routine cleanup: After the process is completed (termination upon success or failure), the cleanup time is calculated using the following formula: ,in, This is the timestamp indicating when file processing was complete; 3600 seconds is the default retention period. At that time, the system command is invoked to delete the temporary folder.

[0154] (2) Extended retention for special scenarios: In special scenarios (such as log auditing and fault diagnosis), the retention time T_extend can be manually extended through the RESTful interface, with a maximum support of 72 hours. The formula is as follows:

[0155] ;

[0156] Instant Spam Removal: If the system determines that an email is spam, it will immediately trigger a task to clean up any temporary files that may have been generated by that email. The system will locate and delete the corresponding temporary folder based on the email's unique identifier or the recipient's email hash value, without waiting for the default 1-hour retention period. This operation log will record in detail the reason for the cleanup (associated with spam) and the amount of data cleaned up, and will be linked to the main spam recycle record for that email.

[0157] The file processing module also includes an intelligent format conversion unit, which is used to adapt the format of scanned files to the application system. Through target format acquisition, intelligent conversion and validity verification, it ensures that the file format meets the requirements of the application system.

[0158] 1. Target format acquisition: Query the target format through the application system metadata interface. The specific implementation is as follows:

[0159] (1) API call: Using RESTful API, the request URL is You need to bring the system authentication token;

[0160] (2) Response parsing: The interface returns JSON format data, for example... ,in That is ;

[0161] (3) Caching strategy: Cache to Redis, cache expiration time To avoid frequent API calls.

[0162] 2. Conversion Rules: When the attachment format is different from the target format, the conversion engine is invoked to perform the conversion. The conversion function is as follows: ;

[0163] in:

[0164] (1) This is the format of the converted attachment. Let be the objective function. The quality level is indicated by the grade; a higher grade results in a clearer file after conversion but also a larger file size. For example: Image resolution 300dpi, high-quality compression in PDF format;

[0165] Image resolution 600dpi, PDF uncompressed.

[0166] (2) Conversion engine selection: The optimal engine is automatically matched based on the file type and target function;

[0167] (3) Queue priority: The priority of conversion tasks is calculated based on urgency + file size, using the following formula: U is the emergency symbol. (Default weight), S is the file size, and the higher the priority, the earlier it will be executed.

[0168] 3. Conversion Validity Verification: Compare key features before and after conversion to determine conversion validity. Validity V(a,a_conv)=1 if and only if the feature deviation rate is ≤10%. Otherwise, V=0 and re-convert, up to 2 times. If it still fails, mark it as a conversion anomaly and log it. Among them, resolution is the key feature for image files and page number is the key feature for document files.

[0169] Content security blocking: During the conversion process, the system calls the content security detection service to scan the attachment content (such as image text extracted by OCR, document text) in real time and match it with the known spam feature database (including spam text patterns, malicious link features, sensitive keywords, etc.);

[0170] Judgment and handling: If the matching degree exceeds the preset risk threshold, the current and all subsequent conversion tasks will be immediately stopped, a security event alarm will be generated, and the entire processing task of the attachment and its associated email will be transferred to the spam recycling unit to perform the same isolation, logging and cleaning process as the intercepted email.

[0171] The file sending service module includes an interface adaptation unit, which is used to pre-configure interface parameters for multiple application systems, ensuring that the file sending service module can adapt to the interface requirements of different systems and achieve efficient and secure data transmission.

[0172] 1. Basic parameters

[0173] (1) Pre-configure the interface URL, request method and initial timeout T_timeout. The parameters are stored in the distributed configuration center and support multi-environment isolation.

[0174] (2) Calculate the timeout period dynamically based on the interface's historical response time and network latency, using the following formula: ,

[0175] in: The historical average response time of the interface (based on log statistics of the average of the last 100 requests).

[0176] This is the network latency coefficient. (The baseline latency is the average latency when the system is idle.)

[0177] A coefficient of 1.5 is reserved for buffering to ensure that requests are not interrupted prematurely;

[0178] Upper and lower limit constraints ensure that the timeout period is within a reasonable range.

[0179] 2. Safety parameters

[0180] (1) Pre-configure authentication methods and data encryption algorithms; authentication credentials and encryption keys are stored in the key management service.

[0181] (2) Dynamically select the encryption algorithm based on data sensitivity. Defined as: ,

[0182] when At this time, RSA encryption is used (high security level, suitable for scanned documents containing core secrets);

[0183] when When using AES-256 encryption (balancing security and performance);

[0184] when When using Base64 encoding (lightweight, suitable for non-sensitive files), use Base64 encoding.

[0185] Authentication token automatic renewal mechanism: The remaining validity period of the token is set to... ,when When the time comes, the token refresh interface will be automatically invoked.

[0186] 3. Adaptation rules: Adjust the data encapsulation format according to the application system type, support both form submission and JSON data modes, and store the format mapping relationship in the rule engine.

[0187] The file sending service module also includes a data encapsulation and sending unit, which is responsible for encapsulating the processed files and metadata according to the application system interface requirements, performing the sending operation, and determining the sending status based on the returned result to trigger subsequent processes.

[0188] 1. Data Encapsulation: Encapsulate and send data according to the application system interface requirements. The general format is:

[0189] ,

[0190] (1) Output from the data processing and parsing module must meet the following requirements. , Register a set of users for the application system;

[0191] (2) :: A list of files after format conversion. If the file size exceeds the interface limit, file splitting will be performed.

[0192] Slice size: ,

[0193] Number of fragments: ( (to round up)

[0194] (3) The metadata set output by the metadata parsing unit is converted into a JSON string format.

[0195] (4) Data signature, used by the interface to verify data integrity, is calculated using the following formula:

[0196] ,

[0197] in This is the current timestamp. The key pre-configured for the system. This is a hash algorithm.

[0198] 2. Send Execution

[0199] (1) Send to the application system interface using an HTTP client Set a timeout period T_timeout to record the request start time. and end time Response time .

[0200] (2) Sending status determination: Determine based on the returned result. ,

[0201] ,

[0202] like This triggers the retry mechanism of the process control module;

[0203] like Record successful transmission logs, including file ID, recipient system, and other information;

[0204] Spam blocking:

[0205] Pre-send blocking check: Before executing the HTTP send request, the system will double-check the global status identifier of the email associated with the task. If the identifier shows that the email has been marked as spam or malicious content by the spam blocking unit or the intelligent format conversion unit, the current and all retry send tasks will be canceled immediately.

[0206] Post-blocking processing: After the sending is blocked, the system calls a unified spam recycling process to ensure that all encapsulated data and temporary files related to the task are properly disposed of. At the same time, a log is recorded to the log management unit, with details including the reason for the block, the original task ID and the associated spam interception ID, forming a complete security closed loop.

[0207] The process control module includes a retry control unit and a log management unit;

[0208] 1. The test control unit adopts a dynamic backoff strategy: a preset maximum number of retries N_max, and a retry interval for the kth retry: ;

[0209] (1) : Basic retry interval (default 3 minutes, can be dynamically adjusted through system configuration);

[0210] (2) : A random number in the range [0, 0.1], used to distribute retry time and avoid retry storms;

[0211] (3) : Current number of retries ( (Default 5 times)

[0212] (4) : Indexed retreat factor, among which The maximum backoff multiple (default 8) ensures that the retry interval does not increase excessively, avoiding task timeouts or application system interface timeouts due to excessively long intervals, and ensuring the reliability of the retry mechanism.

[0213] when Retry is triggered at certain times, and after each retry... Increment by 1; if If the task still fails, mark it as pending manual processing and send an alert to the administrator via WeChat, email, or other channels.

[0214] 2. The log management unit records the entire process log in the following format:

[0215] ,

[0216] in For event timestamps, For process stages, For execution status, For more details...

[0217] Storage duration: Log storage duration Elasticsearch is used for storage, and critical operation logs are synchronously backed up to object storage. To ensure long-term data traceability;

[0218] Multi-dimensional query: Supports queries by combined dimensions such as Email_addr, Username (U), Status, and Stage, with fast query response time. 500ms;

[0219] The log management unit establishes an independent category for spam processing logs, recording the entire process from interception and recycling to final disposal. It supports specific queries and report generation based on sender, interception reason, and time range, providing data support for optimizing spam defense strategies.

[0220] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus.

[0221] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A general method for sending scans to an application, characterized in that, Includes the following steps: S1: The scanning device attaches the scanned document to an email containing the body text and sends it to the system via the SMTP protocol; S2: The SMTP email receiving service module parses emails, filters valid attachments according to the set of valid attachments, blocks spam emails, and extracts recipient addresses and body text. S3: The data processing and parsing module extracts the email address from the recipient's address, obtains the username U through the local user mapping table or the unified identity authentication interface and verifies its validity, and at the same time parses and generates a set of metadata. S4: The file processing module creates a temporary folder to store valid attachments, queries the target format of the application system, and performs format conversion and validity verification through the attachment format conversion function. S5: The file sending service module encapsulates the file and metadata according to the data sending format, calls the application system interface to send the file, and determines the sending status. S6: If the process control module detects that the sending status is failed, it will retry according to the formula of the kth retry interval; if it still fails after reaching the maximum number of retries, it will clean up the temporary file; if the sending is successful, it will clean up the temporary file after the log management unit completes the full process log recording and archives the logs. S7: The entire process is recorded by the log management unit, forming a traceable log chain. Logs that exceed the log storage time are automatically archived.

2. The general method for sending scanned data to an application according to claim 1, characterized in that: The SMTP email receiving service module is used to receive emails containing scanned documents sent by the scanning device, parse the email structure, and filter valid scanned data. The data processing and parsing module is used to extract user identity information and scan relevant metadata from emails, and to complete user identity matching and metadata standardization. The file processing module is used to manage temporary file storage and scan file format conversion to ensure that the files meet the requirements of the application system. The file sending service module is used to encapsulate the processed file and metadata, and call the application system interface to complete the data transmission. The process control module is used to monitor the execution status of the entire process, execute the retry mechanism, and record operation logs to ensure process traceability.

3. The general method for sending scanned data to an application according to claim 2, characterized in that: The SMTP email receiving service module includes an email structure parsing unit, a valid data filtering unit, and a spam interception unit. The email structure parsing unit is used to separate the email header, attachment data stream and body text, parse the encoding format and convert it into a unified encoding; The effective data filtering unit is used to filter attachments and text that meet the scanning characteristics. The set of effective attachments is defined by the following formula: ， in, This is in attachment format. This is a set of preset scanning formats. For attachment size, Maximum capacity for a single attachment. The creation time of the attachment. This refers to the effective time threshold for emails. The spam blocking unit is used to block invalid emails based on sender IP whitelists and subject keywords. The effectiveness of the blocking must meet the following requirements: for suspected spam emails identified by the rules, the blocking success rate must be [percentage missing]. , The number of spam emails blocked / the number of suspected spam emails. Suspected spam emails refer to emails that trigger the blocking rules.

4. The general method for sending scanned data to an application according to claim 2, characterized in that: The data processing and parsing module includes a user identity extraction unit, which is used to extract a set of email addresses from the email recipient field. ,when Simultaneous generation of parallel processing tasks; For a single Address, obtain the application system username through a priority strategy: (1) First, match the local user mapping table. If a record exists, output it directly. ,in For local user mapping tables, Email address; (2) When there is no local mapping, the unified identity authentication interface is called through the function. Obtain, among which For authentication interface functions, the return value must meet the following requirements. , Register a set of users for the application system; (3) If authentication fails three times in a row, the user will be marked as invalid, triggering an alarm and terminating the file processing flow corresponding to the email address.

5. A general method for sending scanned data to an application according to claim 2, characterized in that: The data processing and parsing module further includes a metadata parsing unit, which is used to extract key metadata from the email body and attachment attributes: (1) Basic metadata: Scanning device identifier Scanning time ; (2) Content metadata: Content tags are generated through keyword weighting algorithms, and the tag set is... ,in The weights are: ， Only keep Tags; (3) Output metadata set: , used for auxiliary information verification when sending files; If an email is identified as spam, its metadata is also recorded in the spam retrieval log for post-event analysis and strategy optimization.

6. A general method for sending scanned data to an application according to claim 2, characterized in that: The file processing module includes a temporary storage management unit, which manages temporary files according to the following rules: (1) Folder naming: ， Hash() is an irreversible hash function to avoid storing sensitive information in plaintext, and T_stamp is the received timestamp. (2) Storage monitoring: Real-time monitoring of the total capacity of the temporary directory, and when the usage... At that time, files that have not been processed for more than 3 days are cleaned up according to the principle of least recent use. Preset total capacity; (3) Automatic cleanup: The temporary folder is deleted within 1 hour after processing. In special scenarios, the retention time can be manually extended through the interface; Spam Removal: If an email is identified as spam, its corresponding temporary files will be marked and prioritized for removal immediately after interception, and the relevant storage records will be updated to the spam removal log.

7. A general method for sending scanned data to an application according to claim 2, characterized in that: The file processing module also includes a format intelligent conversion unit, which is used to achieve format adaptation between the scanned file and the application system. (1) Target format acquisition: Query the target format through the application system interface metadata interface; (2) Conversion rules: When the attachment format is not equal to the target format, the conversion engine is called to perform the conversion. The conversion function is: , This is the format of the converted attachment. Let be the objective function. The quality level is indicated by the grade; the higher the grade, the clearer the converted file, but the larger the file size. (3) Conversion verification: Compare the key features before and after conversion to determine the validity of the conversion. The validity V(a,a_conv)=1 if and only if the feature deviation rate is ≤10%, otherwise V=0 and the conversion is repeated; among them, the resolution is the key feature for image files and the number of pages is the key feature for document files. Spam content blocking: If the content of an attachment is detected to be highly matched with a known spam signature database during the format conversion process, the conversion process will be immediately stopped, and the attachment and the entire email will be transferred to the spam recycling unit for processing.

8. A general method for sending scanned data to an application according to claim 2, characterized in that: The file sending service module includes an interface adaptation unit, which is used to pre-configure interface parameters for multiple application systems. The parameter set includes: (1) Basic parameters: interface URL, request method, timeout T_timeout; (2) Security parameters: authentication method, data encryption algorithm; (3) Adaptation rules: Adjust the data encapsulation format according to the application system type, and support both form submission and JSON data modes.

9. A general method for sending scanned data to an application according to claim 2, characterized in that: The file sending service module further includes a data encapsulation and sending unit, which is used for: Data encapsulation: Encapsulate and send data according to the application system interface requirements. The general format is: ， Sign is a data signature used by the interface to verify data integrity. Send execution: Send data to the interface in the specified format, and determine the status based on the returned result. If the sending status is successful, the return code is ∈ {200, 201} and the response time is less than or equal to the timeout time; If the sending status is failed, and the return code is ∈{4xx,5xx} or the response times out, the retry mechanism of the process control module is triggered. Spam delivery blocking: If the email-related task is found to be marked as spam during the preparation phase or log backtesting, the delivery task will be immediately canceled, the spam recycling process will be invoked to handle the relevant data, and the blocking event will be recorded in the log management unit.

10. A general method for sending scanned data to an application according to claim 2, characterized in that: The process control module includes a retry control unit and a log management unit; The retry control unit adopts a dynamic backoff strategy: a preset maximum number of retries N_max, and a retry interval for the kth retry. ,in Based on the interval, Use a random value between 0 and 0.1 to avoid retry storms; if the task still fails after k=maximum number of retries, mark it as pending manual processing and push an alert to the administrator; The log management unit records the entire process log in the following format: ， Log storage duration T_log=90 days, supports multi-dimensional query by email address, username U, status, and critical operation logs need to be backed up to offline storage simultaneously; The log management unit establishes an independent category for spam processing logs, recording the entire process from interception and recycling to final disposal. It supports specific queries and report generation based on sender, interception reason, and time range, providing data support for optimizing spam defense strategies.

Images