Mail monitoring and structuring method and system based on automation
Patent Information
- Application Number
- CN202411142004.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-20
- Publication Date
- 2024-11-26
AI Technical Summary
[0002]随着经济的蓬勃发展,对外贸易也日新月异,促使海关各进出口的口岸对船舶运输货物检查也越发严格,货运船每过一次检查关口都需要花费大量的时间来等待检查完成;对于没有使用期限的货物来说并无太大影响,但是对于食品,特别是水果蔬菜肉质这类存在保质期的货物来说,等待检查时间越长,经济损失就越大
[0015] 1: In terms of efficiency, filling in material information into the business system is faster than manual filling. With the standardization of submitted materials and the enrichment of the template library, the filling efficiency will gradually increase.
Smart Images

Figure CN119027079A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of mail processing, and in particular to an automated mail monitoring and structuring method and system. Background Art
[0002] With the booming economy, foreign trade is also changing with each passing day, prompting the customs import and export ports to conduct increasingly strict inspections on cargo transported by ships. Each time a freight ship passes an inspection checkpoint, it takes a lot of time to wait for the inspection to be completed; for goods with no expiration date, there is not much impact, but for food, especially fruits, vegetables and meat, which have a shelf life, the longer the waiting time for inspection, the greater the economic loss.
[0003] The reason for this problem is that manual review is slow, there is a lag in the submission of ship materials, and the process of customs staff reviewing the materials is delayed. Common file reception is divided into FTP service, website form upload and cloud storage, etc., which are diverse and difficult to operate and build, making the cycle longer. Summary of the invention
[0004] In view of the above-mentioned deficiencies in the prior art, the technical problem to be solved by this patent application is how to provide an automated email monitoring and structuring method and system with high efficiency, strong scalability and high economic benefits.
[0005] In order to solve the above technical problems, the present invention adopts the following technical solutions:
[0006] An automated email monitoring and structuring method comprises the following steps: S1: retrieving emails from an inbox; S2: determining whether there are unread emails, if so, traversing the unread email list, if not, entering a waiting cycle, and re-entering step S1 after the waiting cycle ends; S3: determining whether there are attachments after traversing the unread email list, if so, saving the email information, if not, entering a waiting cycle, and re-entering step S1 after the waiting cycle ends; S4: searching an attachment template library to determine whether there is a corresponding template, if not, entering step S5; if yes, entering step S6: S5: notifying an administrator to create a corresponding template and entering step S6; S6: generating structured data of attachment materials based on the template, and requesting a business interface to implement related business.
[0007] As an optimization, the POP3 / IMAP service is enabled for the mailbox being retrieved, the authorization code of the mailbox is obtained, the mailbox is logged in through Python code, and the emails are operated.
[0008] As an optimization, a running environment is required for email monitoring and structuring. Anaconda3 is installed to run Python code. The third-party dependency libraries imaplib, selenium, pymysql, and email used to operate mailbox emails are installed through the pip command. The MySQL database is installed to realize data storage.
[0009] As an optimization, after retrieving the unread emails with attachments in step S3, it is necessary to extract the email information, which includes the email title, subject, content, attachments, sender, and sending time, and save the information to the database. By extracting the key fields of the email information, the category of the email is determined.
[0010] As an optimization, the attachment template in steps S4 and S5 is created based on the attachment content and business needs, and is completed by the data that can be obtained from the attachment and the data to be used to achieve related business needs through the attachment.
[0011] As an optimization, in step S6, the specific operation is to parse and extract the values of the corresponding fields in the material from the fields in the JSON string in the attachment comparison template, construct a complete JSON string, and use this JSON string as a parameter to request the business interface to implement related business.
[0012] An automated mail monitoring and structuring system, characterized in that it comprises a mail monitoring module and a mail structuring module; the mail monitoring module is used to periodically retrieve unread mails from a target monitoring mailbox, traverse the unread mails, check attachments, extract attachments, save them to an attachment library, record mail information, and change the status of unread mails to read; the mail structuring module is used to classify attachments, create and save attachment structured templates, and perform structured operations on attachment content.
[0013] As an optimization, the email monitoring module includes a login submodule, a retrieval submodule, a traversal submodule and an extraction submodule; the email structuring module includes an email classification submodule, an email structuring template submodule, an email content structuring submodule, an email library and a template library.
[0014] Beneficial effects:
[0015] 1: In terms of efficiency, filling material information into the business system is faster than manually filling it in. With the standardization of submitted materials and the enrichment of template libraries, this filling efficiency will gradually improve.
[0016] 2: The structure is not complicated, with only two parts: email monitoring and email structuring. It is also relatively simple to implement, requiring only simple Python language and database-related knowledge. It also becomes particularly simple to expand the functions of this system. Through preliminary communication, important information that is difficult to parse is added to the email content, thereby reducing the difficulty of parsing attachments and improving business efficiency.
[0017] 3: In terms of cost, it is economical and affordable. The mailbox service for receiving materials is provided free of charge by large manufacturers and no additional purchase is required. The recognition service involved can be achieved by building a training model and integrating it into the system. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 The present invention is a flowchart of an automated email monitoring and structuring method.
[0019] Figure 2 This is a structural framework diagram of an automated mail monitoring and structuring system described in the present invention. DETAILED DESCRIPTION
[0020] The present invention is further described in detail below in conjunction with the accompanying drawings. In the description of the present invention, it should be understood that the directions or positional relationships indicated by directional words such as "upper, lower" and "top, bottom" are usually based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description. Unless otherwise stated, these directional words do not indicate or imply that the devices or components referred to must have a specific direction or be constructed and operated in a specific direction, and therefore cannot be understood as limiting the scope of protection of the present invention; the directional words "inside and outside" refer to the inside and outside relative to the contours of each component itself.
[0021] Example 1
[0022] like Figure 1 As shown, an automated email monitoring and structuring method includes the following steps: S1: retrieving inbox emails; S2: determining whether there are unread emails, if so, traversing the unread email list, if not, entering a waiting period, and re-entering step S1 after the waiting period ends; S3: after traversing the unread email list, determining whether there are attachments, if so, saving the email information, if not, entering a waiting period, and re-entering step S1 after the waiting period ends; S4: searching the attachment template library to see if there is a corresponding template, if not, entering step S5; if so, entering step S6: S5: notifying the administrator to create a corresponding template and entering step S6; S6: generating structured data of the attachment material based on the template, and requesting the business interface to implement related business.
[0023] In this embodiment, in step S1, the retrieved mailbox starts the POP3 / IMAP service, obtains the authorization code of the mailbox, logs in to the mailbox through the python code, and operates the email.
[0024] In this embodiment, an operating environment is required for email monitoring and structuring. Anaconda3 is installed to run Python code. The third-party dependency libraries imaplib, selenium, pymysql, and email used to operate mailbox emails are installed through the pip command. The MySQL database is installed to realize data storage.
[0025] Specifically, when logging in and reading emails through the imaplib library, imaplib is mainly used to connect to the mailbox server, and is a Python third-party library that can set, delete, and read email status; you can directly connect to the mailbox service through the email_server=imaplib.IMAP4_SSL() method, the parameter is the address of the mailbox service, and log in through the email_server.login() method, the parameter is the mailbox account and the applied authorization code; after a successful login, find all unread emails through the email_server.search(None,'UNSEEN') method, and get the email content through the email_server.fetch() and email.message_from_string() methods.
[0026] Specifically, when reading emails through the selenium library, the selenium library is used to realize web page automation operations. On the mailbox web page, the selenium library is used to simulate human operations to realize mailbox account login, email reading, and attachment download operations. The main methods used to implement this method are creating a browser object: driver = webdriver.Chrome(), opening the target URL: driver.get(), locating web page elements: driver.find_element(), clicking operations: element.click(), and input operations: element.send_keys().
[0027] In this embodiment, after retrieving the unread emails with attachments in step S3, it is necessary to extract the email information, which includes the title, subject, content, attachments, sender, and sending time of the email, and save the information to the database. By extracting the key fields of the email information, the category of the email is determined.
[0028] Specifically, there are many different types of customs clearance materials, including import / export declarations, verification lists, packing lists, and draft lists. Due to the diversity, the templates of the main types of customs clearance materials are pre-saved in the attachment template database as the default templates.
[0029] In this embodiment, the attachment template in steps S4 and S5 is created based on the attachment content and business needs, and is completed through the data that can be obtained from the attachment and the data to be used to achieve related business needs through the attachment.
[0030] Specifically, when the attachment material is an Excel table related to the customs declaration, it is necessary to loop through each Excel workbook and each cell of each workbook, and convert them into JSON strings without changing the grouping structure of the data in the table. Similarly, when the attachment is in a format such as Word or PDF, the corresponding template is also created based on the content of the material and the corresponding business needs.
[0031] In this embodiment, in step S6, the specific operation is to parse the fields in the JSON string in the attachment reference template and extract the values of the corresponding fields in the material, construct a complete JSON string, and use this JSON string as a parameter to request the business interface to implement related business.
[0032] Specifically, different uploaded data formats are diverse, and the data parsing code needs to have a specific connection plan;
[0033] For Excel tables, Python directly processes them through the third-party library pandas, which makes data parsing and extraction simple. At the same time, if the file format is .xls, it can also be operated through the xlrd or xlwt libraries; if the file format is .xlsx, it can also be operated through the openpyxl library;
[0034] For word documents, Python has two libraries, python-docx and pywin32, to implement related operations; the former is mainly used to process word documents in .docx format, and the latter can only run on Windows systems, but it can process both .doc and .docx word documents at the same time;
[0035] For PDF documents, there are many third-party libraries for python to read PDF files, PyPDF2 and pdfplumber are commonly used; the former is simpler to use, but may not be sophisticated enough when processing complex PDFs; the latter provides more powerful text extraction functions, especially for the processing of tables and graphics.
[0036] Specifically, the content of word and pdf attachments is presented in the form of relevant instructions plus table pictures. When extracting data, you need to use external recognition services. The returned results are data in json format and need to be extracted according to the corresponding fields.
[0037] Example 2
[0038] like Figure 2 As shown, an automated mail monitoring and structuring system is characterized by comprising a mail monitoring module and a mail structuring module;
[0039] The mail monitoring module is used to regularly retrieve unread mails in the target monitoring mailbox, traverse the unread mails, check attachments, extract attachments, save them to the attachment library and record mail information, and change the status of unread mails to read;
[0040] The mail structuring module is used to classify attachments, create and save attachment structuring templates, and perform structuring operations on attachment contents.
[0041] In this embodiment, the email monitoring module includes a login submodule, a retrieval submodule, a traversal submodule and an extraction submodule;
[0042] The mail structuring module includes a mail classification submodule, a mail structuring template submodule, a mail content structuring submodule, a mail library and a template library.
[0043] Beneficial effects:
[0044] 1: In terms of efficiency, filling material information into the business system is faster than manually filling it in. With the standardization of submitted materials and the enrichment of template libraries, this filling efficiency will gradually improve.
[0045] 2: The structure is not complicated, with only two parts: email monitoring and email structuring. It is also relatively simple to implement, requiring only simple Python language and database-related knowledge. It also becomes particularly simple to expand the functions of this system. Through preliminary communication, important information that is difficult to parse is added to the email content, thereby reducing the difficulty of parsing attachments and improving business efficiency.
[0046] 3: In terms of cost, it is economical and affordable. The mailbox service for receiving materials is provided free of charge by large manufacturers and no additional purchase is required. The recognition service involved can be achieved by building a training model and integrating it into the system.
[0047] Finally, it should be noted that those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims and their equivalents, the present invention is also intended to include these modifications and variations.
Claims
1. An automated email monitoring and structuring method, characterized in that: The following steps are involved: S1: Retrieve inbox emails; S2: Determine whether there are any unread emails. If yes, traverse the unread email list. If no, enter the waiting period. After the waiting period ends, re-enter step S1. S3: After traversing the unread email list, determine whether there is an attachment. If so, save the email information. If not, enter a waiting period. After the waiting period ends, re-enter step S1; S4: Search the attachment template library to see if there is a corresponding template. If not, proceed to step S5; if yes, proceed to step S6: S5: Notify the administrator to create a corresponding template and proceed to step S6; S6: Generate structured data of the attachment material based on the template, and request the business interface to implement related business.
2. The automated email monitoring and structuring method according to claim 1, characterized in that: In step S1, the retrieved mailbox opens the POP3 / IMAP service, obtains the mailbox's authorization code, logs in to the mailbox through Python code, and operates the email.
3. The automated email monitoring and structuring method according to claim 2, characterized in that: A running environment is required for email monitoring and structuring. Anaconda3 is installed to run Python code. The third-party dependency libraries imaplib, selenium, pymysql, and email used to operate mailbox emails are installed through the pip command. The MySQL database is installed to realize data storage.
4. The automated email monitoring and structuring method according to claim 3, characterized in that: After retrieving the unread emails with attachments in step S3, it is necessary to extract the email information, which includes the email title, subject, content, attachments, sender, and sending time, and save the information to the database. By extracting the key fields of the email information, the category of the email is determined.
5. The automated email monitoring and structuring method according to claim 4, characterized in that: The attachment template in steps S4 and S5 is created based on the attachment content and business needs, and is completed by the data that can be obtained from the attachment and the data to be used to achieve related business needs through the attachment.
6. The automated email monitoring and structuring method according to claim 5, characterized in that: In step S6, the specific operation is to parse and extract the values of the corresponding fields in the material from the fields in the JSON string in the attachment comparison template, construct a complete JSON string, and use this JSON string as a parameter to request the business interface to implement related business.
7. An automated mail monitoring and structuring system, characterized in that: Includes mail monitoring module and mail structuring module; The mail monitoring module is used to regularly retrieve unread mails in the target monitoring mailbox, traverse the unread mails, check the attachments, extract the attachments, save them to the attachment library and record the mail information, and change the status of the unread mails to read; The mail structuring module is used to classify attachments, create and save attachment structuring templates, and perform structuring operations on attachment contents.
8. The automated mail monitoring and structuring system according to claim 7, characterized in that: The mail monitoring module includes a login submodule, a retrieval submodule, a traversal submodule and an extraction submodule; The mail structuring module includes a mail classification submodule, a mail structuring template submodule, a mail content structuring submodule, a mail library and a template library.
