Mail processing method, system and device, medium and computer program product

Abstract

The invention provides a mail processing method, system and device, a medium and a computer program product. The processing method comprises the following steps: acquiring a user mail; in response to the fact that the user mail is an effective and single-demand target mail, inputting the target mail into a large language model to obtain a classification result of the target mail; and analyzing the target mail according to the classification result. According to the method, the effective and single-demand target mails are classified based on the large language model, the classification result of the target mails is obtained, automatic processing of customer mail demands is achieved, the mail processing time is effectively shortened, the processing efficiency and accuracy of various mail demands are improved, and the customer satisfaction degree is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of information processing technology, and in particular to a mail processing method, system, device, medium and computer program product. Background Art

[0002] In the current TMC (travel management) industry, many corporate customers initiate travel bookings and configure travel accounts via email;

[0003] Usually, when customers express their needs through emails, there are various ways and forms of expression. However, it is difficult to understand the diversity of customers' expressions at present, and there is a lack of understanding of business travel scenarios and internal systems. The processing efficiency and accuracy of various email needs of customers are low;

[0004] As the scale of enterprises continues to expand and travel needs become more frequent, customers' requirements and expectations for processing timeliness are constantly increasing. The current internal team is already under tremendous pressure under traditional manual identification and processing methods. It is imperative to use technical means to optimize and assist in processing email needs. Summary of the invention

[0005] The technical problem to be solved by the present disclosure is to overcome the defects of low processing efficiency and low accuracy in the prior art of using traditional manual identification and manual processing to handle customers' email needs, and to provide an email processing method, system, device, medium and computer program product.

[0006] The present invention solves the above technical problems through the following technical solutions:

[0007] A first aspect of the present disclosure provides a method for processing an email, the method comprising:

[0008] Get user email;

[0009] In response to the user email being a valid target email with a single demand, inputting the target email into a large language model to obtain a classification result of the target email;

[0010] The target email is parsed according to the classification result.

[0011] Preferably, the processing method further comprises:

[0012] In response to the user email being an invalid email, filtering the invalid email.

[0013] Preferably, in response to the user email being an invalid email, the step of filtering the invalid email includes:

[0014] Obtaining the recipient and sender of the user's email;

[0015] In response to the recipient and the sender hitting a blacklist maintained by the service, determining that the user email is an invalid email, and filtering the invalid email;

[0016] and / or,

[0017] In response to the user email being an invalid email, the step of filtering the invalid email further includes:

[0018] Obtain the email body of the user's email;

[0019] In response to the email body containing invalid keywords, the user email is determined to be an invalid email, and the invalid email is filtered.

[0020] Preferably, in response to the user email being an invalid email, the step of filtering the invalid email further comprises:

[0021] In response to the recipient and sender of the user email not hitting the blacklist maintained by the service, and the email body of the user email not containing invalid keywords, obtaining a concatenation vector of the email subject and the email body of the user email;

[0022] Inputting the concatenated vector into a large language model to obtain a probability value of the user's email;

[0023] In response to the probability value being greater than a preset probability value, determining that the user email is an invalid email, and filtering the invalid email;

[0024] and / or,

[0025] The processing method also includes:

[0026] In response to the user mail being a valid and non-single-required target mail, the valid and non-single-required target mail is stored in a manual processing area.

[0027] Preferably, the processing method further comprises:

[0028] Obtaining the distance value between the vector of the target email and the vectors of each email type;

[0029] In response to the distance value being less than a distance threshold, determining that the target mail belongs to a majority class;

[0030] Wherein, in the case that the target email includes a majority of categories, the classification result of the target email includes at least one of a card opening type, a level type, and a modification type.

[0031] Preferably, the classification result of the target email includes a level type, and the step of parsing the target email according to the classification result includes:

[0032] Obtaining the email content of the target email, wherein the email content includes the email subject, email body and email attachments;

[0033] Utilizing the large language model to parse the email content, and obtaining the information of the person to be checked;

[0034] In response to the information of the person to be checked meeting the checkpoint conditions, closing the travel account corresponding to the information of the person to be checked;

[0035] and / or,

[0036] The classification result of the target email includes a modification type, and the step of parsing the target email according to the classification result includes:

[0037] Obtaining the email content of the target email, wherein the email content includes the email subject, email body and email attachments;

[0038] Extracting the information to be modified from the email content using the large language model;

[0039] Obtaining preset modified information corresponding to the information to be modified;

[0040] In response to the information to be modified meeting the automatic modification condition, the information to be modified and the preset modified information are parsed using the large language model, and the parsing result is returned in a lightweight data exchange format;

[0041] In response to the person information to be modified in the information to be modified in the analysis result being true, the information to be modified is modified to the preset modified information.

[0042] A second aspect of the present disclosure provides a mail processing system, the processing system comprising:

[0043] The first acquisition module is used to acquire user emails;

[0044] A second acquisition module is used for inputting the target email into a large language model in response to the user email being a valid target email with a single demand, to obtain a classification result of the target email;

[0045] The parsing module is used to parse the target email according to the classification result.

[0046] Preferably, the processing system further comprises:

[0047] The filtering module is used for filtering the invalid email in response to the user email being an invalid email.

[0048] Preferably, the filtering module comprises:

[0049] A first acquisition unit, used to acquire the recipient and sender of the user's email;

[0050] A first determination unit, configured to determine that the user email is an invalid email in response to the recipient and the sender hitting a blacklist maintained by the service, and filter the invalid email;

[0051] and / or,

[0052] The filtering module also includes:

[0053] A second acquisition unit, used to acquire the email body of the user email;

[0054] The second determination unit is configured to determine that the user email is an invalid email in response to the email body containing an invalid keyword, and filter the invalid email.

[0055] Preferably, the filtering module further includes:

[0056] A third acquisition unit is used to acquire a concatenation vector of the email subject and the email body of the user email in response to the recipient and the sender of the user email not hitting the blacklist maintained by the service and the email body of the user email not containing invalid keywords;

[0057] A fourth acquisition unit, configured to input the concatenated vector into a large language model to obtain a probability value of the user email;

[0058] A third determining unit, configured to determine that the user email is an invalid email in response to the probability value being greater than a preset probability value, and filter the invalid email;

[0059] and / or,

[0060] The processing system further comprises:

[0061] The storage module is used for storing the valid and non-single-requirement target mail in the manual processing area in response to the user mail being a valid and non-single-requirement target mail.

[0062] Preferably, the processing system further comprises:

[0063] A third acquisition module, used to acquire the distance value between the vector of the target email and the vectors of each email type;

[0064] a determination module, configured to determine, in response to the distance value being less than a distance threshold, that the target mail belongs to a majority class;

[0065] Wherein, in the case that the target email includes a majority of categories, the classification result of the target email includes at least one of a card opening type, a level type, and a modification type.

[0066] Preferably, the classification result of the target email includes a level type, and the parsing module includes:

[0067] A fifth acquisition unit, configured to acquire the email content of the target email, wherein the email content includes the email subject, email body, and email attachments;

[0068] A first parsing unit, configured to parse the email content using the large language model to obtain the information of the person to be checked;

[0069] A closing unit, configured to close the travel account corresponding to the information of the person to be checked in, in response to the information of the person to be checked in meeting the check-in condition;

[0070] and / or,

[0071] The classification result of the target email includes a modification type, and the parsing module includes:

[0072] A sixth acquisition unit, configured to acquire the email content of the target email, wherein the email content includes the email subject, email body, and email attachments;

[0073] An extraction unit, configured to extract the information to be modified from the email content by using the large language model;

[0074] A seventh obtaining unit, used to obtain preset modified information corresponding to the information to be modified;

[0075] A second parsing unit is used for parsing the information to be modified and the preset modified information using the large language model in response to the information to be modified meeting the automatic modification condition, and returning the parsing result in a lightweight data exchange format;

[0076] The modification unit is used for modifying the information to be modified into the preset modified information in response to the person information to be modified in the information to be modified in the analysis result being true.

[0077] A third aspect of the present disclosure provides an electronic device, including a memory, a processor, and a computer program stored in the memory and used to run on the processor, and when the processor executes the computer program, the email processing method described in the first aspect is implemented.

[0078] A fourth aspect of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method for processing mails described in the first aspect is implemented.

[0079] A fifth aspect of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the mail processing method as described in the first aspect.

[0080] On the basis of being in accordance with the common sense in the art, the above-mentioned preferred conditions can be arbitrarily combined to obtain the preferred embodiments of the present disclosure.

[0081] The positive and progressive effects of this disclosure are:

[0082] The present invention classifies valid and single-demand target emails based on a large language model to obtain classification results of the target emails, thereby realizing automated processing of customer email demands, effectively shortening email processing time, improving processing efficiency and accuracy for various email demands, and improving customer satisfaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] Figure 1 This is a flowchart of the email processing method provided in Example 1 of the present disclosure.

[0084] Figure 2 A schematic diagram of the modules of the mail processing system provided in Embodiment 2 of the present disclosure.

[0085] Figure 3 This is a schematic diagram of the structure of an electronic device for implementing the email processing method of embodiment 3 of the present disclosure. DETAILED DESCRIPTION

[0086] The present disclosure is further described below by way of examples, but the present disclosure is not limited to the scope of the examples.

[0087] Prefixes such as "first" and "second" are used in the embodiments of the present disclosure only to distinguish different description objects, and have no limiting effect on the position, order, priority, quantity or content of the described objects. The use of prefixes such as ordinal numbers to distinguish description objects in the embodiments of the present disclosure does not constitute a limitation on the described objects. For the statement of the described objects, please refer to the description in the context of the claims or embodiments, and no unnecessary limitation should be constituted due to the use of such prefixes. In addition, in the description of the present embodiment, unless otherwise specified, the meaning of "plurality" is two or more.

[0088] In the embodiments of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0089] Example 1

[0090] Figure 1 A flowchart of a method for processing mail provided in Embodiment 1 of the present disclosure, such as Figure 1 As shown, the processing method includes:

[0091] S1. Get user emails;

[0092] S2, in response to the user email being a valid target email with a single demand, inputting the target email into a large language model (LLMs) to obtain a classification result of the target email;

[0093] S3. Analyze the target email according to the classification result.

[0094] This embodiment classifies valid and single-demand target emails based on a large language model to obtain classification results of the target emails, thereby realizing automated processing of customer email demands, effectively shortening email processing time, improving processing efficiency and accuracy for various email demands, and improving customer satisfaction.

[0095] In an optional embodiment, the processing method further includes:

[0096] S0. In response to the user's email being invalid, filter the invalid email.

[0097] In this embodiment, it is determined whether the user email is an invalid email, for example, it is determined whether the user email is a polite question or answer message or other user email with no actual demand or no need for reply.

[0098] In an optional embodiment, S0 includes:

[0099] Get the recipients and senders of user emails;

[0100] In response to a recipient or sender hitting a blacklist maintained by the business, determining that the user's email is invalid, and filtering the invalid email;

[0101] and / or,

[0102] S0 also includes:

[0103] Get the email body of the user's email;

[0104] In response to the email body containing an invalid keyword, the user email is determined to be an invalid email, and the invalid email is filtered.

[0105] In an optional embodiment, S0 further includes:

[0106] In response to the recipient and the sender of the user's email not hitting the blacklist maintained by the business, and the email body of the user's email not containing invalid keywords, obtaining a concatenation vector of the email subject and the email body of the user's email;

[0107] Input the concatenated vector into a large language model to obtain the probability value of the user's email;

[0108] In response to the probability value being greater than a preset probability value, determining that the user email is an invalid email, and filtering the invalid email;

[0109] In this embodiment, the preset probability value is set according to actual conditions and is not specifically limited here.

[0110] In the specific implementation process, the invalid email judgment process is divided into two steps. The first step is to judge whether the email is invalid through the vocabulary and rules. If the first step is judged to be valid, it is necessary to enter the second step to finally determine whether the email is valid through a large language model; specifically, the rule judgment is mainly divided into invalid senders and recipients and minority language keyword hits. If the sender and recipient of an email overlaps with the address blacklist maintained by the business, or the body of the email hits the keywords of the maintained minority language, the email will be judged as an invalid email. That is, the rule judgment includes the judgment of the sender and recipient and the content of the body. For the sender and recipient, by judging whether it hits the blacklist maintained by the business, if the sender and recipient overlaps with the blacklist, the user's email is judged to be an invalid email; for the body of the email, by judging whether it contains invalid keywords of the minority language maintained in advance, if it does, the user's email is judged to be an invalid email.

[0111] If the recipient and sender of the user's email do not hit the blacklist maintained by the business, and the body of the user's email does not contain invalid keywords, this stage will be used to determine whether the user's email is ultimately valid. The subject and body of an email will be converted into two vectors through text vectorization technology and concatenated and passed into the large language model. Finally, the probability value output by the large language model is used to determine whether the user's email is an invalid email. Specifically, a large language model is needed to determine whether the user's email is valid. The input of the large language model is the vectorized representation of the email. The title and body of the email are converted into 1536-dimensional vectors respectively through the interface of the text vectorization technology, and then concatenated to obtain a 3072-dimensional vector as the input of the large language model. The structure of the large language model is a 2-layer neural network model. The first layer is an LSTM structure, which is responsible for accepting a 3072-dimensional vector, extracting feature information, and converting it into a 128-dimensional latent vector; the second layer is a linear layer, which is responsible for mapping the 128-dimensional latent vector into a scalar, and finally converting the output of the neural network model to between 0 and 1 through the SIGMOD function. The numerical value means the probability value of the user's email being invalid. If the probability value is greater than the preset probability value (for example, the preset probability value is 0.5), the user's email is judged to be invalid, otherwise it is judged to be valid.

[0112] In an optional embodiment, the processing method further includes:

[0113] In response to the user mail being a valid target mail that is not a single-demand (eg, multiple-demand) target mail, the valid target mail that is not a single-demand is stored in a manual processing area.

[0114] In this embodiment, if the user's email is valid and not a single required target email, then the target email is a mixed email. For the mixed email, it will not enter the next step of automatic analysis, but will be manually processed.

[0115] In an optional embodiment, the processing method further includes:

[0116] Get the distance value between the target email vector and the vectors of each email type;

[0117] In response to the distance value being less than the distance threshold, determining that the target email belongs to the majority class;

[0118] Wherein, in the case where the target email includes a majority of categories, the classification result of the target email includes at least one of a card opening type, a level type, and a modification type.

[0119] In this embodiment, in response to the distance value being not less than the distance threshold, it is determined that the target email belongs to the minority class;

[0120] It should be noted that the distance threshold is set according to actual conditions and is not specifically limited here.

[0121] In the specific implementation process, the classification of single demand mail is divided into two stages: a) division between majority and minority classes; b) specific type judgment;

[0122] a) In the data distribution of emails, the three types of emails, modification, level, and activation, are the most common and are classified into the majority class, while other email types are classified into the minority class. In the division of the majority class and the minority class, the distance value between the email vectorized and each email type is calculated. If the distance value falls into the modification, level, and activation (for example, the distance value is less than the distance threshold), the email is judged to belong to the majority class, otherwise, the email is judged to belong to the minority class.

[0123] b) If the email is judged to belong to the majority class, the pre-trained majority class model is used to output that the email belongs to a specific class of modification, level, or card opening; if the email is judged to belong to the minority class, the pre-trained minority class model is used to output that the email belongs to a specific class in the minority class.

[0124] Specifically, if an email is determined to be valid, the email type will be determined to facilitate subsequent automated analysis. There are a total of 8 types of emails, among which modification, card opening and leveling account for the highest proportion and are recorded as the majority category, and the remaining 5 categories are recorded as minority categories. The email classification process first determines whether the email belongs to the majority category or the minority category, and then performs segmentation in the corresponding type. For example: If an email is determined to be in the majority category, then in the segmentation results, its type will only be one of the modification type, card opening type and leveling type.

[0125] 1) The method for determining the majority class and the minority class is to calculate the distance value between the email vector and the corresponding type vector. The email vectorization method is the same as above, and the calculation logic of the type vector is the mean of the vector set of historical emails of the corresponding type. By calculating the distance value between the email and each type and sorting the distance values ​​in ascending order, if the email type with the closest distance value falls into one of the modification, opening and closing (for example, the distance value is less than the distance threshold), then the email is determined to be the majority class; otherwise, it is determined to be the minority class.

[0126] 2) After determining whether the email belongs to the majority class or the minority class, the model will be used to determine the specific type of the email. The input and first-layer structure of the model are consistent with the large language model, and the output structure of the model depends on whether the email belongs to the majority class. If the email belongs to the majority class, the model outputs a 3-dimensional vector, which is converted into the probability value corresponding to [modify, open card, level] through the SIGMOD function to determine the specific type of the email; if the email belongs to the minority class, the model outputs a 5-dimensional vector, and the subsequent process is consistent with the majority class.

[0127] It should be noted that if an email is determined to be a level or modified type, it will be determined whether it is a mixed email. If the email is determined to be a mixed email, it will not enter the next step of automatic analysis.

[0128] The classification of mixed mails is based on two types of mixed mails: <level, other> and <modification, other>. That is, after a single demand mail is classified as a level type or a modification type, the classification of the mixed mail will be determined. The mixed mail is determined by a pre-trained mixed mail classification model. The training data of the mixed mail classification model here only accepts single examples of the corresponding type and its corresponding mixed mail. For example, the training data of the level mixed mail classification model only includes mails with a single demand level and mixed mails that contain levels.

[0129] The structure of the hybrid email classification model is an 8-layer linear network. A BatchNorm layer is added between every two layers to improve the model's fitting ability. A ReLU activation function is added to improve the model's nonlinear expression ability. The input of the hybrid email classification model is the vectorized concatenation of the email title and body, which is consistent with the large language model. The output of the hybrid email classification model is a 2-dimensional vector, which is mapped to the probability distribution of [hybrid email, non-hybrid email] through the softmax function. If the corresponding probability value exceeds the preset probability value (for example, the preset probability value is 0.5), it is determined to be the corresponding type, that is, the email is a hybrid work order.

[0130] In an optional embodiment, the classification result of the target email includes a level type, and S3 includes:

[0131] Get the target email content, including email subject, email body and email attachments;

[0132] Use a large language model to parse the email content and obtain the information of the person to be checked;

[0133] In response to the information of the person to be checked meeting the checkpoint conditions, closing the travel account corresponding to the information of the person to be checked;

[0134] In this embodiment, the checkpoint condition includes that the information of the person to be checked in is real and the ID of the person to be checked in is unique.

[0135] In the specific implementation process, for the specific type of a single demand email, if it is a level type, then enter the content parsing phase of this part. The specific process is as follows: Get the email content of the target email, including the email subject, email body and email attachments. If the email attachment is a spreadsheet file such as Excel, the Excel text content needs to be extracted;

[0136] Call the large language model to parse all the extracted email contents, extract the required personnel information, and return it in JSON format;

[0137] Clean and format the results returned by the large language model, and verify the returned results. The verification method is to call the internal interface to query whether the extracted personnel information of the required checkpoints really exists and the user ID is unique. If the personnel information of the required checkpoints is true, the automated checkpoints will be performed later.

[0138] In an optional embodiment, the classification result of the target email includes a modification type, and S3 includes:

[0139] Get the target email content, including email subject, email body and email attachments;

[0140] Use a large language model to extract the information to be modified from the email content;

[0141] Obtain the preset modified information corresponding to the information to be modified;

[0142] In response to the information to be modified meeting the automatic modification conditions, the information to be modified and the preset modified information are parsed using a large language model, and the parsing result is returned in a lightweight data exchange format;

[0143] In response to the fact that the person information to be modified in the information to be modified in the analysis result is true, the information to be modified is modified to preset modified information.

[0144] In this embodiment, the automatic modification condition is high-frequency modification of information, such as modification of personal information.

[0145] In the specific implementation process, for the specific type of a single demand email, if it is a modification type, it will enter the content parsing phase of this part. The specific process is as follows:

[0146] 1) First, obtain the target email content, including the email subject, email body and email attachments. If the email attachment is an Excel spreadsheet file, you need to extract its text content first. If the body is a text in HTML format, you need to extract the text information and remove extra invalid characters.

[0147] 2) Since there are many user information fields that can be modified, it is necessary to determine whether it is a high-frequency modification information (that is, whether it meets the conditions for automated modification), such as modifying personal information. This part will enter the subsequent automated modification, while non-high-frequency modification information will enter the manual processing area. This step requires the use of a large language model to determine whether the modification information mentioned in the email content is in the high-frequency modification field, and then perform different processing. The extraction method is to construct a prompt to call the large language model interface to extract the modified field list, and then use keywords to determine whether it is a high-frequency modification information;

[0148] 3) If it is a high-frequency modification information, the large language model interface is called again to parse the original information and the modified information that need to be modified, and return them in JSON format; if it is a low-frequency modification information, all the modified contents are extracted and returned in table format;

[0149] 4) Clean, format and verify the extracted information of frequently modified information. The verification method is to call the internal interface to query whether the extracted personnel information that needs to be modified really exists and the user ID is unique. If the personnel information that needs to be modified is true, the subsequent process of automatically modifying the information will be carried out.

[0150] This embodiment uses a large language model and combines it with business scenarios to intelligently identify corporate customer emails, extract key information and intelligently parse email content, including identifying invalid user reply emails, classifying emails into 9 major subject types, and parsing and extracting valid information for different types, filling in fixed-structure text forms to filter invalid emails, classify emails of different categories, and semantically parse and structure under each category, thereby automating the processing of account configuration tasks, effectively shortening the internal team's email processing time, improving internal service efficiency and customer responsiveness, and ultimately improving customer satisfaction. The large language model will return the following content: whether it is an invalid email (for example, whether it is a customer email with no actual demand or no need to reply, such as a polite question and answer); whether it is a mixed type email (for example, a customer mentioned multiple different types of needs in one email); specific email type: parsed fields under a specific type (including company, verified user ID, operations to be performed, and fields to be modified).

[0151] This embodiment combines LLMs (large language models), artificial intelligence and actual business scenarios to automatically process emails. It has been applied to the following scenarios: the service team uses this technology to mark and automatically filter invalid reply emails from corporate users, improving the operational efficiency of the service team; the account team uses this technology to classify corporate user emails, realize automatic task assignment, and perform semantic recognition and analysis. After converting them into structured fields, they can be linked with the internal account system and process configuration tasks with one click. The application in the above scenarios optimizes resource allocation, allowing the internal team to focus on higher-value tasks, effectively improving the team's processing efficiency for various types of user email needs, and ensuring service quality and consistency to a greater extent, thereby improving customer satisfaction.

[0152] Example 2

[0153] Corresponding to the aforementioned embodiment of a method for processing mail, the present disclosure also provides an embodiment of a system for processing mail.

[0154] Figure 2 A module diagram of a mail processing system provided in Embodiment 2 of the present disclosure, such as Figure 2 As shown, the processing system includes: a first acquisition module 21, a second acquisition module 22, and a parsing module 23;

[0155] The first acquisition module 21 is used to acquire user emails;

[0156] The second acquisition module 22 is used for inputting the target email into the large language model in response to the user email being a valid target email with a single demand, and obtaining a classification result of the target email;

[0157] The parsing module 23 is used to parse the target email according to the classification result.

[0158] This embodiment classifies valid and single-demand target emails based on a large language model to obtain classification results of the target emails, thereby realizing automated processing of customer email demands, effectively shortening email processing time, improving processing efficiency and accuracy for various email demands, and improving customer satisfaction.

[0159] In an optional embodiment, the processing system further includes:

[0160] The filtering module is used for filtering invalid emails in response to the user email being invalid email.

[0161] In this embodiment, it is determined whether the user email is an invalid email, for example, it is determined whether the user email is a polite question or answer message or other user email with no actual demand or no need for reply.

[0162] In an optional embodiment, the filtering module includes:

[0163] A first acquisition unit, used to acquire the recipient and sender of the user's email;

[0164] A first determination unit, configured to determine that a user's email is invalid email in response to a recipient and a sender hitting a blacklist maintained by the service, and filter the invalid email;

[0165] and / or,

[0166] The filtering module also includes:

[0167] The second acquisition unit is used to acquire the email body of the user's email;

[0168] The second determination unit is used to determine that the user email is an invalid email in response to the email body containing an invalid keyword, and filter the invalid email.

[0169] In an optional embodiment, the filtering module further includes:

[0170] A third acquisition unit is used to acquire a concatenation vector of an email subject and an email body of the user email in response to the recipient and sender of the user email not hitting the blacklist maintained by the service and the email body of the user email not containing invalid keywords;

[0171] A fourth acquisition unit, used for inputting the concatenated vector into a large language model to obtain a probability value of the user's email;

[0172] A third determination unit, configured to determine that the user email is an invalid email in response to the probability value being greater than a preset probability value, and filter the invalid email;

[0173] In this embodiment, the preset probability value is set according to actual conditions and is not specifically limited here.

[0174] In the specific implementation process, the invalid email judgment process is divided into two steps. The first step is to judge whether the email is invalid through the vocabulary and rules. If the first step is judged to be valid, it is necessary to enter the second step to finally determine whether the email is valid through a large language model; specifically, the rule judgment is mainly divided into invalid senders and recipients and minority language keyword hits. If the sender and recipient of an email overlaps with the address blacklist maintained by the business, or the body of the email hits the keywords of the maintained minority language, the email will be judged as an invalid email. That is, the rule judgment includes the judgment of the sender and recipient and the content of the body. For the sender and recipient, by judging whether it hits the blacklist maintained by the business, if the sender and recipient overlaps with the blacklist, the user's email is judged to be an invalid email; for the body of the email, by judging whether it contains invalid keywords of the minority language maintained in advance, if it does, the user's email is judged to be an invalid email.

[0175] If the recipient and sender of the user's email do not hit the blacklist maintained by the business, and the body of the user's email does not contain invalid keywords, this stage will be used to determine whether the user's email is ultimately valid. The subject and body of an email will be converted into two vectors through text vectorization technology and concatenated and passed into the large language model. Finally, the probability value output by the large language model is used to determine whether the user's email is an invalid email. Specifically, a large language model is needed to determine whether the user's email is valid. The input of the large language model is the vectorized representation of the email. The title and body of the email are converted into 1536-dimensional vectors respectively through the interface of the text vectorization technology, and then concatenated to obtain a 3072-dimensional vector as the input of the large language model. The structure of the large language model is a 2-layer neural network model. The first layer is an LSTM structure, which is responsible for accepting a 3072-dimensional vector, extracting feature information, and converting it into a 128-dimensional latent vector; the second layer is a linear layer, which is responsible for mapping the 128-dimensional latent vector into a scalar, and finally converting the output of the neural network model to between 0 and 1 through the SIGMOD function. The numerical value means the probability value of the user's email being invalid. If the probability value is greater than the preset probability value (for example, the preset probability value is 0.5), the user's email is judged to be invalid, otherwise it is judged to be valid.

[0176] In an optional embodiment, the processing system further includes:

[0177] The storage module is used for storing the valid and non-single-requirement target mail into the manual processing area in response to the user mail being a valid and non-single-requirement target mail.

[0178] In this embodiment, if the user's email is valid and not a single required target email, then the target email is a mixed email. For the mixed email, it will not enter the next step of automatic analysis, but will be manually processed.

[0179] In an optional embodiment, the processing system further includes:

[0180] The third acquisition module is used to obtain the distance value between the vector of the target email and the vectors of each email type;

[0181] a determination module, configured to determine that the target mail belongs to the majority class in response to the distance value being less than a distance threshold;

[0182] Wherein, in the case where the target email includes a majority of categories, the classification result of the target email includes at least one of a card opening type, a level type, and a modification type.

[0183] In this embodiment, in response to the distance value being not less than the distance threshold, it is determined that the target email belongs to the minority class;

[0184] It should be noted that the distance threshold is set according to actual conditions and is not specifically limited here.

[0185] In the specific implementation process, the classification of single demand mail is divided into two stages: a) division between majority and minority classes; b) specific type judgment;

[0186] a) In the data distribution of emails, the three types of emails, modification, level, and activation, are the most common and are classified into the majority class, while other email types are classified into the minority class. In the division of the majority class and the minority class, the distance value between the email vectorized and each email type is calculated. If the distance value falls into the modification, level, and activation (for example, the distance value is less than the distance threshold), the email is judged to belong to the majority class, otherwise, the email is judged to belong to the minority class.

[0187] b) If the email is judged to belong to the majority class, the pre-trained majority class model is used to output that the email belongs to a specific class of modification, level, or card opening; if the email is judged to belong to the minority class, the pre-trained minority class model is used to output that the email belongs to a specific class in the minority class.

[0188] Specifically, if an email is determined to be valid, the email type will be determined to facilitate subsequent automated analysis. There are a total of 8 types of emails, among which modification, card opening and leveling account for the highest proportion and are recorded as the majority category, and the remaining 5 categories are recorded as minority categories. The email classification process first determines whether the email belongs to the majority category or the minority category, and then performs segmentation in the corresponding type. For example: If an email is determined to be in the majority category, then in the segmentation results, its type will only be one of the modification type, card opening type and leveling type.

[0189] 1) The method for determining the majority class and the minority class is to calculate the distance value between the email vector and the corresponding type vector. The email vectorization method is the same as above, and the calculation logic of the type vector is the mean of the vector set of historical emails of the corresponding type. By calculating the distance value between the email and each type and sorting the distance values ​​in ascending order, if the email type with the closest distance value falls into one of the modification, opening and closing (for example, the distance value is less than the distance threshold), then the email is determined to be the majority class; otherwise, it is determined to be the minority class.

[0190] 2) After determining whether the email belongs to the majority class or the minority class, the model will be used to determine the specific type of the email. The input and first-layer structure of the model are consistent with the large language model, and the output structure of the model depends on whether the email belongs to the majority class. If the email belongs to the majority class, the model outputs a 3-dimensional vector, which is converted into the probability value corresponding to [modify, open card, level] through the SIGMOD function to determine the specific type of the email; if the email belongs to the minority class, the model outputs a 5-dimensional vector, and the subsequent process is consistent with the majority class.

[0191] It should be noted that if an email is determined to be a level or modified type, it will be determined whether it is a mixed email. If the email is determined to be a mixed email, it will not enter the next step of automatic analysis.

[0192] The classification of mixed mails is based on two types of mixed mails: <level, other> and <modification, other>. That is, after a single demand mail is classified as a level type or a modification type, the classification of the mixed mail will be determined. The mixed mail is determined by a pre-trained mixed mail classification model. The training data of the mixed mail classification model here only accepts single examples of the corresponding type and its corresponding mixed mail. For example, the training data of the level mixed mail classification model only includes mails with a single demand level and mixed mails that contain levels.

[0193] The structure of the hybrid email classification model is an 8-layer linear network. A BatchNorm layer is added between every two layers to improve the model's fitting ability. A ReLU activation function is added to improve the model's nonlinear expression ability. The input of the hybrid email classification model is the vectorized concatenation of the email title and body, which is consistent with the large language model. The output of the hybrid email classification model is a 2-dimensional vector, which is mapped to the probability distribution of [hybrid email, non-hybrid email] through the softmax function. If the corresponding probability value exceeds the preset probability value (for example, the preset probability value is 0.5), it is determined to be the corresponding type, that is, the email is a hybrid work order.

[0194] In an optional embodiment, the classification result of the target email includes a level type, and the parsing module includes:

[0195] A fifth acquisition unit, used to acquire the email content of the target email, the email content including the email subject, email body and email attachments;

[0196] The first parsing unit is used to parse the email content using a large language model to obtain the information of the person to be checked;

[0197] A closing unit, configured to close the travel account corresponding to the information of the person to be checked in, in response to the information of the person to be checked in meeting the check-in conditions;

[0198] In this embodiment, the checkpoint condition includes that the information of the person to be checked in is real and the ID of the person to be checked in is unique.

[0199] In the specific implementation process, for the specific type of a single demand email, if it is a level type, then enter the content parsing phase of this part. The specific process is as follows: Get the email content of the target email, including the email subject, email body and email attachments. If the email attachment is a spreadsheet file such as Excel, the Excel text content needs to be extracted;

[0200] Call the large language model to parse all the extracted email contents, extract the required personnel information, and return it in JSON format;

[0201] Clean and format the results returned by the large language model, and verify the returned results. The verification method is to call the internal interface to query whether the extracted personnel information that needs to be checked really exists and the user ID is unique. If the personnel information that needs to be checked is true, then the automated checkpoint will be performed later.

[0202] In an optional embodiment, the classification result of the target email includes a modification type, and the parsing module includes:

[0203] The sixth acquisition unit is used to acquire the email content of the target email, where the email content includes the email subject, email body and email attachments;

[0204] An extraction unit, used for extracting information to be modified from the email content using a large language model;

[0205] A seventh obtaining unit, used to obtain preset modified information corresponding to the information to be modified;

[0206] A second parsing unit is used for parsing the information to be modified and the preset modified information using a large language model in response to the information to be modified meeting the automatic modification condition, and returning the parsing result in a lightweight data exchange format;

[0207] The modification unit is used to modify the information to be modified into preset modified information in response to the person information to be modified in the information to be modified in the analysis result being true.

[0208] In this embodiment, the automatic modification condition is high-frequency modification of information, such as modification of personal information.

[0209] In the specific implementation process, for the specific type of a single demand email, if it is a modification type, it will enter the content parsing phase of this part. The specific process is as follows:

[0210] 1) First, obtain the target email content, including the email subject, email body and email attachments. If the email attachment is an Excel spreadsheet file, you need to extract its text content first. If the body is a text in HTML format, you need to extract the text information and remove extra invalid characters.

[0211] 2) Since there are many user information fields that can be modified, it is necessary to determine whether it is a high-frequency modification information (that is, whether it meets the conditions for automated modification), such as modifying personal information. This part will enter the subsequent automated modification, while non-high-frequency modification information will enter the manual processing area. This step requires the use of a large language model to determine whether the modification information mentioned in the email content is in the high-frequency modification field, and then perform different processing. The extraction method is to construct a prompt to call the large language model interface to extract the modified field list, and then use keywords to determine whether it is a high-frequency modification information;

[0212] 3) If it is a high-frequency modification information, the large language model interface is called again to parse the original information and the modified information that need to be modified, and return them in JSON format; if it is a low-frequency modification information, all the modified contents are extracted and returned in table format;

[0213] 4) Clean, format and verify the extracted information of frequently modified information. The verification method is to call the internal interface to query whether the extracted personnel information that needs to be modified really exists and the user ID is unique. If the personnel information that needs to be modified is true, the subsequent process of automatically modifying the information will be carried out.

[0214] This embodiment uses a large language model and combines it with business scenarios to intelligently identify corporate customer emails, extract key information and intelligently parse email content, including identifying invalid user reply emails, classifying emails into 9 major subject types, and parsing and extracting valid information for different types, filling in fixed-structure text forms to filter invalid emails, classify emails of different categories, and semantically parse and structure under each category, thereby automating the processing of account configuration tasks, effectively shortening the internal team's email processing time, improving internal service efficiency and customer responsiveness, and ultimately improving customer satisfaction. The large language model will return the following content: whether it is an invalid email (for example, whether it is a customer email with no actual demand or no need to reply, such as a polite question and answer); whether it is a mixed type email (for example, a customer mentioned multiple different types of needs in one email); specific email type: parsed fields under a specific type (including company, verified user ID, operations to be performed, and fields to be modified).

[0215] This embodiment combines LLMs (large language models), artificial intelligence and actual business scenarios to automatically process emails. It has been applied to the following scenarios: the service team uses this technology to mark and automatically filter invalid reply emails from corporate users, improving the operational efficiency of the service team; the account team uses this technology to classify corporate user emails, realize automatic task assignment, and perform semantic recognition and analysis. After converting them into structured fields, they can be linked with the internal account system and process configuration tasks with one click. The application in the above scenarios optimizes resource allocation, allowing the internal team to focus on higher-value tasks, effectively improving the team's processing efficiency for various types of user email needs, and ensuring service quality and consistency to a greater extent, thereby improving customer satisfaction.

[0216] As for the system embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The system embodiment described above is only illustrative, wherein the units described as separate components may or may not be physically separated, and the components as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the disclosed solution.

[0217] Example 3

[0218] Figure 3 This is a structural diagram of an electronic device shown in Embodiment 3 of the present disclosure, wherein the electronic device includes a memory, a processor, and a computer program stored in the memory and used to run on the processor, and when the processor executes the computer program, the method for processing emails described in any of the above embodiments is implemented. Figure 3The electronic device 90 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0219] like Figure 3 As shown, the electronic device 90 may be in the form of a general-purpose computing device, for example, it may be a server device. The components of the electronic device 90 may include, but are not limited to: at least one processor 91, at least one memory 92, and a bus 93 connecting different system components (including the memory 92 and the processor 91).

[0220] The bus 93 includes a data bus, an address bus, and a control bus.

[0221] The memory 92 may include a volatile memory, such as a random access memory (RAM) 921 and / or a cache memory 922 , and may further include a read only memory (ROM) 923 .

[0222] The memory 92 may also include a program tool 925 (or utility) having a set (at least one) of program modules 924, such program modules 924 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0223] The processor 91 executes various functional applications and data processing by running the computer programs stored in the memory 92, such as the mail processing method provided in any of the above embodiments.

[0224] The electronic device 90 may also communicate with one or more external devices 94 (e.g., keyboards, pointing devices, etc.). Such communication may be performed via an input / output (I / O) interface 95. Furthermore, the electronic device 90 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 96. Figure 3 As shown, the network adapter 96 communicates with other modules of the electronic device 90 via the bus 93. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the electronic device 90, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems, etc.

[0225] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided into multiple units / modules to be embodied.

[0226] Example 4

[0227] Embodiment 4 of the present disclosure further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method for processing mails provided in any of the above embodiments is implemented.

[0228] The readable storage medium may include but is not limited to: a portable disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device or any suitable combination of the above.

[0229] Example 5

[0230] Embodiment 5 of the present disclosure further provides a computer program product, including a computer program, which implements any of the above-mentioned methods for processing emails when executed by a processor.

[0231] Among them, the program code for executing the computer program product of the present disclosure can be written in any combination of one or more programming languages, and the program code can be executed completely on the user device, partially on the user device, as an independent software package, partially on the user device and partially on a remote device, or completely on the remote device.

[0232] Although the specific embodiments of the present disclosure are described above, those skilled in the art should understand that this is only an example, and the protection scope of the present disclosure is defined by the appended claims. Those skilled in the art may make various changes or modifications to these embodiments without departing from the principles and essence of the present disclosure, but these changes and modifications all fall within the protection scope of the present disclosure.

Claims

1. A method for processing mail, characterized in that: The processing method comprises: Get user email; In response to the user email being a valid target email with a single demand, inputting the target email into a large language model to obtain a classification result of the target email; The target email is parsed according to the classification result.

2. The mail processing method according to claim 1, characterized in that: The processing method also includes: In response to the user email being an invalid email, filtering the invalid email.

3. The mail processing method according to claim 2, characterized in that: In response to the user email being an invalid email, the step of filtering the invalid email includes: Obtaining the recipient and sender of the user's email; In response to the recipient and the sender hitting a blacklist maintained by the service, determining that the user email is an invalid email, and filtering the invalid email; and / or, In response to the user email being an invalid email, the step of filtering the invalid email further includes: Obtain the email body of the user's email; In response to the email body containing invalid keywords, the user email is determined to be an invalid email, and the invalid email is filtered.

4. The mail processing method according to claim 3, characterized in that: In response to the user email being an invalid email, the step of filtering the invalid email further includes: In response to the recipient and the sender of the user's email not hitting the blacklist maintained by the service, and the email body of the user's email not containing invalid keywords, obtaining a concatenation vector of the email subject and the email body of the user's email; Inputting the concatenated vector into a large language model to obtain a probability value of the user's email; In response to the probability value being greater than a preset probability value, determining that the user email is an invalid email, and filtering the invalid email; and / or, The processing method also includes: In response to the user mail being a valid target mail that is not a single-requirement mail, the valid target mail that is not a single-requirement mail is stored in a manual processing area.

5. The mail processing method according to claim 1, characterized in that: The processing method also includes: Obtaining the distance value between the vector of the target email and the vectors of each email type; In response to the distance value being less than a distance threshold, determining that the target mail belongs to a majority class; Wherein, in the case that the target email includes a majority of categories, the classification result of the target email includes at least one of a card opening type, a level type, and a modification type.

6. The mail processing method according to claim 5, characterized in that: The classification result of the target email includes a level type, and the step of parsing the target email according to the classification result includes: Obtaining the email content of the target email, wherein the email content includes the email subject, email body and email attachments; Utilizing the large language model to parse the email content, and obtaining the information of the person to be checked; In response to the information of the person to be checked meeting the checkpoint conditions, closing the travel account corresponding to the information of the person to be checked; and / or, The classification result of the target email includes a modification type, and the step of parsing the target email according to the classification result includes: Obtaining the email content of the target email, wherein the email content includes the email subject, email body and email attachments; Extracting the information to be modified from the email content using the large language model; Obtaining preset modified information corresponding to the information to be modified; In response to the information to be modified meeting the automatic modification condition, the information to be modified and the preset modified information are parsed using the large language model, and the parsing result is returned in a lightweight data exchange format; In response to the person information to be modified in the information to be modified in the analysis result being true, the information to be modified is modified to the preset modified information.

7. A mail processing system, characterized in that: The processing system comprises: The first acquisition module is used to acquire user emails; A second acquisition module is used for inputting the target email into a large language model in response to the user email being a valid target email with a single demand, to obtain a classification result of the target email; The parsing module is used to parse the target email according to the classification result.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and used to run on the processor, characterized in that: When the processor executes the computer program, the mail processing method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the mail processing method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the mail processing method according to any one of claims 1 to 6 is implemented.

Images