A marketing email delivery system and method

Abstract

The present invention relates to a marketing email delivery system and method, belonging to the technical field of email delivery. The delivery system includes an email content generation module, an email delivery module, an email prediction module, a collection and analysis module, and a customer feedback module. The present invention realizes personalized email content generation and multi-modal email content detection through the email content generation module, reducing the costs of manual writing of email template content and manual review of email content. The email delivery module and the email prediction module are used to predict the email content elements before delivery, and make a decision to deliver or cancel the delivery of the current email based on the prediction score, thereby improving the overall delivery success rate of emails. The collection and analysis module realizes continuous optimization of email content and delivery success rate and effectively supplements the local spam feature library. The customer feedback module reduces the complaint rate and improves the customer experience.

Description

Technical Field

[0001] The present invention relates to the technical field of email delivery, and particularly to a marketing email delivery system and method. Background Art

[0002] With the accelerating trend of digital transformation, financial institutions and organizations such as banks, insurance companies, and securities firms are increasingly inclined to try diverse message delivery methods to promptly inform customers of information and conduct business marketing activities. As an indispensable message delivery channel in people's daily lives and work, email is often used by banks to send credit card statements, online marketing campaigns, institutional announcements, etc. to customers, and to push content and information that customers are interested in.

[0003] Currently, there are some drawbacks in the existing email delivery methods in the banking industry. Firstly, there is a lack of personalized email content. That is, most of the email content is manually written and formulated within the institution, unable to adapt to the rapidly changing market environment and customized customer needs. Once customers are not interested in the email content, the sender's email address is easily blocked by customers, resulting in the inability to deliver emails to relevant customers subsequently. Secondly, the delivery strategies and email processing models are single and fixed. The delivered emails are likely to trigger the security or risk control strategies of email service operators, resulting in a large number of emails being intercepted or the delivery failing in a short period of time when delivered to customers. At the same time, there are many email attachment formats and the email content lacks regular verification. Some email features are easily detected as spam, presenting compliance risks.

[0004] Therefore, it is necessary to provide a marketing email delivery system and method to improve the success rate of email delivery and the conversion rate of marketing campaign operations, and to improve the email marketing effect and the customer experience of email services. Summary of the Invention

[0005] The purpose of the present invention is to provide a marketing email delivery system and method to improve the success rate of email delivery and the conversion rate of marketing campaign operations, and to improve the email marketing effect and the customer experience of email services.

[0006] To solve the problems existing in the prior art, the present invention provides a marketing email delivery system, including:

[0007] An email content generation module, configured to load email templates and corresponding content of Prompt prompts from a database as inputs to a natural language large model, and perform semantic splicing according to the context where the email template variables are located and the corresponding content of the Prompt prompts. The natural language large model outputs corresponding text content, and the generated text content fills the placeholders and variables in the email template;

[0008] The email delivery module includes a multi-modal processing layer, a word segmentation layer, an embedding layer, a RoBERTa pre-trained model, and an optimization unit. The RoBERTa pre-trained model consists of multiple layers of bidirectional Transformer encoders. The RoBERTa pre-trained model uses the Sigmoid function as the activation function of the output layer neurons for binary classification prediction, mapping the output vector to the range [0, 1]. The optimization unit is configured to optimize the RoBERTa pre-trained model using the binary cross-entropy function as the Loss function. The Loss function is as follows: where n is the number of layers of the Transformer encoder, i is any layer of the Transformer encoder, y is the binary label value 0 or 1, and p(y) refers to the probability of belonging to the y label;

[0009] The email prediction module is configured to comprehensively integrate the outputs of multiple large models through ensemble learning. Among the multiple large models, there is a RoBERTa pre-trained model. The outputs of each large model are weighted and averaged to obtain an email prediction score, and a decision to deliver or cancel the delivery of the current email is made based on the prediction score.

[0010] Optionally, in the marketing email delivery system, the email content generation module is also configured to perform content review and rule review.

[0011] Optionally, in the marketing email delivery system,

[0012] The multi-modal processing layer is configured to divide the multi-modal fusion input composed of the email title, email body, and email attachment into three separate inputs;

[0013] The word segmentation layer is configured to split all content to obtain multiple language units, process multiple language units and various special characters, and convert all content of the original email into a vector sequence;

[0014] The embedding layer is configured to map the vector sequence into a dense vector of a fixed dimension. The dense vector output by the embedding layer will be used as the input of the Transformer encoder after position encoding;

[0015] The Transformer encoder of the RoBERTa pre-trained model is configured to extract the input content using the multi-head attention mechanism; perform residual connection after fusing the context semantics to retain the basic features of the input; normalize the result of the residual connection to ensure the stability and convergence speed during the training process of the RoBERTa pre-trained model; input the normalized input content into the feed-forward neural network; map the output result processed by the feed-forward neural network to a standard normal distribution through secondary normalization;

[0016] The Transformer decoder of the RoBERTa pre-trained model is configured to transform the complex semantic features output by the Transformer encoder into an interpretable probability distribution. The Transformer decoder includes three different linear transformations and a Sigmoid activation function.

[0017] Optionally, in the marketing email delivery system,

[0018] When the email prediction module makes a prediction, it queries the email template ID to be predicted from the database, and obtains the list of detectable large models and the corresponding Prompt prompts associated with this email template ID. The Prompt prompts and the email content to be detected are used as the inputs of each large model, and each large model makes a prediction.

[0019] Optionally, in the marketing email delivery system, the delivery system further includes a collection and analysis module, which is configured to collect according to the received sending results and the log information printed by the email service middleware, and associate it with the delivery detail data of each email attempted to be delivered, and then deduce the delivery result of each email attempted to be delivered, and reissue according to the deduced delivery result and the reissue rule set.

[0020] Optionally, in the marketing email delivery system, according to the deduced delivery result, and by comparing the old and new Prompt prompts, optimize the configuration interface of the email template.

[0021] Optionally, in the marketing email delivery system, the delivery system further includes a customer feedback module, which is configured to provide a clear unsubscribe link at the bottom of the email and regularly collect user feedback in the form of questionnaires and feedback forms.

[0022] The present invention also provides a method for delivering marketing emails, which uses the described delivery system. The method includes the following steps:

[0023] S1: Load the corresponding content of the email template and the Prompt prompt from the database as the input of the natural language large model, and perform semantic splicing according to the context where the email template variable is located and the corresponding content of the Prompt prompt. The natural language large model outputs the corresponding text content, and the generated text content fills the placeholders and variables in the email template;

[0024] S21: Divide the multi-modal fusion input composed of the email title, the email body, and the email attachment into three separate inputs;

[0025] S22: Split all the content to obtain multiple language units, process the multiple language units and various special characters, and convert all the content of the original email into a vector sequence;

[0026] S23: Map the vector sequence to a dense vector of a fixed dimension. The dense vector output by the embedding layer will be used as the input of the Transformer encoder after positional encoding;

[0027] S24: The Transformer encoder of the RoBERTa pre-trained model uses the multi-head attention mechanism to extract the input content; after fusing the context semantics, a residual connection is performed to retain the basic features of the input; the result of the residual connection is normalized to ensure the stability and convergence speed during the training process of the RoBERTa pre-trained model; the normalized input content is input into the feed-forward neural network; the output result processed by the feed-forward neural network is mapped to a standard normal distribution through a second normalization;

[0028] S25: The Transformer decoder of the RoBERTa pre-trained model converts the complex semantic features output by the Transformer encoder into an interpretable probability distribution. The Transformer decoder includes three different linear transformations and a Sigmoid activation function. The Sigmoid function is used as the activation function of the output layer neurons for binary classification prediction, and the output vector is mapped to the range [0, 1];

[0029] S26: Optimize the RoBERTa pre-trained model by using the binary cross-entropy function as the Loss function. The Loss function is as follows: where n is the number of layers of the Transformer encoder, i is any layer of the Transformer encoder, y is the binary label value 0 or 1, and p(y) refers to the probability of belonging to the y label;

[0030] S3: Synthesize the outputs of multiple large models through ensemble learning. Multiple large models include the RoBERTa pre-trained model. Weight-average the outputs of each large model to obtain the email prediction score, and make a decision to deliver or cancel the delivery of the current email based on the prediction score.

[0031] Optionally, in the marketing email delivery method, the following steps are further included:

[0032] S4: Collect the received sending results and the log information printed by the email service middleware, and associate them with the delivery detail data of each email attempted to be delivered, and then deduce the delivery result of each email attempted to be delivered, and perform re-delivery according to the deduced delivery result and the re-delivery rule set.

[0033] Optionally, in the marketing email delivery method, the following steps are further included:

[0034] S5: Provide a clear unsubscribe link at the bottom of the email and regularly collect user feedback through questionnaires and feedback forms.

[0035] Compared with the prior art, the present invention has the following advantages:

[0036] (1) The personalized email content generation and multi-modal email content detection are realized through the email content generation module, reducing the costs of manually writing the email template content and manually reviewing the email content.

[0037] (2) The email delivery module and the email prediction module are adopted to predict the email content elements before delivery, and make a decision on whether to deliver or cancel the delivery of the current email according to the prediction score, thereby improving the overall sending success rate of email delivery.

[0038] (3) The delivery results are obtained by collecting the email receipts of different Internet email operators through the acquisition and analysis module, and the email features are generated and analyzed for the emails with low sending success rate and being intercepted in large quantities, realizing the continuous optimization of the email content and delivery success rate and the effective supplement of the local spam feature library;

[0039] The email delivery strategy is associated with the delivery situation, and the delivery strategy is dynamically configured and continuously optimized, avoiding the interception strategy of the email service provider being triggered by a single strategy and resulting in sending failure, and improving the overall delivery success rate.

[0040] (4) The customer behavior processing mode after email delivery is optimized through the customer feedback module, providing a clear unsubscribe process and questionnaire survey management process, reducing the costs of customer opinion surveys, the email customer blacklisting rate and the overall customer complaint rate, and improving the customer experience.

[0041] (5) Reducing the development and management costs of relevant organizations and institutions in delivering emails to customers, optimizing the email delivery strategy and customer feedback processing mode, improving the email delivery success rate and the business conversion rate of marketing activities, and improving the email marketing effect and the customer experience of email services.

[0042] (6) Presenting the email sending result statistical report and delivery details query, and optimizing the business marketing effect and system operation and maintenance capabilities. Description of the Drawings

[0043] Figure 1 It is a module diagram of the marketing email delivery system provided by the embodiment of the present invention;

[0044] Figure 2 It is a flowchart of email delivery and prediction provided by the embodiment of the present invention;

[0045] Figure 3Schematic diagram of the RoBERTa pre-trained model provided by the embodiments of the present invention;

[0046] Figure 4 Schematic diagram of the prediction integrating multiple large models provided by the embodiments of the present invention;

[0047] Figure 5 Schematic diagram of the email display effect provided by the embodiments of the present invention. Detailed implementation manners

[0048] The following will describe the detailed implementation manners of the present invention in more detail with reference to the schematic diagrams. According to the following description, the advantages and features of the present invention will be clearer. It should be noted that the drawings are all in very simplified forms and use non-precise scales, only for the purpose of facilitating and clearly assisting in explaining the objectives of the embodiments of the present invention.

[0049] In the following text, if the methods described herein include a series of steps, the order of these steps presented herein is not necessarily the only order in which these steps can be executed, and some of the described steps can be omitted and / or some other steps not described herein can be added to the method.

[0050] Currently, there are some drawbacks in the existing email delivery methods in the banking industry. First, there is a lack of personalized email content. That is, most of the email content is manually written and formulated by the institution internally, unable to adapt to the rapidly changing market environment and customized customer needs. Once customers are not interested in the email content, the sender's email box of the institution is easily blocked by customers, resulting in the inability to deliver emails to relevant customers subsequently. Second, the delivery strategies and email processing modes are single and fixed. The delivered emails are likely to trigger the security or risk control strategies of email service operators, resulting in a large number of interceptions or delivery failures of emails to customers in a short period of time. At the same time, there are many email attachment formats and the email content lacks regularized verification. Some email features are easily detected as spam, presenting compliance risks.

[0051] To solve the problems existing in the prior art, the present invention provides a marketing email delivery system, as Figure 1 shown, including:

[0052] To achieve a personalized customer experience with different content for different people and improve the email marketing effect and business efficiency, after completing email variable configuration, picture configuration, and template configuration through a front-end configuration interface developed by Vue or React, for the content that needs to be customized and written, such as holiday greetings, event introduction words, product recommendation copywriting, etc., corresponding copywriting content can be generated with the help of a natural language large model trained by artificial intelligence technology.

[0053] The email content generation module is configured to load the email template and the corresponding content of the Prompt from the database as the input for the large natural language model, and perform semantic splicing based on the context where the email template variables are located and the corresponding content of the Prompt. The large natural language model outputs the corresponding text content (i.e., AI content generation), and the generated text content fills the placeholders and variables in the email template (i.e., email variable filling);

[0054] Among them, the Prompt includes the email subject, customer group information, demand clues, and keywords. The email subject and customer group information are required fields, and the two fields of demand clues and keywords are optional fields. The email subject can be the email subject of emails such as birthday greeting cards, credit card statements, credit card application promotion emails, wealth management product recommendation emails, etc. The customer group information can be the customer group label information obtained by dividing customers according to the ECIF (Enterprise Customer Information Facility) system, such as the customer group for pension disbursement, the customer group for newly opened debit cards, the customer group for holding specified wealth management products, etc. The demand clues can be the demand information identified from data such as user preferences and historical interaction behaviors under the corresponding customer group, such as potential housing purchase and renovation loan demands identified based on the behavior data of searching for "mortgage rate" or "renovation loan rate" within the App or large changes in the balance of provident fund deposit accounts, and foreign exchange service demands identified from details of frequent cross-border consumption of foreign currency credit cards and records of foreign exchange settlement and sale transactions. The wealth management and asset allocation optimization demands can be identified from behaviors such as clicking to view money market funds and short-term wealth management funds and calling the customer service to consult low-risk preference wealth management products. The keywords can be keywords related to the email subject and content, such as the name of the recommended wealth management product, the name of the credit card application promotion, the activity time, and the activity requirements, etc.

[0055] Specifically, in the email template configuration interface, enter the email subject, customer group information, demand clues, keywords and other Prompt, associate the corresponding template ID, and bind the variable sequence to be replaced in the template, and then enter and persist it in the database. When the email template triggers a sending task, the large natural language model will, according to the predefined workflow, first load the email template content and the corresponding content of the Prompt from the database as the input for the large natural language model, and perform semantic splicing based on the context where the email template variables are located and the corresponding content of the Prompt, and output the corresponding text content. The generated content will replace the placeholders and variables in the original email template in the form of "${variable}". The interaction process between modules is as Figure 1 shown.

[0056] Preferably, the email content generation module is also configured to perform content review and rule review. The specific work of the email content generation module is as follows: The background management program organizes the email content results filled by the natural language large model, the sending program logs, and the template variables into a JSON string, and uses the email content value in the key-value pair as the input of the content review large model. The content review large model queries the review rules corresponding to the email template from the Prompt library, predicts the email interception probability, and stops delivering emails that exceed the set probability threshold. It also judges whether the email content meets the requirements of the regulatory agency, such as data privacy protection, common email format check, sensitive information review, misleading statement verification, to ensure that the email content complies with the format constraints and regulatory compliance requirements of the corresponding email template. After the content review large model passes the detection, it will also go through the rule library for rule verification to judge whether the email to be delivered meets the current delivery strategy, prevent frequent email sending from accidentally triggering the interception strategy of the email service provider, and improve the email delivery success rate.

[0057] After passing the rule library review, the email will be delivered to the email service provider through the Spring Boot program, and the details of the delivery process and the abnormal situation data will be persistently processed. At the same time, the email opening rate, activity link click-through rate, marketing conversion funnel data, etc. after email delivery will be collected in real time to quantify the impact of the Prompt prompt words and model parameters on the marketing reach effect. So that in the subsequent collection and analysis module, analysis can be carried out based on the email delivery data, and by comparing the new and old Prompt prompt words, continuous optimization can be carried out on the email template configuration interface, such as adjusting the keyword weight, context modifier, model input content, email template content, etc., to form a continuously optimized email delivery link, so as to achieve continuously improved delivery effects and customer experiences.

[0058] The email delivery module, as Figure 2 and 3 shown, includes a multi-modal processing layer, a tokenization layer, an embedding layer, a RoBERTa pre-trained model, and an optimization unit. The RoBERTa pre-trained model consists of multiple layers of bidirectional Transformer encoders and has been optimized during the pre-training stage to achieve better prediction effects.

[0059] Specifically, the original email is split into an email title, an email body, and email attachments. The multi-modal processing layer is configured to divide the multi-modal fusion input composed of the email title, the email body, and the email attachments into three separate inputs, and perform multi-modal processing on the email title, the email body, and the email attachment content. When processing image type files, OCR recognition and text extraction are performed through the pytesseract library, and the text data involved is extracted from image files such as png, jpg, jpeg type, etc. When processing PDF type files, PDF text information extraction is performed through the pdfminer library. When processing Excel type files, it is processed through the pandas library. When processing Word type files, it is processed through the python-doc and python-docx libraries. After the program processing, the email title text, the email body, and the email attachments are respectively extracted from the email to be detected as the input of the tokenization layer.

[0060] The tokenization layer is configured to split all the content into multiple language units (such as splitting a complete sentence into words or sub-words), and process multiple language units and various special characters, converting all the content of the original email into a vector sequence. For example, if the text content A to be detected extracted from the email content consists of n words, the RoBERTa pre-trained model can convert the email text A into tokens through the tokenization layer. i The vector sequence, and its formula is as follows: RoBERTa(A)={token1,…,token i ,…token n}

[0061] The embedding layer is configured to map the vector sequence into a dense vector of a fixed dimension. For example, a word in the token vector sequence may be converted into a semantically rich 768-dimensional vector, enabling the RoBERTa pre-trained model to capture the semantic relationships between words, and thus better understand the email sending intention. For example, "balance" and "statement" may be closer in the vector space because they often appear in the same email content. The dense vector output by the embedding layer will be used as the input of the Transformer encoder after position encoding.

[0062] Reference Figure 3 , the structural schematic diagram of the RoBERTa pre-trained model, where E N and X NThey are the input vector and the output vector respectively. T is the Transformer encoder, C is the classification label output by the RoBERTa pre-trained model, and CLS is the input information aggregation vector. The Transformer encoder of the RoBERTa pre-trained model is configured to extract the input content by using the multi-head attention mechanism, so as to achieve parallel attention to different subspaces of the input sequence to improve the computing efficiency; after fusing the context semantics, a residual connection is performed to retain the basic features of the input, so that in the case of poor learning effect of the sublayer, a large amount of key information can be avoided from being lost; the result of the residual connection is normalized to ensure the stability and convergence speed during the training process of the RoBERTa pre-trained model, so that the RoBERTa pre-trained model can also maintain good performance in the case of small samples, and the hidden layer is normalized to the standard normal distribution; the input content after normalization is input into the feed-forward neural network (Feed Forward Network, that is, FFN) to prevent local details that may be ignored by the attention mechanism, and the output of the FFN may change due to parameter updates. Therefore, the output result after being processed by the feed-forward neural network is mapped to the standard normal distribution through secondary normalization to avoid the decoder from decreasing stability due to input scale differences;

[0063] The Transformer decoder of the RoBERTa pre-trained model is configured to transform the complex semantic features output by the Transformer encoder into an interpretable probability distribution to adapt to the task requirements of the downstream. The Transformer decoder includes three different linear transformations and the Sigmoid activation function. After three linear transformations, the feature dimension is gradually compressed, and at the same time, the weight distribution of the three layers can be adjusted according to different contents to enhance the fine control and flexibility of the information flow and improve the accuracy of the final decision result.

[0064] Preferably, the RoBERTa pre-trained model uses the Sigmoid function as the activation function of the output layer neurons for binary classification prediction, and maps the output vector to the range of [0,1], indicating the probability that the email is detected as spam.

[0065] Furthermore, the optimization unit is configured to optimize the RoBERTa pre-trained model by using the binary cross-entropy function as the Loss loss function. The Loss loss function is as follows:

[0066] Among them, n is the number of layers of the Transformer encoder, i is any layer of the Transformer encoder, y is the binary label value 0 or 1, and p(y) refers to the probability of belonging to the y label. Since in the email interception prediction scenario, the positive example is the intercepted email and the negative example is the un-intercepted email, binary cross-entropy can be used to compare the probability after Sigmoid activation with the true label, measure the gap between the probability value of the prediction result of the RoBERTa pre-trained model and the true value, calculate the loss, and then adjust the parameters of the RoBERTa pre-trained model through backpropagation to improve the prediction accuracy, so as to improve the classification accuracy and training effect of the RoBERTa pre-trained model.

[0067] As Figure 4 shown, the email prediction module is configured to synthesize the outputs of multiple large models (such as combining the DeepSeek-V3 large model deployed privately, the Llama-3 large model, the Qwen2.5-VL large model, etc.) through the way of ensemble learning. Among the multiple large models, there is the RoBERTa pre-trained model. The outputs of each large model are weighted and averaged to obtain the email prediction score, and a decision to deliver or cancel the delivery of the current email is made based on the prediction score. In one embodiment, when a total of N available large models are deployed within the organization, the weight of each large model is denoted as K i , and each large model makes a prediction on the current email to obtain the result P i (A), and the final expected result E RES is obtained after weighting and averaging the prediction results. The formula is as follows:

[0068] For example: when the prediction score ≥ 95%, it is classified as spam and the delivery is cancelled to prevent the normal email sending from failing due to repeatedly triggering the interception rules of the email operation service provider. When the prediction score < 95%, after the normal email is delivered, the final result of the email delivery is obtained by collecting the status report returned by the email operation service provider, and it is compared with the prediction result of the email prediction module, so as to continuously optimize the prediction accuracy of this delivery system.

[0069] For the emails to be sent after being predicted by each large model, record the email feature information, which is convenient for subsequent collection of delivery detail data, and summarize it into a statistical report corresponding to the email template for data analysis. When the sending success rate of the corresponding email template is relatively low, extract the corresponding email features to expand the local spam feature library, which is one of the supplementary methods for the recipient risk control rules.

[0070] Preferably, when the email prediction module makes a prediction, it queries the email template ID to be predicted from the database, and obtains the list of detectable large models associated with this email template ID (such as RoBERTa pre-trained model, DeepSeek-V3 large model combined with private deployment, Llama-3 large model, Qwen2.5-VL large model, etc.) and the corresponding Prompt prompt words. The Prompt prompt words and the email content to be detected are used as the inputs of each large model, and each large model makes a prediction. After multiple large models make predictions, the prediction results are comprehensively calculated to improve the accuracy and reliability of the detection.

[0071] During the process of email delivery, since customer email boxes often come from different email operation service providers, and the versions of email server and the types of middleware used by different service providers are different, the sending result receipts and log information received by the sender after sending emails are different, resulting in the inability to timely know the email sending status. Therefore, the delivery system further includes a collection and analysis module, configured to collect according to the received sending results and the log information printed by the email service middleware, and associate it with the delivery detail data of each email attempted to be delivered, and then deduce the delivery result of each email attempted to be delivered. According to the deduced delivery result and the reissuing rule set, reissuing is performed. For example, if the deduced delivery result is delivery failure and the email meets the reissuing rules, reissuing is performed; if it does not meet the reissuing rules, reissuing is not performed. The present invention also realizes a dynamic delivery strategy. When a large number of emails are found to trigger a delivery failure scenario during the regular monitoring task, the interception information is first recorded and a warning notice is sent to the system administrator, and then the delivery strategy is automatically adjusted according to the configured rules, so as to reduce the subsequent email delivery failure risk, and the delivery strategy can be continuously optimized according to the subsequent email transaction situation with the operator.

[0072] Optionally, in the marketing email delivery system, the delivery system further includes a customer feedback module, configured to provide a clear unsubscribe link at the bottom of the email, and regularly collect user feedback in the form of questionnaires and feedback forms. With the help of data such as the email read rate, user interest preferences and true intentions are investigated in detail, the feedback of users on the email content and frequency is understood, and corresponding adjustments are made. To simplify customer service operations, ensure that the email content remains intuitive and clear, the unsubscribe operation and process are simple and easy to understand, so that users can quickly complete operations such as email content browsing or unsubscribing, and analyze the unsubscribe data, questionnaire and feedback form results, so as to reduce the complaint rate and improve the customer experience. Finally, the email display effect is as Figure 5 shown, Figure 5 including examples of email layout and sample effects.

[0073] Based on the customer group information generated from dimensions such as the consumption behavior, interest preferences, and interaction history of target customers, combined with the demand clue data and email marketing keywords of the corresponding customer groups in the CRM (Customer Relationship Management) system as the input of the natural language large model, under the structured content framework, the natural language large model outputs personalized email dynamic content such as product introduction words, marketing activity recommendation words, and multimedia content elements. And improvements are made in multiple aspects such as configuration management before email delivery, feature detection and content optimization during delivery, status collection and customer feedback after delivery, and analysis of observable indicators, to improve the overall sending efficiency, delivery success rate, and marketing conversion rate of customer emails, improve the customer experience, and increase customer satisfaction.

[0074] The present invention also provides a method for delivering marketing emails, which uses the described delivery system. The method includes the following steps:

[0075] S1: Load the email template and the corresponding content of the Prompt prompt word from the database as the input of the natural language large model, and perform semantic splicing according to the context of the position where the email template variable is located and the corresponding content of the Prompt prompt word. The natural language large model outputs the corresponding text content (i.e., AI content generation), and the generated text content fills the placeholders and variables in the email template (i.e., email variable filling);

[0076] Among them, the Prompt includes the email subject, customer group information, demand clues, and keywords. The email subject and customer group information are required fields, while the demand clues and keywords are optional fields. The email subject can be the subject of emails such as birthday greeting card emails, credit card statement emails, credit card application promotion emails, wealth management product recommendation emails, etc. The customer group information can be customer group label information for customer segmentation generated according to the ECIF (Enterprise Customer Information Facility) system, such as the customer group for pension disbursement, the customer group for newly opened debit cards, the customer group for holding designated wealth management products, etc. The demand clues can be demand information identified from data such as user preferences and historical interaction behaviors under the corresponding customer group, such as potential housing purchase and renovation loan demands identified based on behavior data such as searching for "mortgage rate" or "renovation loan rate" within the App, large changes in the balance of provident fund deposit accounts, etc., foreign exchange service demands identified from details of frequent cross-border consumption of foreign currency credit cards, record of foreign exchange settlement and sale transactions, etc., and wealth management and asset allocation optimization demands identified from behaviors such as clicking to view money market funds and short-term wealth management funds, calling the customer service to consult low-risk preference wealth management products. The keywords can be keywords related to the email subject and email content, such as the name of the recommended wealth management product, the name of the credit card application promotion, the activity time and requirements, etc.

[0077] Specifically, in the email template configuration interface, enter Prompt such as the email subject, customer group information, demand clues, keywords, etc., associate the corresponding template ID, and bind the variable sequence required to be replaced in the template, and then enter and persist it in the database. When this email template triggers a sending task, the natural language large model will, according to the pre-defined workflow process, first load the email template content and the corresponding content of the Prompt from the database as the input of the natural language large model, and perform semantic splicing according to the context where the email template variables are located and the corresponding content of the Prompt, and output the corresponding text content. The generated content will replace the placeholders and variables in the original email template in the form of "${variable}". The interaction process between modules is as Figure 1 shown.

[0078] Preferably, the email content generation module is also configured to perform content review and rule review. The specific work of the email content generation module is as follows: The background management program collates the email content result filled by the natural language large model, the sending program log, and the template variables into a JSON string, and uses the email content value in the key-value pair as the input of the content review large model. The content review large model queries the review rules corresponding to the email template from the Prompt library, predicts the email interception probability, and stops delivering emails that exceed the set probability threshold. It also judges whether the email content meets the requirements of the regulatory agency, such as data privacy protection, common email format check, sensitive information review, and misleading statement verification, to ensure that the email content complies with the format constraints and regulatory compliance requirements of the corresponding email template. After the content review large model passes the detection, it will also undergo rule verification through the rule library to judge whether the email to be delivered meets the current delivery strategy, prevent frequent email sending from accidentally triggering the interception strategy of the email service provider, and improve the email delivery success rate.

[0079] After passing the rule library review, the email will be delivered to the email service provider through the Spring Boot program, and the details of the delivery process and abnormal situation data will be persistently processed. At the same time, the email opening rate, activity link click-through rate, marketing conversion funnel data, etc. after email delivery will be collected in real time to quantify the impact of the Prompt prompt words and model parameters on the marketing reach effect. This enables subsequent analysis based on the email delivery data in the collection and analysis module, and through comparison of the old and new Prompt prompt words, continuous optimization can be carried out on the email template configuration interface, such as adjusting keyword weights, context modifiers, model input content, email template content, etc., to form a continuously optimized email delivery link, thereby achieving continuously improved delivery effects and customer experiences.

[0080] As Figure 2 shown,

[0081] S21: Split the original email into an email title, email body, and email attachments. Divide the multi-modal fusion input composed of the email title, email body, and email attachments into three separate inputs, and perform multi-modal processing on the email title, email body, and email attachment content. When processing image type files, perform OCR recognition and text extraction through the pytesseract library, and extract the text data involved from image files such as png, jpg, jpeg types, etc. When processing PDF type files, perform PDF text information extraction through the pdfminer library. When processing Excel type files, process them through the pandas library. When processing Word type files, process them through the python-doc and python-docx libraries. After program processing, extract the email title text, email body, and email attachments from the email to be detected as the input of the tokenization layer;

[0082] S22: Split all content into multiple language units (e.g., split a complete sentence into words or sub - words), and process multiple language units and various special characters to convert all content of the original email into a vector sequence. For example, if the text content A to be detected extracted from the email content consists of n words, the RoBERTa pre - trained model can convert the email text A into tokens through the tokenization layer i vector sequence, and its formula is as follows: RoBERTa(A) = {token1, …, token i , … token n};

[0083] S23: Map the vector sequence to a dense vector of a fixed dimension. For example, a word in the token vector sequence may be converted into a semantically rich 768 - dimensional vector, enabling the RoBERTa pre - trained model to capture the semantic relationships between words and thus better understand the email sending intention. For example, "balance" and "statement" may be closer in the vector space because they often appear in the same email content. The dense vector output by the embedding layer will be used as the input to the Transformer encoder after position encoding;

[0084] S24: Refer to Figure 3 , the structural schematic diagram of the RoBERTa pre - trained model, where E N and X N are the input vector and output vector respectively, T is the Transformer encoder, C is the classification label output by the RoBERTa pre - trained model, and CLS is the input information aggregation vector. The Transformer encoder of the RoBERTa pre - trained model uses the multi - head attention mechanism to extract the input content, thereby achieving parallel attention to different sub - spaces of the input sequence to improve the computational efficiency; after fusing the context semantics, a residual connection is made to retain the basic features of the input, so that in the case of poor sub - layer learning effect, a large amount of key information is not lost; the result of the residual connection is normalized to ensure the stability and convergence speed during the training process of the RoBERTa pre - trained model, enabling the RoBERTa pre - trained model to maintain good performance even in the case of small samples, and normalizing the hidden layer to a standard normal distribution; the normalized input content is input into the feed - forward neural network (Feed Forward Network, i.e., FFN) to prevent local details that may be ignored by the attention mechanism, and the output of the FFN may change numerically due to parameter updates. Therefore, the output result after being processed by the feed - forward neural network is mapped to a standard normal distribution through secondary normalization to avoid the decoder's stability degradation caused by input scale differences;

[0085] S25: The Transformer decoder of the RoBERTa pre-trained model converts the complex semantic features output by the Transformer encoder into an interpretable probability distribution to adapt to the requirements of downstream tasks. The Transformer decoder includes three different linear transformations and a Sigmoid activation function. After three linear transformations, the feature dimension is gradually compressed. At the same time, the weight distribution of the three layers can be adjusted according to different contents, enhancing the fine control and flexibility of the information flow and improving the accuracy of the final decision result.

[0086] Preferably, the RoBERTa pre-trained model uses the Sigmoid function as the activation function of the output layer neurons for binary classification prediction, mapping the output vector to the range of [0, 1], which represents the probability that the email is detected as spam.

[0087] S26: The RoBERTa pre-trained model is optimized by using the binary cross-entropy function as the Loss function. The Loss function is as follows:

[0088] Among them, n is the number of layers of the Transformer encoder, i is any layer of the Transformer encoder, y is the binary label value 0 or 1, and p(y) refers to the probability belonging to the y label. Because in the email interception prediction scenario, the positive example is the intercepted email, and the negative example is the un-intercepted email. Using binary cross-entropy can compare the probability after Sigmoid activation with the true label, measure the gap between the probability value of the prediction result of the RoBERTa pre-trained model and the true value, and adjust the parameters of the RoBERTa pre-trained model through backpropagation after calculating the loss to improve the prediction accuracy and enhance the classification accuracy and training effect of the RoBERTa pre-trained model.

[0089] S3: By means of ensemble learning, the outputs of multiple large models (such as combining the DeepSeek-V3 large model, Llama-3 large model, Qwen2.5-VL large model, etc. deployed privately) are integrated. The multiple large models include the RoBERTa pre-trained model. The outputs of each large model are weighted and averaged to obtain the email prediction score, and a decision to deliver or cancel the delivery of the current email is made based on the prediction score. In one embodiment, when a total of N available large models are deployed within the organization, the weight of each large model is denoted as K i , and each large model makes a prediction on the current email to obtain the result P i (A), and the final expected result E is obtained after weighted averaging of the prediction results RES , and its formula is as follows: For example: when the prediction score ≥ 95%, it is classified as spam and the delivery is cancelled to prevent the normal email sending from failing due to repeated triggering of the email operation service provider's interception rules. When the prediction score < 95%, after the normal email delivery, the status report returned by the email operation service provider is collected to obtain the final result of the email delivery, and it is compared with the prediction result of the email prediction module, so as to continuously optimize the prediction accuracy of this delivery system.

[0090] For the emails to be sent after being predicted by various models, record the email feature information to facilitate the subsequent collection of delivery detail data, which is summarized into a statistical report corresponding to the email template for data analysis. When the sending success rate of the corresponding email template is relatively low, extract the corresponding email features to expand the local spam email feature library, which is one of the supplementary methods for the recipient risk control rules.

[0091] Preferably, when the email prediction module makes a prediction, query the ID of the email template to be predicted from the database, and obtain the list of detectable large models associated with this email template ID (such as RoBERTa pre-trained model, DeepSeek-V3 large model combined with private deployment, LlaMA-3 large model, Qwen2.5-VL large model, etc.) and the corresponding Prompt prompt words. Take the Prompt prompt words and the email content to be detected as the input of each large model, and each large model makes a prediction. After multiple large models make predictions, comprehensively calculate the prediction results to improve the accuracy and reliability of the detection.

[0092] S4: Collect according to the received sending results and the log information printed by the email service middleware, and associate it with the delivery detail data of each email attempted to be delivered, and then calculate the delivery result of each email attempted to be delivered. According to the calculated delivery result and the reissuance rule set, reissue. For example, if the calculated delivery result is delivery failure and the email meets the reissuance rules, reissue; if it does not meet the reissuance rules, do not reissue. The present invention also realizes a dynamic delivery strategy. When the timing monitoring task discovers a large number of emails triggering a delivery failure scenario, first record the interception information and send a warning notice to the system administrator, and then automatically adjust the delivery strategy according to the configured rules, so as to reduce the subsequent risk of email delivery failure, and can continuously optimize the delivery strategy according to the subsequent email transaction situation with the operator.

[0093] S5: Provide a clear unsubscribe link at the bottom of the email, and regularly collect user feedback in the form of questionnaires and feedback forms. Conduct a detailed investigation of user interest preferences and true intentions by means of data such as email read rates, understand user feedback on email content and frequency, and make corresponding adjustments. To simplify customer service operations, ensure that the email content remains intuitive and clear, the unsubscribe operation and process are simple and easy to understand, so that users can quickly complete operations such as browsing email content or canceling subscriptions, analyze the unsubscribe data, questionnaire and feedback form results, thereby reducing the complaint rate and enhancing the customer experience. Finally, the email display effect is as Figure 5 shown, Figure 5 including an example of the email layout and sample effect.

[0094] In summary, compared with the prior art, the present invention has the following advantages:

[0095] (1) The personalized email content generation and multi-modal email content detection are realized through the email content generation module, reducing the costs of manual writing of email template content and manual review of email content.

[0096] (2) The email delivery module and the email prediction module are adopted to predict the email content elements before delivery, and make a decision on delivering or canceling the delivery of the current email according to the prediction score, thereby improving the overall sending success rate of email delivery.

[0097] (3) The delivery results are obtained by collecting and analyzing email receipts of different Internet email operators through the collection and analysis module, analyzing the email characteristics of emails with low sending success rate and being intercepted in large numbers, realizing the continuous optimization of email content and delivery success rate and the effective supplement of the local spam feature library;

[0098] Associate the email delivery strategy with the delivery situation, dynamically configure and continuously optimize the delivery strategy, avoid a single strategy from triggering the interception strategy of the email service provider and resulting in sending failure, and improve the overall delivery success rate.

[0099] (4) The customer behavior processing mode after email delivery is optimized through the customer feedback module, providing a clear unsubscribe process and questionnaire survey management process, reducing the costs of customer opinion surveys, the email customer blacklisting rate and the overall customer complaint rate, and enhancing the customer experience.

[0100] (5) It reduces the development and management costs of relevant organizations and institutions in delivering emails to customers, optimizes the email delivery strategy and customer feedback processing mode, improves the email delivery success rate and the business conversion rate of marketing activities, and improves the email marketing effect and the customer experience of email services.

[0101] (6) Presents the statistical report of email sending results and the query of delivery details, optimizing the business marketing effect and system operation and maintenance capabilities.

[0102] The above are only the preferred embodiments of the present invention and do not impose any limitation on the present invention. Any person skilled in the art within the technical field, without departing from the technical solution of the present invention, makes any form of equivalent substitution or modification and other changes to the technical solution and technical content disclosed by the present invention, all of which belong to the content within the technical solution of the present invention and are still within the protection scope of the present invention.

Claims

1. A marketing email delivery system, characterized in that, Including: A mail content generation module, configured to load the mail template and the corresponding content of the Prompt from the database as the input of the natural language large model, and perform semantic splicing according to the context where the mail template variables are located and the corresponding content of the Prompt. The natural language large model outputs the corresponding text content, and the generated text content fills the placeholders and variables in the mail template; A mail delivery module, including a multi-modal processing layer, a tokenization layer, an embedding layer, a RoBERTa pre-trained model, and an optimization unit; the multi-modal processing layer is configured to divide the multi-modal fusion input composed of the mail title, the mail body, and the mail attachment into three separate inputs; the tokenization layer is configured to split all the mail content to obtain multiple language units, and process multiple language units and various special characters to convert all the original mail content into a vector sequence; the embedding layer is configured to map the vector sequence into a dense vector with a fixed dimension, and the dense vector output by the embedding layer will be used as the input of the Transformer encoder after position encoding; the RoBERTa pre-trained model consists of multiple layers of bidirectional Transformer encoders. The RoBERTa pre-trained model uses the Sigmoid function as the activation function of the output layer neurons for binary classification prediction, and maps the output vector to the range of [0,1]; the Transformer encoder of the RoBERTa pre-trained model is configured to use the multi-head attention mechanism to extract the input content; after fusing the context semantics, perform residual connection to retain the basic features of the input; Perform normalization processing on the result of the residual connection to ensure the stability and convergence speed during the training process of the RoBERTa pre-trained model; The input content after normalization processing is input into the feed-forward neural network; Map the output result processed by the feed-forward neural network to a standard normal distribution through secondary normalization; the Transformer decoder of the RoBERTa pre-trained model is configured to convert the complex semantic features output by the Transformer encoder into an interpretable probability distribution. The Transformer decoder includes three different linear transformations and a Sigmoid activation function; the optimization unit is configured to optimize the RoBERTa pre-trained model by using the binary cross-entropy function as the Loss function. The Loss function is as follows: Among them, n is the number of layers of the Transformer encoder, i is any layer of the Transformer encoder, y is the binary label value 0 or 1, and p(y) refers to the probability of belonging to the y label; A mail prediction module, configured to comprehensively integrate the outputs of multiple large models through ensemble learning. Among the multiple large models, there is a RoBERTa pre-trained model. The outputs of each large model are weighted and averaged to obtain a mail prediction score, and a decision to deliver or cancel the delivery of the current mail is made based on the prediction score.

2. The marketing email delivery system according to claim 1, wherein The mail content generation module is also configured to perform content review and rule review.

3. The marketing email delivery system according to claim 1, wherein When the email prediction module makes a prediction, it queries the email template ID to be predicted from the database, and obtains the list of detectable large models and the corresponding Prompt prompts associated with this email template ID. The Prompt prompts and the content of the email to be detected are used as the input for each large model, and each large model makes a prediction.

4. The marketing email delivery system according to claim 1, characterized in that The delivery system further includes a collection and analysis module, configured to collect based on the received sending results and the log information printed by the email service middleware, and associate it with the delivery detail data of each email attempted to be delivered, and then deduce the delivery results of each email attempted to be delivered, and reissue according to the deduced delivery results and the reissue rule set.

5. The marketing email delivery system according to claim 4, wherein According to the deduced delivery results, and by comparing the old and new Prompt prompts, optimize the configuration interface of the email template.

6. The marketing email delivery system according to claim 1, wherein The delivery system further includes a customer feedback module, configured to provide a clear unsubscribe link at the bottom of the email, and regularly collect user feedback in the form of questionnaires and feedback forms.

7. A method for delivering marketing emails, characterized in that, Adopt the delivery system according to any one of claims 1-6, the method includes the following steps: S1: Load the corresponding content of the email template and the Prompt prompt from the database as the input of the natural language large model, and perform semantic splicing according to the context of the position of the email template variable and the corresponding content of the Prompt prompt. The natural language large model outputs the corresponding text content, and the generated text content fills the placeholders and variables in the email template; S21: Divide the multi-modal fusion input composed of the email title, email body, and email attachment into three separate inputs; S22: Split all the content to obtain multiple language units, process the multiple language units and various special characters, and convert all the content of the original email into a vector sequence; S23: Map the vector sequence to a dense vector with a fixed dimension, and the dense vector output by the embedding layer will be used as the input of the Transformer encoder after position encoding; S24: The Transformer encoder of the RoBERTa pre-trained model uses the multi-head attention mechanism to extract the input content; after fusing the context semantics, perform a residual connection to retain the basic features of the input; normalize the result of the residual connection to ensure the stability and convergence speed during the training process of the RoBERTa pre-trained model; the normalized input content is input into the feed-forward neural network; the output result processed by the feed-forward neural network is mapped to a standard normal distribution through secondary normalization; S25: The Transformer decoder of the RoBERTa pre-trained model converts the complex semantic features output by the Transformer encoder into an interpretable probability distribution. The Transformer decoder includes three different linear transformations and a Sigmoid activation function, and uses the Sigmoid function as the activation function of the output layer neurons for binary classification prediction, and maps the output vector to the range of [0,1]; S26: Optimize the RoBERTa pre-trained model by using the binary cross-entropy function as the Loss function. The Loss function is as follows: Among them, n is the number of layers of the Transformer encoder, i is any layer of the Transformer encoder, y is the binary label value 0 or 1, and p(y) refers to the probability of belonging to the y label; S3: Integrate the outputs of multiple large models through ensemble learning. Among the multiple large models, there is the RoBERTa pre-trained model. Perform weighted averaging on the outputs of each large model to obtain the email prediction score, and make a decision to deliver or cancel the delivery of the current email based on the prediction score.

8. The marketing email delivery method according to claim 7, characterized in that, It also includes the following steps: S4: Collect the received sending results and the log information printed by the email service middleware, and associate them with the delivery details data of each email attempted to be delivered. Then, deduce the delivery result of each email attempted to be delivered, and perform reissuance based on the deduced delivery result and the reissuance rule set.

9. The marketing email delivery method according to claim 7, wherein It also includes the following steps: S5: Provide a clear unsubscribe link at the bottom of the email, and regularly collect user feedback in the form of questionnaires and feedback forms.

Images