Email processing methods and systems, storage media, and software products

Abstract

This application provides an email processing method and system, storage medium, and program product. The method includes: extracting multimodal features from a target email to obtain multimodal feature information corresponding to the target email; performing cross-modal interaction based on the multimodal feature information to generate cross-modal interaction information for the target email; classifying intents based on the cross-modal interaction information to obtain the intent type of the target email and its corresponding confidence level; assessing the risk of the target email under the intent type based on the intent type and its corresponding confidence level, combined with historical email information related to the target email, to obtain a risk score for the target email; and determining a risk label for the target email based on the risk score. The risk label is used to characterize the degree of risk of the target email under the intent type. This application improves the intelligence and security of the email processing process.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and data processing technology, and in particular to email processing methods and systems, storage media, and program products. Background Technology

[0002] With the rapid development of information and internet technologies, email has become an important tool for internal collaboration, external business communication, and information transmission within enterprises. While carrying daily business information, email has also gradually become a significant carrier of various cyberattacks, information leaks, and fraudulent activities, posing a threat to enterprise information security and business operations. However, current email processing solutions struggle to simultaneously address multimodal content understanding, accurate intent recognition, and reasonable risk assessment, resulting in low levels of intelligence and security.

[0003] Based on this, embodiments of this application provide email processing methods and systems, storage media, and program products to improve related technologies. Summary of the Invention

[0004] The purpose of this application is to provide email processing methods and systems, storage media, and program products, to improve the problem that related email processing solutions cannot simultaneously take into account multimodal content understanding, accuracy of intent recognition, and rationality of risk assessment, thereby improving the intelligence level and security of the email processing process.

[0005] In a first aspect, embodiments of this application provide an email processing method, comprising: performing multimodal feature extraction based on a target email to obtain multimodal feature information corresponding to the target email; the target email includes at least two of text information, image information, attachment information, and link information, and the multimodal feature information includes at least two of text feature information, image feature information, attachment feature information, and link feature information; performing cross-modal interaction based on the multimodal feature information to generate cross-modal interaction information of the target email; classifying intent based on the cross-modal interaction information to obtain the intent type of the target email and its corresponding confidence level; the intent type includes at least one of business communication, financial transfer request, data leakage, phishing fraud, malware propagation, and social engineering attack; assessing the risk of the target email under the intent type based on the intent type and its corresponding confidence level, combined with historical email information related to the target email, to obtain a risk score for the target email; determining a risk label for the target email based on the risk score; the risk label is used to characterize the degree of risk of the target email under the intent type.

[0006] In some embodiments, the multimodal feature extraction based on the target email includes: performing multimodal content extraction on the target email to obtain at least two of the following: text content, image content, attachment content, and link content corresponding to the target email; and performing feature encoding on at least two of the following: text content, image content, attachment content, and link content, to obtain at least two of the following: text feature information, image feature information, attachment feature information, and link feature information.

[0007] In some embodiments, the cross-modal interaction based on the multimodal feature information includes: performing a correlation assessment between feature information of different modalities in the multimodal feature information to obtain corresponding correlation assessment information; and performing feature fusion on feature information of different modalities in the multimodal feature information based on the correlation assessment information to generate cross-modal interaction information of the target email.

[0008] In some embodiments, the correlation assessment between feature information of different modalities in the multimodal feature information includes: using an attention model to assess the correlation between feature information of different modalities in the multimodal feature information, and calculating a correlation weight matrix between different modal features; the correlation weight matrix serves as the correlation assessment information.

[0009] In some embodiments, determining the risk label of the target email based on the risk score includes: determining the risk label of the target email based on the risk score and a plurality of preset risk score thresholds; the plurality of risk score thresholds are used to divide the value range of the risk score into a plurality of numerical intervals.

[0010] In some embodiments, the risk label includes at least one of high risk, medium risk, and low risk. The method further includes: determining a processing strategy for the target email based on at least one of the target email's intent type and its corresponding confidence level, risk score, and risk label using a processing strategy model; the processing strategy includes at least one of interception, isolation, marking, encryption, deletion, forwarding, and notification; receiving feedback information regarding the processing strategy, and forming training data for training the processing strategy model based on the feedback information and at least one of the target email's intent type and its corresponding confidence level, risk score, and risk label.

[0011] In some embodiments, the text information includes at least one of email subject, email body, email header fields, and email signature, wherein the email header fields include at least one of sender address, recipient address, sending time, and CC information; and / or, the attachment information includes at least one of attachment file type, file size, and file content feature information, wherein the file content feature information includes at least one of file header information and file hash information; and / or, the link information includes URL links contained in the email body and the domain name information pointed to by the URL links, wherein the domain name information includes at least one of domain name string length, registration time information, certificate information, path information, and historical reputation information; and / or, the historical email information includes at least one of sender historical email behavior information, recipient historical email interaction information, and email time sequence information related to the target email.

[0012] In some embodiments, assessing the risk of the target email under the intent type includes: extracting multiple risk factors based on the target email and historical email information related to the target email, and assigning a risk value to each risk factor; assessing the importance of each risk factor based on the intent type and its corresponding confidence level to determine the weight parameter of each risk factor; and weighting and fusing the risk values ​​of multiple risk factors according to their corresponding weight parameters to obtain a risk score for the target email.

[0013] In some embodiments, the process of determining the target email includes: upon detecting a preset trigger event for at least one email, determining each email as the target email; wherein the preset trigger event includes at least one of the following: a new email arrives; a user-initiated manual review request for an email is received; or a timed scanning task detects emails that are not empty.

[0014] Secondly, embodiments of this application provide an email processing system, comprising: a feature extraction module, configured to perform multimodal feature extraction based on a target email to obtain multimodal feature information corresponding to the target email; the target email includes at least two of text information, image information, attachment information, and link information, and the multimodal feature information includes at least two of text feature information, image feature information, attachment feature information, and link feature information; a multimodal fusion module, configured to perform cross-modal interaction based on the multimodal feature information to generate cross-modal interaction information of the target email; an intent recognition module, configured to classify intents based on the cross-modal interaction information to obtain the intent type of the target email and its corresponding confidence level; the intent type includes at least one of business communication, financial transfer request, data leakage, phishing fraud, malware propagation, and social engineering attack; a risk assessment module, configured to assess the risk of the target email under the intent type based on the intent type and its corresponding confidence level, combined with historical email information related to the target email, to obtain a risk score for the target email; and determine a risk label for the target email based on the risk score; the risk label is used to characterize the degree of risk of the target email under the intent type.

[0015] Thirdly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the above-described email processing methods.

[0016] Fourthly, embodiments of this application provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps of any of the above-described email processing methods.

[0017] This application provides an email processing method and system, storage medium, and program product. By extracting multimodal features from target emails and introducing a cross-modal interaction mechanism, it achieves joint modeling of various information such as email text, images, attachments, and links, thereby improving the problem of incomplete intent understanding caused by relying solely on single-modal information. By classifying intent based on cross-modal interaction information and outputting corresponding confidence scores, the business intent and risk intent of emails can be accurately identified within a unified intent space, thus improving the reliability of intent recognition results. Based on the intent classification results, it further combines historical email information to assess the risk of target emails under specific intent types, giving the risk assessment process clear semantic premises and contextual basis, thereby improving the problem of the disconnect between risk judgment and the true intent of the email. By determining risk labels based on risk scores, it achieves a graded expression of email risk levels, facilitating differentiated processing of subsequent security strategies. In summary, to address the challenge that existing email processing solutions struggle to simultaneously achieve multimodal content understanding, accurate intent recognition, and reasonable risk assessment, this embodiment employs a concept based on multimodal feature interaction for email intent recognition and risk assessment. This enables comprehensive perception of email content, accurate identification of email intent, and reasonable assessment of email risks, thereby improving the intelligence and security of the email processing process. Attached Figure Description

[0018] The embodiments of this application are further described below with reference to the accompanying drawings and specific implementation details.

[0019] Figure 1 This is a flowchart illustrating an email processing method provided in an embodiment of this application.

[0020] Figure 2 This is a structural block diagram of an email processing system provided in an embodiment of this application.

[0021] Figure 3 This is a structural block diagram of a computer device provided in an embodiment of this application. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the embodiments of this application.

[0023] In the description of the embodiments of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, features defined with "first" and "second" may explicitly or implicitly include one or more of the stated features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0024] As enterprises undergo increasingly sophisticated digital transformation, email has become a crucial medium for internal collaboration and external business communication, carrying a wealth of critical information related to operations, financial transactions, and business decisions. Simultaneously, email has also become a significant entry point for cyberattacks and information theft, posing a persistent threat to enterprise information security. Current email security solutions often employ rule-based content filtering or keyword matching to perform security checks on email content. However, these solutions have gradually revealed numerous limitations in practical application, proving ill-suited to the increasingly complex and diverse email attack methods.

[0025] First, the existing solutions generally suffer from the limitation of single-modal processing, primarily focusing on the text content of emails while lacking effective comprehensive analysis capabilities for non-text information such as images, attachments, and links contained within the email. Attackers can bypass these detection mechanisms by embedding malicious information in images, carrying malicious programs in attachments, or linking to high-risk domains, thereby reducing the protective effectiveness of these solutions.

[0026] Secondly, rule-based or keyword-based detection methods have shortcomings in semantic understanding. These methods often struggle to understand the contextual semantics and true intent of email content, and are prone to misjudgment or missed detection in attack emails that are highly similar in wording to normal business emails (such as phishing emails disguised with phrases like "urgent payment" or "account abnormality").

[0027] Furthermore, most email security solutions lack the ability to deeply identify the business intent of emails, making it impossible to determine the true purpose of emails from an overall perspective, such as whether they involve financial transfer requests, data breach risks, or social engineering attacks. As a result, it is difficult to adopt targeted security strategies for different types of emails, leading to high false positive and false negative rates.

[0028] Meanwhile, the relevant methods often fail to make full use of email context information, such as the sender's historical email behavior, the interaction relationship between the sender and receiver, and the time sequence characteristics of email sending, making it difficult to identify complex attack behaviors that are persistent and covert, especially in social engineering attack scenarios, where the protection effect is limited.

[0029] For the reasons mentioned above, there is an urgent need for an email content perception and intent recognition technology solution that can fuse and analyze multimodal information in emails, possess deep semantic understanding capabilities, and accurately identify email business intent, so as to improve the accuracy, robustness, and intelligence level of email security monitoring.

[0030] See Figure 1 , Figure 1 This is a flowchart illustrating an email processing method provided in an embodiment of this application.

[0031] As mentioned above, existing email security solutions primarily focus on analyzing the text content of emails, lacking the comprehensive perception and processing capabilities for multimodal information such as images, attachments, and links contained within emails. This makes it difficult to effectively respond to attacks carried out through multimodal carriers. Furthermore, these solutions generally lack deep semantic understanding capabilities and contextual analysis mechanisms for email content, making it difficult to accurately identify the true business intent and potential risk intent carried by the email. In addition, these solutions struggle to uniformly integrate and collaboratively analyze multi-dimensional data such as email header information, body content, attachment features, link information, and historical behavioral information related to the email, making it difficult to form a comprehensive perception and judgment of email content, thus affecting the accuracy and reliability of email security detection.

[0032] To address the issue that related email processing solutions struggle to simultaneously address multimodal content understanding, accurate intent recognition, and reasonable risk assessment, this application provides an email processing method comprising steps S101 to S105.

[0033] Step S101: Perform multimodal feature extraction based on the target email to obtain multimodal feature information corresponding to the target email; the target email includes at least two of the following: text information, image information, attachment information, and link information; the multimodal feature information includes at least two of the following: text feature information, image feature information, attachment feature information, and link feature information.

[0034] Step S102: Perform cross-modal interaction based on the multimodal feature information to generate cross-modal interaction information for the target email.

[0035] Step S103: Classify the intent based on the cross-modal interaction information to obtain the intent type of the target email and its corresponding confidence level. The intent type includes at least one of business communication, financial transfer request, data breach, phishing scam, malware propagation, and social engineering attack.

[0036] Step S104: Based on the intent type and its corresponding confidence level, and combined with historical email information related to the target email, assess the risk of the target email under the intent type to obtain a risk score for the target email.

[0037] Step S105: Determine the risk label of the target email based on the risk score; the risk label is used to characterize the risk level of the target email under the intent type.

[0038] In this embodiment, the target email can refer to an email subject to intent identification and risk assessment, which may originate from an enterprise's internal email system, email security gateway, or cloud email service platform. The target email can be a newly received email or a stored historical email. Target emails typically contain various types of information carriers, such as text content, image content, attachments, and hyperlinks. This embodiment does not limit the specific format or transmission protocol of the target email; the target email can be a plain text email, an HTML format email, or a composite email containing various rich media content.

[0039] Multimodal feature extraction refers to the process of extracting information that characterizes the content features of different information modalities present in a target email. Unlike methods based solely on text analysis, the multimodal feature extraction in this embodiment covers at least two or more of the following: text information, image information, attachment information, and link information. Through multimodal feature extraction, attackers can be less likely to bypass single-modal detection mechanisms by embedding text in images, disguising attachments, or using malicious links, thus providing a more comprehensive and reliable feature foundation for subsequent intent recognition and risk assessment.

[0040] Multimodal feature information refers to a set of information used to describe the multidimensional attributes of a target email, obtained by extracting features from different modal content. This multimodal feature information may include at least two of the following: text feature information, image feature information, attachment feature information, and link feature information. Each modal feature information can be represented in vector, tensor, or other structured data forms to characterize the semantic, structural, or behavioral features of the corresponding modality, providing a unified input basis for subsequent cross-modal interaction and intent classification.

[0041] Textual feature information can be used to characterize the semantic and structural features of the text content in a target email. This textual feature information can originate from text content such as the email subject, email body, email header fields, or email signature. For example, textual feature information may include keyword distribution features, contextual semantic features, semantic embedding vectors, syntactic structure features, or tone tendency features. By extracting semantic features from the text content, a foundation can be provided for identifying different intentions, such as business communication, financial requests, or fraudulent rhetoric.

[0042] Image feature information can be used to characterize the visual features of image content in a target email. The image content may include pictures embedded in the email body, image attachments, or QR code images. For example, image feature information may include visual embedding features, texture features, structural features, or image content recognition results. By introducing image feature information, it is possible to effectively identify email scenarios that use images to disguise text content, impersonate company logos, or embed malicious information.

[0043] Attachment feature information can be used to characterize the attributes and content features of attachment files in a target email. This feature information may include the attachment's file type, file size, file extension, header features, and content characteristics. By extracting and analyzing this feature information, it is possible to identify attachment content that is inconsistent with the email body description or to identify potential malicious file propagation behaviors, thus providing an important basis for risk assessment.

[0044] Link feature information can be used to characterize the relevant features of links contained in a target email. This link feature information may include the URL string to which the link points, domain name characteristics, path structure characteristics, and historical behavioral characteristics related to the link. By extracting link feature information, short links, abnormal domains, or external redirects unrelated to normal business can be identified, thereby helping to determine whether an email carries the risk of phishing or social engineering attacks.

[0045] Cross-modal interaction refers to the process of establishing relationships between multimodal feature information to characterize the mutual influence and consistency between different modal features. In this embodiment, cross-modal interaction is not a simple feature concatenation, but rather a modeling of the correlation between different modal feature information, enabling information such as text, images, attachments, and links to complement or verify each other in a unified semantic space. For example, by calculating the weight relationships between different modal features, modal information that is consistent or conflicting can be highlighted, thereby generating cross-modal interaction information.

[0046] Cross-modal interaction information refers to comprehensive feature information generated through cross-modal interaction processing, used to comprehensively represent the semantic and structural features of the target email. This cross-modal interaction information can reflect the degree of consistency or conflict between different modal features, indicating whether the modal information in the target email collectively points to the same intent. By introducing cross-modal interaction information, misjudgments of a single modality can be reduced, and the stability and accuracy of intent classification results can be improved.

[0047] Intent classification refers to the process of judging the likelihood of a target email falling under multiple preset intent types based on cross-modal interaction information. Intent classification can be implemented using classification models or rule models to identify the email's true business purpose or potential risky purpose from a comprehensive feature perspective. The result of intent classification not only provides the intent type of the target email but also outputs a corresponding confidence score to reflect the reliability of the intent judgment result.

[0048] Intent type can be used to characterize the business or risk semantic category corresponding to the target email, which can include at least one of business communication, financial transfer request, data breach, phishing scam, malware propagation, and social engineering attack. Confidence score is used to characterize the credibility of the intent classification result, for example, expressed in numerical form. By outputting both the intent type and its confidence score, clear semantic premises and weighted references can be provided for subsequent risk assessment.

[0049] Historical email information refers to contextual information associated with the target email and used to assist in risk assessment. This historical email information may include the sender's past sending behavior, the recipient's past interaction records, and email frequency or time distribution characteristics. By introducing historical email information, it is possible to further determine whether the target email exhibits anomalies under a specific intent type, based on the intent classification results, thereby improving the reasonableness of the risk assessment.

[0050] Risk assessment refers to the process of comprehensively judging the potential security risks posed by a target email under that intent type after determining its intent type and confidence level. Risk scores are used to quantitatively characterize the degree of risk of the target email, while risk tags are used to grade the risk scores, such as high risk, medium risk, or low risk. By basing risk assessment on clear intent types and contextual information, risk judgments can be reduced by avoiding detachment from the email's semantic context, thus improving the interpretability and practicality of the risk assessment results.

[0051] The email processing method provided in this embodiment first obtains information from at least two different modalities from the target email and performs feature extraction to form multimodal feature information that can characterize the multidimensional attributes of the email content. Based on this, cross-modal interaction modeling is performed on the different modal feature information to enable the features to be correlated and complementary, thereby generating cross-modal interaction information that comprehensively characterizes the overall semantic and structural features of the target email. Subsequently, based on the cross-modal interaction information, the probability of the target email under multiple candidate intent types is analyzed to obtain the intent type and its confidence level. Then, combining the intent type, the corresponding confidence level, and historical email information related to the target email, the potential risk of the target email under the current intent type is assessed to obtain a risk score reflecting the degree of risk. Finally, based on the risk score, a risk label is determined for the target email to characterize the risk level of the target email under its respective intent type, thereby achieving joint identification and assessment of email intent and risk.

[0052] This embodiment extracts multimodal features from target emails and introduces a cross-modal interaction mechanism to achieve joint modeling of various information such as email text, images, attachments, and links, thereby improving the problem of incomplete intent understanding caused by relying solely on single-modal information. By classifying intent based on cross-modal interaction information and outputting corresponding confidence scores, the business intent and risk intent of the email can be accurately identified within a unified intent space, thus improving the reliability of intent recognition results. Based on the intent classification results, further combining historical email information to assess the risk of the target email under specific intent types provides a clear semantic premise and contextual basis for the risk assessment process, thereby improving the problem of the disconnect between risk judgment and the true intent of the email. By determining risk labels based on risk scores, a graded expression of email risk levels is achieved, facilitating differentiated processing of subsequent security strategies. In summary, to address the challenge that existing email processing solutions struggle to simultaneously achieve multimodal content understanding, accurate intent recognition, and reasonable risk assessment, this embodiment employs a concept based on multimodal feature interaction for email intent recognition and risk assessment. This enables comprehensive perception of email content, accurate identification of email intent, and reasonable assessment of email risks, thereby improving the intelligence and security of the email processing process.

[0053] In some embodiments, the multimodal feature extraction based on the target email may include: performing multimodal content extraction on the target email to obtain at least two of the following: text content, image content, attachment content, and link content corresponding to the target email; and performing feature encoding on at least two of the following: text content, image content, attachment content, and link content, to obtain at least two of the following: text feature information, image feature information, attachment feature information, and link feature information.

[0054] Multimodal content extraction refers to the process of parsing and obtaining raw content data of different modalities from a target email. This extraction can include identifying and separating text content, image content, attachment content, and link content contained in the email. It is not required that all of these modalities exist simultaneously; at least two must be extracted. The email structure is parsed to distinguish different types of information carriers, enabling subsequent processing using corresponding feature encoding methods for each modality.

[0055] Text content can be used to represent information presented in text form in a target email, and it can include at least one of the following: email subject, email body, email header fields, and email signature. Text content can carry the main semantic information of the email, such as business instructions, directives, or persuasive language. Extracting text content can provide basic input for subsequent semantic analysis, intent classification, and risk assessment.

[0056] Image content can be used to represent information in the form of images within a target email. This can include pictures embedded in the email body, image files attached as attachments, or QR code images included in the email. Image content may be used to convey key information, impersonate legitimate identities, or hide malicious information. By extracting image content separately, the amount of information carried by images that is overlooked can be reduced, thereby improving the ability to handle complex email scenarios.

[0057] Attachments can be used to represent file information sent along with the target email. These can include document files, compressed files, executable files, or other file types. Attachment content includes not only the file itself but also metadata information related to the file, such as file type, file size, or file format. Extracting attachment content can provide a basis for subsequently determining whether the attachment is semantically consistent with the email and whether there are any potential risks.

[0058] Link content can be used to characterize hyperlink information contained in target emails. This can include URLs embedded in the email body, link text, and the domain information the links point to. Link content is often used to redirect recipients to external pages, thus posing a high risk in phishing scams or social engineering attacks. Extracting link content can provide foundational data for subsequent analysis of the link's legitimacy and relevance.

[0059] Feature encoding refers to the process of converting extracted raw content data into a feature representation that can be processed by a model. In this embodiment, different modalities can be processed using different feature encoding methods to generate corresponding feature information. The results of feature encoding are usually represented in vector, matrix, or other structured forms to unify the information expression forms of different modalities, thereby supporting subsequent cross-modal interaction and intent classification.

[0060] For example, text feature information can be obtained by encoding text content using a text encoder. Text feature information can include semantic embedding vectors, keyword distribution features, contextual features, or syntactic structure features. Through text feature information, the business purpose or risk intent expressed in the email text can be modeled. Image feature information can be obtained by encoding image content using an image encoder. Image feature information can include overall visual features, local structural features, or image content recognition results. Through image feature information, visual information inconsistent with the text content or abnormal image content can be identified, thereby assisting in intent judgment. Attachment feature information can be obtained by encoding attachment content using an attachment encoder. Attachment feature information can include file type features, file size features, file header features, or file content summary features. Through attachment feature information, it can be determined whether the attachment semantically matches the email and whether there are potential risks. Link feature information can be obtained by encoding link content using a link encoder. Link feature information can include domain name structure features, path features, or historical behavior-related features. Through link feature information, it can help determine whether the link matches the intent type of the target email, thereby improving the accuracy of subsequent intent recognition and risk assessment.

[0061] In some embodiments, the cross-modal interaction based on the multimodal feature information may include: performing a correlation assessment between feature information of different modalities in the multimodal feature information to obtain corresponding correlation assessment information; and performing feature fusion on feature information of different modalities in the multimodal feature information based on the correlation assessment information to generate cross-modal interaction information of the target email.

[0062] Relevance assessment refers to the process of analyzing and quantifying the degree of correlation between features from different modalities within multimodal feature information. This assessment determines whether different modal features point to the same email intent at the semantic or behavioral level, or whether there are significant inconsistencies. Through relevance assessment, collaborative or conflicting relationships between text content, image content, attachment content, and linked content can be identified, thus providing a weighting basis for feature fusion.

[0063] Relevance assessment information refers to intermediate results obtained through a correlation assessment process, used to characterize the degree of association between different modal features. This correlation assessment information can be represented using weight values, similarity scores, association matrices, or other forms to reflect the degree of mutual influence between different modal features. In subsequent feature fusion processes, the correlation assessment information guides the contribution ratio of different modal features during fusion, thereby achieving more reasonable multimodal information integration.

[0064] Feature fusion refers to the process of comprehensively processing feature information from different modalities in multimodal feature information based on relevance assessment information. Feature fusion is not limited to a fixed fusion method, but can be achieved through weighted fusion, attention mechanisms, or cross-modal interaction. Through feature fusion, feature information from different modalities can be integrated into a unified feature representation, enabling a complete expression of the overall semantic and structural features of the target email.

[0065] In some embodiments, the correlation assessment between feature information of different modalities in the multimodal feature information may include: using an attention model to assess the correlation between feature information of different modalities in the multimodal feature information, and calculating a correlation weight matrix between different modal features; the correlation weight matrix serves as the correlation assessment information.

[0066] An attention model can be a model structure used to characterize the importance distribution among different input features. It dynamically assigns weights to different features in the overall representation based on the degree of correlation between them. In this embodiment, the attention model is used to model the relevance of feature information from different modalities within multimodal feature information. This allows modal features highly relevant to the target email intent to receive higher weights, while modal features with lower relevance or conflicting relationships receive lower weights. The attention model can be implemented using self-attention mechanisms, cross-modal attention mechanisms, or other attention computation methods.

[0067] A relevance weight matrix is ​​a data structure obtained by calculating the degree of correlation between different modal feature information through an attention model. The relevance weight matrix is ​​used to characterize the relative importance or correlation strength between any two modal features. The numerical values ​​of each element in the matrix reflect the weight of the corresponding modal feature in the current email semantic modeling, thus providing a clear weight basis for the subsequent feature fusion stage.

[0068] In this embodiment, the relevance weight matrix can be used as relevance assessment information. By inputting the relevance weight matrix as relevance assessment information into subsequent processing, the contribution ratio of different modal features can be dynamically adjusted during the feature fusion stage, enabling the fusion result to more accurately reflect the overall semantic and structural features of the target email.

[0069] In some embodiments, determining the risk label of the target email based on the risk score may include: determining the risk label of the target email based on the risk score and a plurality of preset risk score thresholds; the plurality of risk score thresholds are used to divide the value range of the risk score into a plurality of numerical intervals.

[0070] A risk score can be a numerical indicator used to quantify the potential risk level of a target email under its corresponding intent type. In this embodiment, the risk score can be calculated comprehensively based on factors such as the intent type of the target email, the corresponding confidence level, and historical email information related to the target email. The range of the risk score can be set according to the system design, for example, it can be a continuous numerical range or a discrete numerical range, and its specific value is used to reflect the relative level of risk of the target email.

[0071] A risk scoring threshold can refer to a preset numerical boundary used to divide risk scores into intervals. In this embodiment, the risk scoring threshold may include multiple thresholds to divide the range of risk scores into different numerical intervals. For example, the risk scoring threshold may include a low-risk threshold, a medium-risk threshold, and a high-risk threshold, etc., to distinguish different risk levels. The specific value of the risk scoring threshold can be configured and adjusted according to the application scenario, security policy, or historical data.

[0072] Multiple risk scoring thresholds refer to at least two threshold parameters set within the same risk assessment system to distinguish risk levels. By setting multiple risk scoring thresholds, the overall range of risk scores can be divided into multiple continuous or discrete numerical intervals, each corresponding to a risk level. Using multiple risk scoring thresholds helps to achieve refined classification of email risk levels, thereby supporting differentiated handling strategies for different risk levels.

[0073] Risk labels refer to information used to classify and identify the risk level of a target email. In this embodiment, risk labels are used to characterize the risk level of a target email under its intended type. For example, risk labels can include various forms such as high risk, medium risk, low risk, or normal. The risk labels can also adopt other level forms, such as normal or suspicious. The setting of risk labels is not limited to specific names; its purpose is to express the risk level of the target email in a graded manner. Outputting risk assessment results through risk labels can improve the interpretability of the risk assessment results and facilitate subsequent decision-making by email processing systems or human personnel.

[0074] Risk label determination based on risk scores refers to the process of classifying target emails into corresponding numerical ranges and assigning them appropriate risk labels based on the relationship between the risk score corresponding to the target email and a preset risk score threshold. In this embodiment, the risk score is compared with multiple risk score thresholds. When the risk score falls into the numerical range corresponding to a certain risk label, that risk label is determined as the risk label for the target email. This method maintains the continuity of risk assessment results while achieving a discrete expression of risk levels, facilitating system processing and security policy execution. This classification method provides clear judgment criteria for the risk assessment process, reduces frequent changes in risk levels due to fluctuations in risk scores, and thus improves the stability and usability of email risk assessment results.

[0075] In some embodiments, the risk label includes at least one of high risk, medium risk, and low risk, and the method may further include: determining the corresponding processing priority based on the risk label of the target email.

[0076] High risk, medium risk, and low risk are different levels of risk classification for target emails, reflecting the relative severity of their security implications. High risk indicates that the target email has a high probability of posing a security threat under the current intent type, such as potentially involving phishing scams, malware distribution, or data breaches. Medium risk indicates that the target email exhibits some unusual characteristics, but not enough to be considered a clear threat. Low risk indicates that the overall characteristics of the target email are largely consistent with normal business emails. The specific meanings of these risk levels can be configured and adjusted according to application scenarios, security policies, or business needs.

[0077] Prioritizing processing based on risk tags refers to the process of differentiating the response order or processing strategy of target emails in subsequent processing flows based on the risk tags they correspond to. In this embodiment, risk tags are used as a basis for high-level decision-making, with different risk levels corresponding to different processing priorities, thereby achieving a reasonable allocation of limited processing resources. For example, high-risk emails can be assigned a higher processing priority for priority security handling; medium-risk emails can be arranged for further analysis or manual review; and low-risk emails can be processed according to the regular process.

[0078] Processing priority refers to a control parameter in an email processing system that indicates the relative importance of different target emails in terms of processing order and resource allocation. Processing priority can be represented in the form of levels, numerical values, or queue order, and this embodiment does not limit this. By introducing a processing priority mechanism, the email processing system can dynamically adjust its processing strategy according to the level of risk, improving overall processing efficiency and reducing the possibility of high-risk emails not being processed in a timely manner.

[0079] This embodiment uses risk tags as trigger conditions to adopt differentiated processing strategies for target emails. By decoupling the risk assessment results from the processing flow, risk identification and risk handling can work collaboratively but independently, thereby improving the system's flexibility and scalability. Through this processing mechanism, the backend processing strategy can be flexibly adjusted without changing the frontend risk identification algorithm to adapt to different security policies or business needs.

[0080] In some embodiments, the text information may include at least one of email subject, email body, email header fields, and email signature. The email header fields may include at least one of sender address, recipient address, sending time, and CC information. The email subject can be used to summarize the core message of the email, the email body can be used to describe the specific content, the email header fields can be used to reflect the basic communication attributes of the email, and the email signature can be used to identify the sender's identity or organizational information.

[0081] Email header fields refer to structured information attached to the email transmission process, used to describe the communication attributes and delivery relationship of the email. In this embodiment, email header fields may include at least one of the following: sender address, recipient address, sending time, and CC information. The sender address and recipient address are used to identify the two parties in the email communication, the sending time reflects the time attribute of the email, and the CC information describes the scope of the email's dissemination. By extracting email header fields, a basis can be provided for subsequent contextual analysis combined with historical email information, thereby assisting in determining whether there are any anomalies in email behavior.

[0082] In some embodiments, the attachment information may include at least one of the attachment's file type, file size, and file content characteristic information, wherein the file content characteristic information includes at least one of file header information and file hash information.

[0083] File content feature information refers to the characteristic information used to describe the internal attributes of attachment files. File header information can be used to reflect the true format and structural characteristics of the file, while file hash information can be used to uniquely identify the file content. By extracting file content feature information, attachment files can be quickly identified and compared without directly parsing the entire file content, thereby helping to determine whether the attachments are abnormal or have potential risks.

[0084] In some embodiments, the link information may include a URL link contained in the email body and the domain name information pointed to by the URL link. The domain name information includes at least one of the following: domain name string length, registration time information, certificate information, path information, and historical reputation information.

[0085] Domain information refers to the domain-related attribute information corresponding to the URL link, used to characterize the network identity features of the linked target. In this embodiment, domain information may include at least one of the following: domain string length, registration time information, certificate information, path information, and historical reputation information. Domain string length and path information can be used to describe the domain's structural characteristics, registration time information can be used to reflect the domain's usage history, certificate information can be used to reflect secure communication attributes, and historical reputation information can be used to characterize the domain's security performance in historical communications. Through comprehensive analysis of domain information, it is possible to help determine whether the link has abnormal or high-risk characteristics.

[0086] In some embodiments, the historical email information may include at least one of the following: sender's historical email behavior information, recipient's historical email interaction information, and email time series information related to the target email.

[0087] Sender's historical email behavior information can be used to describe the sender's sending frequency, recipients, or behavioral patterns. Recipient's historical email interaction information can be used to reflect the historical communication relationship between the recipient and the sender. Email time series information can be used to characterize the distribution characteristics of emails in the time dimension.

[0088] In some embodiments, the method may further include: determining a processing strategy for the target email based on at least one of the target email's intent type and its corresponding confidence level, risk score, and risk label using a processing strategy model; the processing strategy includes at least one of interception, isolation, marking, encryption, deletion, forwarding, and notification; receiving feedback information regarding the processing strategy, and forming training data for training the processing strategy model based on the feedback information and at least one of the target email's intent type and its corresponding confidence level, risk score, and risk label.

[0089] In this embodiment, based on the identification of the intent type and risk assessment of the target email, a processing strategy model is further introduced to achieve email processing decisions and adaptive optimization of the model. Specifically, by using at least one of the intent type of the target email and its corresponding confidence level, risk score, and risk tag as input features, the processing strategy model is invoked to comprehensively judge the target email and determine a processing strategy that matches its risk level and business attributes. The processing strategy includes at least one of interception, isolation, marking, encryption, deletion, forwarding, and notification. After the processing strategy is executed, feedback information is received regarding the processing strategy, such as manual confirmation results, misjudgment correction information, or handling effect information. This feedback information is associated with the intent type, confidence level, risk score, and risk tag of the target email to form training data for training the processing strategy model. This allows the processing strategy model to be updated through continuous learning, enabling it to more accurately output processing strategies that meet actual needs in subsequent email processing.

[0090] This embodiment automatically determines the email processing strategy based on the target email's intent type and its corresponding confidence level, risk score, and risk label using a processing strategy model. This achieves effective linkage between email risk assessment results and specific actions, thus improving the rigidity caused by relying solely on manual rules or fixed thresholds. By receiving and utilizing feedback information after the processing strategy is executed, the model is trained to continuously optimize its decision logic based on actual handling effects, thereby improving the accuracy and adaptability of the email processing strategy. Addressing the problem in related email processing schemes where risk assessment results are difficult to translate into effective handling decisions and where processing strategies lack self-optimization mechanisms, this embodiment adopts a concept combining intent recognition and risk assessment result-driven processing strategy modeling with feedback learning. This achieves dynamic optimization of email processing decisions, improving the intelligence level and practical application effectiveness of email security protection.

[0091] In one specific embodiment, the target email's corresponding indicators are as follows: Risk Score: 85 points; Risk Tag: High Risk; Intent Type: Phishing; The three risk factors with the highest risk values ​​are: Abnormal Link, First Communication, and External Domain. Using a processing strategy model, the processing strategy for the target email is determined from the processing strategy rule base, for example, matching two alternative processing strategies: "High-Risk Email Interception Strategy" and "Phishing Email Isolation Strategy." Based on the matching priority, the processing strategy model outputs the "High-Risk Email Interception Strategy." Next, the target email is intercepted, specifically including the following steps: storing the target email in an isolation area; preventing the delivery of the target email; sending an interception notification to security operations personnel; recording the handling log and sending the notification. Afterwards, feedback information regarding the processing strategy is collected. If security operations personnel confirm the interception is correct, the processing strategy needs to be strengthened; if it is a false interception, the processing strategy needs to be adjusted. The feedback information is combined with other indicators to form training data, updating the relevant model parameters of the processing strategy model, thereby optimizing the processing strategy output by the processing strategy model.

[0092] In another specific embodiment, the target email's corresponding indicators are as follows: Risk score: 65 points; Risk tag: Medium risk; Intent type: Suspicious attachment; The two risk factors with the highest risk values ​​are: Unknown attachment type and first-time communication. Using a processing strategy model, the processing strategy for the target email is determined from the processing strategy rule base, for example, matching two alternative processing strategies: "Medium-risk email marking strategy" and "Suspicious attachment warning strategy." Based on the matching priority, the processing strategy model outputs the "Suspicious attachment warning strategy." Next, a marking operation is performed on the target email, specifically including the following steps: adding a "[Risk Warning]" tag before the email subject; inserting a risk warning at the beginning of the email body: "This email contains suspicious attachments, please handle with caution"; delivering the email normally; and sending an alert to the security operations platform. Afterwards, feedback information regarding the processing strategy is collected (such as whether the attachment was clicked, whether a complaint was filed). Based on the feedback information and other indicators, training data is formed to update the relevant model parameters of the processing strategy model, thereby optimizing the processing strategy output by the processing strategy model.

[0093] In some embodiments, assessing the risk of the target email under the intent type may include: extracting multiple risk factors based on the target email and historical email information related to the target email, and assigning a risk value to each risk factor; assessing the importance of each risk factor based on the intent type and its corresponding confidence level to determine the weight parameter of each risk factor; and weighting and fusing the risk values ​​of multiple risk factors according to their corresponding weight parameters to obtain a risk score for the target email.

[0094] In this embodiment, the risk assessment of the target email under the intent type is achieved by quantitative analysis and weighted fusion of multi-dimensional risk factors after completing the email intent classification. Specifically, multiple risk factors of different dimensions are extracted based on the target email itself and historical email information related to the target email, and a corresponding risk value is assigned to each risk factor. Subsequently, the importance of each risk factor under the current intent type is evaluated by combining the intent type of the target email and its corresponding confidence level to determine the weight parameter of each risk factor. Finally, the risk values ​​of multiple risk factors are weighted and fused according to the corresponding weight parameters to obtain a risk score that characterizes the overall risk level of the target email. Through the above method, a multi-dimensional and quantifiable assessment of the risk of the target email is achieved.

[0095] This embodiment extracts risk factors from multiple dimensions based on the target email and its related historical email information, and assigns risk values ​​to each risk factor, achieving a fine-grained characterization of email risk features. By dynamically evaluating the weight of each risk factor in conjunction with the intent type and its corresponding confidence level, the influence of the risk factors can be adaptively adjusted according to different intent types. By weighted and fused calculation of the risk values ​​of multiple risk factors, a unified risk score is obtained, thereby reducing the risk assessment bias caused by a single indicator or fixed rules. This embodiment addresses the problem that related email risk assessment methods have a single risk judgment dimension and are unable to reflect the differences in the importance of risk factors under different intent scenarios. It adopts an intent-driven multi-risk factor weighted assessment concept, thereby improving the accuracy, interpretability, and applicability of the risk scoring results, and providing a reliable basis for subsequent risk label determination and email processing decisions.

[0096] In one specific embodiment, based on the target email and related historical email information, multiple risk factors are extracted, as shown below. Sender reputation risk factor: external domain, SPF verification failed, domain reputation score 20 (low); Sender-receiver relationship risk factor: first communication, no historical records; Content semantic risk factor: contains high-risk keywords such as urgent payment and clickable links, semantic analysis shows fraudulent intent, confidence level 0.85; Attachment security risk factor: no attachments; Link reputation risk factor: contains short links pointing to newly registered domains (registration time <30 days), in the phishing domain database; Historical behavior risk factor: no relevant historical records. The processing flow is as follows. First, the aforementioned risk factors are extracted and assigned risk values. For example, the risk value R1 for the sender reputation risk factor is 75 (low sender reputation), the risk value R2 for the sender-receiver relationship risk factor is 60 (first communication), the risk value R3 for the content semantic risk factor is 90 (high content risk), the risk value R4 for the attachment security risk factor is 0 (no attachment), the risk value R5 for the link reputation risk factor is 95 (high link risk), and the risk value R6 for the historical behavior risk factor is 50 (no history). Next, based on the intent type and its corresponding confidence level, the importance of each risk factor is assessed to determine its weight parameter. For example, the weight parameter for the sender reputation risk factor is 20%, the weight parameter for the sender-receiver relationship risk factor is 15%, the weight parameter for the content semantic risk factor is 30%, the weight parameter for the attachment security risk factor is 5%, the weight parameter for the link reputation risk factor is 25%, and the weight parameter for the historical behavior risk factor is 5%. Next, the risk values ​​of multiple risk factors are weighted and fused according to their corresponding weight parameters to obtain a risk score: Risk_Score = 75×0.2 + 60×0.15 + 90×0.3 + 0×0.05 + 95×0.25 + 50×0.05 = 78.25 points. Since 78.25 points < 80 points (as an example of a risk score threshold), the risk label is determined to be "medium risk" or "high risk". The determined handling strategy is, for example, to deliver normally but insert a risk warning and send an alert.

[0097] In another specific embodiment, based on the target email and related historical email information, multiple risk factors are extracted, as shown below. Sender reputation risk factor: Internal enterprise domain, SPF / DKIM / DMARC verified, domain reputation score 95 (high); Sender-receiver relationship risk factor: Long-term normal communication, high historical communication frequency, strong relationship; Content semantic risk factor: Normal business communication content, semantic analysis shows normal intent, confidence level 0.92; Attachment security risk factor: PDF attachment, high file reputation score, no macros; Link reputation risk factor: No links; Historical behavior risk factor: Similar emails have historically been marked as normal, and recipients frequently reply to such emails. The processing flow is as follows. First, the aforementioned risk factors are extracted and assigned risk values. For example, the risk value R1 for the sender reputation risk factor is 5 (high sender reputation), the risk value R2 for the sender-receiver relationship risk factor is 10 (long-term normal communication), the risk value R3 for the content semantic risk factor is 8 (normal content), the risk value R4 for the attachment security risk factor is 5 (secure attachment), the risk value R5 for the link reputation risk factor is 0 (no link), and the risk value R6 for the historical behavior risk factor is 5 (normal history). Next, based on the intent type and its corresponding confidence level, the importance of each risk factor is assessed to determine its weight parameter. For example, the weight parameter for the sender reputation risk factor is 20%, the weight parameter for the sender-receiver relationship risk factor is 15%, the weight parameter for the content semantic risk factor is 30%, the weight parameter for the attachment security risk factor is 5%, the weight parameter for the link reputation risk factor is 25%, and the weight parameter for the historical behavior risk factor is 5%. Next, the risk values ​​of multiple risk factors are weighted and fused according to their corresponding weight parameters to obtain a risk score: Risk_Score = 5×0.2 + 10×0.15 + 8×0.3 + 5×0.05 + 0×0.25 + 5×0.05 = 6.4 points. Since 6.4 points < 20 points (as an example of a risk score threshold), the risk label is determined to be "normal". The processing strategy determined accordingly is, for example, to record only basic statistical information and deliver normally.

[0098] In some embodiments, the process of determining the target email may include: upon detecting a preset trigger event for at least one email, determining each email as the target email; wherein the preset trigger event includes at least one of the following: a new email arrives; a user initiates a manual review request for an email; or a timed scanning task detects emails that are not empty.

[0099] In this embodiment, the determination of the target email is achieved during the email processing flow initiation phase by detecting preset trigger events. Specifically, it involves detecting whether a preset trigger event for at least one email has occurred, and if such a preset trigger event is detected, each corresponding email is identified as the target email. The preset trigger events can include situations such as the arrival of a new email, receiving a user-initiated manual review request for an email, and emails detected by a scheduled scanning task not being empty. This approach allows the email processing flow to be initiated under different trigger scenarios, thereby incorporating emails that meet the trigger conditions into subsequent multimodal feature extraction, intent classification, and risk assessment processes.

[0100] This embodiment achieves flexible control over the timing of email processing initiation by determining target emails based on preset trigger events. By supporting multiple triggering methods such as new email arrivals, manual review requests, and scheduled scanning tasks, the system can simultaneously address real-time processing needs and periodic security detection requirements, reducing the omission of potentially high-risk emails. Addressing the issue of limited triggering conditions and difficulty in simultaneously satisfying real-time and comprehensive requirements in other email processing solutions, this embodiment adopts a target email determination concept driven by multiple trigger events, thereby improving the coverage, responsiveness, and overall reliability of the email processing workflow.

[0101] In some embodiments, when assessing the risk of a target email based on the intent type and its corresponding confidence level, a corresponding risk assessment strategy or set of risk assessment parameters can be adopted according to different intent types to conduct differentiated risk assessments of the target email. Specifically, the risk assessment strategies corresponding to different intent types differ in at least one of the following: feature weight allocation method, risk calculation model parameters, and the participation method of historical email information. This improves the accuracy of risk assessment and better meets practical application needs.

[0102] In some embodiments, before performing cross-modal interaction based on the multimodal feature information, the set of modal feature information participating in cross-modal interaction can be dynamically determined according to the actual modal types contained in the target email, and only the selected modal feature information is subjected to correlation evaluation and feature fusion processing to reduce the interference of invalid modalities on the cross-modal interaction results.

[0103] In some embodiments, the risk score can be determined by multiple risk sub-scores, each of which corresponds to at least one of text feature information, image feature information, attachment feature information, and link feature information. The risk score is obtained by weighting the multiple risk sub-scores, thereby enabling the risk score to reflect the contribution of different modal features to the overall risk.

[0104] In some embodiments, the historical email information may further include historical risk score sequences or historical intent distribution information related to the target email. During the risk assessment process, the risk score of the target email is corrected or enhanced by analyzing the changing trends or deviations between the risk score corresponding to the target email and the historical risk score sequences. This can improve the accuracy of risk scoring.

[0105] In some embodiments, after determining the processing priority based on the risk label of the target email, corresponding automated processing actions can be triggered according to the risk label. The automated processing actions may include at least one of email delayed delivery, email content isolation, additional security verification, and blocking email transmission.

[0106] In some embodiments, during the risk assessment process, the confidence level corresponding to the intent type can also be used as one of the input factors for risk assessment. When the confidence level is lower than a preset confidence threshold, the risk score of the target email can be increased or the corresponding risk label can be adjusted to reduce the impact of intent recognition uncertainty on the risk assessment results.

[0107] In some embodiments, the correlation weights between different modal features can be adaptively adjusted based on historical email processing results or risk assessment feedback information, so that the influence weights of each modal feature during cross-modal interaction are dynamically updated as the email security situation changes.

[0108] In a specific application scenario, the email processing method may include the following steps.

[0109] First, multimodal feature extraction is performed on the target email to obtain multimodal feature information corresponding to the target email. The target email includes at least two of the following: text information, image information, attachment information, and link information. The multimodal feature information includes at least two of the following: text feature information, image feature information, attachment feature information, and link feature information.

[0110] In some embodiments, the multimodal feature extraction based on the target email may include: performing multimodal content extraction on the target email to obtain at least two of the following: text content, image content, attachment content, and link content corresponding to the target email. Specifically: the text content may include at least one of the following: email subject, email body, and email header fields; the email header fields may include, for example, at least one of the following: sender address, recipient address, sending time, and CC information; the image content may include images embedded in the email body and images attached as attachments; the attachment content may include the attachment's file type, file size, and file content feature information, wherein the file content feature information may include, for example, at least one of the following: file header information and file hash information; the link content may include URL links contained in the email body and the domain information pointed to by the URL links, wherein the domain information may include, for example, at least one of the following: domain string length, registration time information, certificate information, path information, and historical reputation information.

[0111] In some embodiments, the multimodal feature extraction may further include: performing feature encoding on at least two of the text content, the image content, the attachment content, and the link content to obtain at least two of the text feature information, the image feature information, the attachment feature information, and the link feature information. Wherein: the text feature information can be obtained by semantically encoding the email text using a natural language processing model, such as a semantic encoding model based on a pre-trained language model; the image feature information can be obtained by extracting visual features of the image using an image recognition model, such as a model based on a convolutional neural network or a visual converter model; the attachment feature information can be obtained by analyzing the metadata and content features of the attachment, such as extracting file type, file size, and file hash information; the link feature information can be obtained by analyzing the domain name features, path features, and historical reputation features corresponding to the link.

[0112] Subsequently, cross-modal interaction is performed based on the multimodal feature information to generate cross-modal interaction information for the target email.

[0113] In some embodiments, the cross-modal interaction based on the multimodal feature information may include: performing a correlation assessment between feature information of different modalities in the multimodal feature information to obtain corresponding correlation assessment information; and performing feature fusion on feature information of different modalities in the multimodal feature information based on the correlation assessment information to generate cross-modal interaction information of the target email.

[0114] In some embodiments, the correlation assessment between different modal features in the multimodal feature information can be implemented using an attention model to calculate a correlation weight matrix between different modal features; the correlation weight matrix serves as the correlation assessment information. Through this method, information interaction and complementarity between different modal features can be achieved, thereby forming a unified representation of the target email content.

[0115] After obtaining the cross-modal interaction information, intent classification is performed based on the cross-modal interaction information to obtain the intent type of the target email and its corresponding confidence level. The intent type includes at least one of business communication, financial transfer request, data breach, phishing scam, malware propagation, and social engineering attack. The intent classification can be implemented using a preset intent classification model, which is used to distinguish between business intent and risk intent in the target email based on the cross-modal interaction information and output the corresponding confidence level.

[0116] Subsequently, based on the intent type and its corresponding confidence level, and combined with historical email information related to the target email, the risk of the target email under the intent type is assessed to obtain a risk score for the target email. In some embodiments, the historical email information may include at least one of the following: sender's historical email behavior information, recipient's historical email interaction information, and email time series information related to the target email. A risk label is determined based on the risk score, and the risk label characterizes the degree of risk of the target email under the intent type.

[0117] In some embodiments, determining the risk label of the target email based on the risk score may include: determining the risk label of the target email based on the risk score and a plurality of preset risk score thresholds, wherein the plurality of risk score thresholds are used to divide the value range of the risk score into a plurality of numerical intervals. In some embodiments, the risk label includes at least one of high risk, medium risk, and low risk, and the email processing method may further include: determining a corresponding processing priority based on the risk label of the target email.

[0118] In another specific application scenario, the email processing method may include the following steps.

[0119] First, the target email is retrieved. This target email can originate from the enterprise email system or email security gateway, and is used for subsequent processing and analysis.

[0120] Subsequently, multimodal content extraction is performed on the target email to obtain various modal information contained in the target email. The multimodal content may include at least one of text content, image content, attachment content, and link content, wherein text content includes, for example, the email subject and email body; image content includes, for example, an image embedded in the email body; attachment content includes, for example, a file sent with the email; and link content includes, for example, a URL link contained in the email body.

[0121] After acquiring the multimodal content, feature encoding is performed on each of the multimodal contents to generate corresponding multimodal feature representations. For example, text feature vector V_text, image feature vector V_image, attachment feature vector V_attach, and link feature vector V_link can be obtained respectively.

[0122] Based on this, the multimodal features are fused to generate a unified multimodal feature representation. The fusion process can be implemented using an attention mechanism, for example, and its exemplary form can be: V_fused = Attention(V_text, V_image, V_attach, V_link).

[0123] Subsequently, based on the fused multimodal feature representation, intent recognition is performed on the target email. This intent recognition process can be implemented, for example, through an intent classification model, whose exemplary expression can be: Intent = Classify(V_fused), to obtain the intent type corresponding to the target email.

[0124] After completing intent identification, the intent risk of the target email is assessed. This risk assessment process can be implemented, for example, through a risk assessment model, whose exemplary expression can be: Risk_Score = RiskModel(Intent), to obtain the risk score corresponding to the target email.

[0125] Finally, the identification results of the target email are output. The identification results may include the intent type of the target email, the risk score, and the risk label determined based on the risk score, thus completing the entire processing flow from obtaining the target email to outputting the identification results.

[0126] In another specific application scenario, the email processing method may include the following steps.

[0127] First, obtain the target email, for example, obtain the target email to be inspected from the enterprise email system as the processing object.

[0128] Secondly, multimodal content is extracted based on the target email, obtaining at least one of text information, image information, attachment information, and link information from the target email. The text information may come from the email subject and email body; the image information may come from images embedded in the email body or images that exist as attachments; the attachment information may include the metadata and content characteristics of the attachments; and the link information may include links contained in the email body and their corresponding domain names or path information.

[0129] Next, feature encoding is performed on the extracted content of different modalities to obtain corresponding multimodal feature vector representations. Specifically, a natural language processing model can be used to perform semantic encoding on the email text to generate a text feature vector V_text; an image recognition model can be used to extract visual features from the image to generate an image feature vector V_image (when an image exists in the target email); an attachment feature vector V_attachment can be generated based on the metadata and content features of the attachment (when an attachment exists in the target email); and a link feature vector V_link can be generated based on the domain name and path features of the link (when a link exists in the target email).

[0130] In some embodiments, different modal features are fused based on the multimodal feature vector. Specifically, an attention mechanism can be used to model the correlation between different modal features, and a cross-modal attention weight matrix A is calculated to represent the correlation between different modal features. Based on the attention weights, the feature vectors of each modality are weighted and fused to form a unified multimodal representation vector, which is exemplarily represented as follows: V_fused = Attention(V_text, V_image, V_attachment, V_link).

[0131] In some embodiments, the fused multimodal feature vector V_fused is input into an intent classification model to perform intent recognition processing on the target email, thereby outputting the intent category of the target email and its corresponding confidence score. The intent recognition process can be represented, for example, as: Intent = Classify(V_fused), Confidence = Softmax(Intent).

[0132] In some embodiments, the intent risk of a target email is assessed based on the identified intent category and confidence score, combined with historical behavioral patterns associated with the target email. Specifically, a risk score for the target email can be calculated using a risk assessment model, and a corresponding risk label can be assigned based on the risk score. An exemplary representation is as follows: Risk_Score = RiskModel(Intent, Confidence, History), Risk_Label = AssignLabel(Risk_Score).

[0133] In some embodiments, the identification results of the target email are output, which may include the email's intent category, confidence score, risk score, and risk label, for subsequent email risk handling and decision-making.

[0134] The above embodiments, through multimodal content extraction and fusion, can comprehensively perceive various information in emails, including text, images, attachments, and links, effectively improving the detection capability for multimodal attacks and overcoming the limitations of single-modal detection. Through deep semantic understanding and contextual analysis, the true business intent and potential risk intent of emails can be accurately identified, significantly reducing false positive and false negative rates. Through multimodal feature fusion and a unified intent recognition process, comprehensive perception and judgment of email content are achieved, improving the intelligence and accuracy of email security monitoring.

[0135] In some application scenarios, the email processing method provided in this application is applicable to the monitoring and automated processing of emails related to pharmacovigilance (PV) safety information. In pharmacovigilance operations, regulatory agencies, pharmaceutical companies, and medical institutions typically receive adverse drug reaction reports, safety clues, risk warnings, and related supplementary materials via email from medical personnel, patients, or partners. These emails often contain multiple information formats, including text descriptions, attachments (such as case report forms and test reports), images, and external links. Furthermore, the email content is complex and semantically diverse, placing a significant burden on manual screening and traditional rule-based processing methods.

[0136] This application's embodiments, through multimodal feature extraction, cross-modal interaction, intent classification, and risk assessment of target emails, can automatically identify and classify email content involving adverse drug reaction reports, potential safety risk warnings, and emergency risk notifications. This assists relevant pharmacovigilance systems in promptly identifying high-risk safety information, improving the efficiency and accuracy of drug safety monitoring. It should be noted that the technical solution described in this application's embodiments is not limited to pharmacovigilance scenarios but is also applicable to other application scenarios requiring multimodal perception and intent recognition of email content, such as financial risk control, enterprise secure communication, and compliance review.

[0137] See Figure 2 , Figure 2 This is a structural block diagram of an email processing system provided in an embodiment of this application.

[0138] This application also provides an email processing system, including a feature extraction module, a multimodal fusion module, an intent recognition module, and a risk assessment module.

[0139] The feature extraction module is used to perform multimodal feature extraction based on the target email to obtain multimodal feature information corresponding to the target email; the target email includes at least two of the following: text information, image information, attachment information, and link information; the multimodal feature information includes at least two of the following: text feature information, image feature information, attachment feature information, and link feature information.

[0140] The multimodal fusion module is used to perform cross-modal interaction based on the multimodal feature information to generate cross-modal interaction information of the target email.

[0141] The intent recognition module is used to classify intents based on the cross-modal interaction information to obtain the intent type of the target email and its corresponding confidence level; the intent type includes at least one of business communication, financial transfer request, data breach, phishing scam, malware propagation and social engineering attack.

[0142] The risk assessment module is used to assess the risk of the target email under the intent type based on the intent type and its corresponding confidence level, combined with historical email information related to the target email, so as to obtain a risk score for the target email; and to determine a risk label for the target email based on the risk score; the risk label is used to characterize the degree of risk of the target email under the intent type.

[0143] For example, this embodiment provides an email processing system for performing multimodal information perception, feature fusion, intent recognition, and risk assessment on target emails. The email processing system may include a multimodal content extraction module, a feature encoding module, a multimodal fusion module, an intent recognition module, and a risk assessment module. These modules work collaboratively to form a processing flow from target email input to risk assessment result output. The multimodal content extraction module and the feature encoding module can jointly form the feature extraction module.

[0144] The multimodal content extraction module receives the target email and extracts information in multiple modalities from it. This module may include a text extraction unit, an image extraction unit, an attachment extraction unit, and a link extraction unit. The text extraction unit extracts text information from the target email, such as the email subject, body, and header fields. The image extraction unit extracts images embedded in the email body or as attachments. The attachment extraction unit extracts attachment information, such as file type, size, and content characteristics. The link extraction unit extracts link information from the target email, such as URLs in the email body and the domain names they point to.

[0145] The feature encoding module is connected to the multimodal content extraction module and is used to perform feature encoding on content of different modalities to obtain corresponding multimodal feature information. The feature encoding module may include a text encoding unit, an image encoding unit, an attachment encoding unit, and a link encoding unit. The text encoding unit is used to perform semantic feature encoding on text information to obtain text feature information; the image encoding unit is used to perform visual feature encoding on image information to obtain image feature information; the attachment encoding unit is used to perform metadata and content feature encoding on attachment information to obtain attachment feature information; and the link encoding unit is used to perform feature encoding on link information to obtain link feature information.

[0146] A multimodal fusion module is connected to a feature encoding module and is used to perform cross-modal interaction and feature fusion on feature information from different modalities to generate cross-modal interaction information for the target email. The multimodal fusion module may include a cross-modal attention unit and a feature fusion unit. The cross-modal attention unit is used to evaluate the correlation between feature information from different modalities and calculate the correlation weights between features from different modalities. The feature fusion unit is used to perform weighted fusion of feature information from different modalities based on the correlation weights to generate unified cross-modal interaction information.

[0147] The intent recognition module is connected to the multimodal fusion module and is used to classify the intent of the target email based on the cross-modal interaction information to obtain the intent type of the target email and its corresponding confidence level. The intent recognition module may include an intent classification unit, which is implemented, for example, using a multi-layer neural network classifier, to identify whether the target email belongs to at least one intent type among business communication, financial transfer request, data breach, phishing scam, malware propagation, or social engineering attack.

[0148] The risk assessment module is connected to the intent recognition module and is used to assess the risk of the target email under the intent type based on the intent type and its corresponding confidence level, combined with historical email information related to the target email. The risk assessment module may include a risk scoring unit and a risk labeling unit, wherein the risk scoring unit is used to output a risk score for the target email, and the risk labeling unit is used to determine a risk label for the target email based on the risk score.

[0149] Finally, the email processing system can output the intent type, risk score, and risk label corresponding to the target email.

[0150] In some embodiments, the multimodal content extraction module is used to extract information of multiple modalities from the target email. The multimodal information may include at least one of text information, image information, attachment information, and link information. Specifically, the text information may include, for example, the email subject, email body, and email header fields; the email header fields may include sender information, recipient information, and sending time, etc.; the image information may include, for example, images embedded in the email body, image resources in HTML emails, and images as attachments; the attachment information may include, for example, metadata of the attachment and content characteristics of the attachment; the metadata may include filename, file type, file size, MIME (Multipurpose Internet Mail Extensions) type, etc.; the content characteristics may include, for example, file content structure characteristics; and the link information may include, for example, URL links contained in the email, link text, and the domain name information to which the link points.

[0151] In some embodiments, the feature encoding module is used to convert content of different modalities into a unified form of feature vector representation to obtain corresponding multimodal feature information. Specifically, for text information, pre-trained natural language processing models can be used to semantically encode email text, such as using models like BERT (Bidirectional Encoder Representations from Transformers) to generate text feature vectors; for image information, image recognition models can be used to extract visual features of images, such as using models like ResNet (Residual Network) and ViT (Vision Transformer) to generate image feature vectors; for attachment information, metadata and content features of attachments can be encoded, such as extracting file type, file size, extension, file hash value, file header features, etc., to generate attachment feature vectors; for link information, domain name features, path features, and historical reputation features of links can be extracted, such as domain name string length, domain name registration time, TLS (Transport Layer Security) certificate information, etc., to generate link feature vectors.

[0152] In some embodiments, the multimodal fusion module is used to perform cross-modal interaction and feature fusion on feature vectors of different modalities to form a unified multimodal representation of the target email. In an exemplary implementation, an attention mechanism can be used to realize information interaction between features of different modalities. By calculating the correlation weights between features of different modalities, complementary modeling between modalities can be achieved. Based on the correlation weights, the feature vectors of each modality are weighted and fused to generate a unified multimodal representation vector for characterizing the overall features of the target email.

[0153] In some embodiments, the intent recognition module is used to identify the business intent of a target email based on the fused multimodal features. Specifically, a preset intent classification model can be used to classify the multimodal feature vectors. This intent classification model can be implemented, for example, using a multi-layer neural network classifier, to output the intent category corresponding to the target email and its confidence score. The intent category may include, but is not limited to, normal business communication, financial transfer requests, data breach risks, phishing scams, malware propagation, and social engineering attacks.

[0154] In some embodiments, the risk assessment module is used to assess the risk level of the identified email intent. Specifically, it can combine factors such as the intent type of the target email, the corresponding confidence score, and historical behavioral patterns related to the target email to assess the risk of the target email under the intent type, so as to calculate the risk score of the target email; and based on the risk score and a preset risk threshold, assign a corresponding risk label to the target email, such as high risk, medium risk, low risk, or normal.

[0155] For example, the multimodal fusion module can be used to fuse feature information from different modalities. Its input is the feature vectors corresponding to multiple modalities, and its output is a unified multimodal representation vector.

[0156] Specifically, the input to the multimodal fusion module may include a text feature vector V_text, an image feature vector V_image, an attachment feature vector V_attach, and a link feature vector V_link. These feature vectors are used to characterize the feature information of the target email in the text modality, image modality, attachment modality, and link modality, respectively.

[0157] In some embodiments, the multimodal fusion module may first perform cross-modal attention computation between feature vectors of different modalities to evaluate the correlation between features of different modalities. The cross-modal attention computation process may be implemented by a cross-modal attention unit, and its exemplary computation method can be expressed as: A[i, j] = Attention(V_i, V_j), where V_i and V_j represent feature vectors corresponding to any two modalities, and A[i, j] represents the correlation weight between the features of the two modalities, thereby forming an attention weight matrix A.

[0158] After obtaining the attention weight matrix, the multimodal fusion module can perform weighted fusion processing on the feature vectors of each modality based on the attention weights. The weighted fusion process can be implemented by a feature fusion unit, and its exemplary expression can be: V_fused = Σ(A[i, j] × V_i × W_j), where W_j represents the weight parameter associated with the corresponding modal feature vector, and Σ represents the weighted summation operation on different modal features.

[0159] Through the cross-modal attention calculation and feature fusion processing described above, the multimodal fusion module can output a unified multimodal representation vector V_fused. This unified multimodal representation vector is used to comprehensively characterize the feature information of the target email in multiple modalities and serves as the input for subsequent intent classification and risk assessment.

[0160] For example, the intent recognition module can be used to classify the intent of target emails based on the fused multimodal feature vector. Its input can be the unified multimodal feature vector V_fused output by the multimodal fusion module.

[0161] In some embodiments, before inputting the fused multimodal feature vector into the intent classification model, the multimodal feature vector may first undergo preprocessing operations. These preprocessing operations may include, for example, feature normalization and feature dimension adjustment, to ensure that the input features meet the input requirements of the subsequent classification model.

[0162] After feature vector preprocessing, the preprocessed multimodal feature vectors are input into the intent classification model for classification. In some embodiments, the intent classification model can be implemented using a multi-layer neural network classifier. The multi-layer neural network classifier may, for example, include a multi-layer fully connected network structure, wherein each fully connected layer is sequentially connected to a non-linear activation function. For example, the multi-layer neural network classifier may include a first fully connected layer with a ReLU activation function, a second fully connected layer with a ReLU activation function, a third fully connected layer with a ReLU activation function, and a Softmax layer at the output to output the probability distribution corresponding to different intent categories.

[0163] Through the Softmax layer, the intent recognition module can output confidence scores for target emails under multiple preset intent categories. These intent categories may include, for example, normal business communication, financial transfer requests, data breach risks, phishing scams, malware propagation, and social engineering attacks. As an example, the intent recognition module can output classification results in the following format: confidence score of 0.05 for normal business communication, 0.02 for financial transfer requests, 0.01 for data breach risks, 0.92 for phishing scams, 0.00 for malware propagation, and 0.00 for social engineering attacks.

[0164] In the example above, since "phishing scam" has the highest confidence score, the intent recognition module can determine the intent category of the target email as "phishing scam" and output the corresponding confidence score, such as 0.92. The intent category and its corresponding confidence score are used as the intent recognition result for subsequent risk assessment processing.

[0165] In one specific embodiment, the target email is a phishing email. The target email contains multiple modalities, including a text subject line that reads "Urgent: Please process payment as soon as possible," a body containing statements such as "Please click the following link to complete the payment," indicating an urgent payment request; an image embedded in the body displaying the company logo; and a link within the body that is a short link pointing to an external domain.

[0166] In this embodiment, the multimodal content extraction module first performs multimodal content extraction on the target email, obtaining text information, image information, and link information from the target email. Subsequently, the feature encoding module performs feature encoding processing on the extracted different modalities of content.

[0167] During the text feature encoding process, the text encoding unit performs semantic analysis on the email subject and body, identifies high-risk semantic features such as "urgent payment", and the generated text feature vector shows obvious high-risk features.

[0168] During the image feature encoding process, the image encoding unit extracts visual features from the company logo image contained in the email and compares the extracted image features with the corresponding image features in the sender's historical emails. If the image features in the current email do not match the logo features in the historical normal emails, the generated image feature vector will show abnormal features.

[0169] During the link feature encoding process, the link encoding unit parses the short links in the email, detects that the link points to a newly registered external domain name, and the generated link feature vector shows a high risk characteristic.

[0170] In the multimodal feature fusion stage, the multimodal fusion module performs cross-modal interaction analysis on text feature vectors, image feature vectors, and link feature vectors based on an attention mechanism. It discovers significant inconsistencies between different modal features; for example, text content may indicate an urgent payment intent, image content may superficially display a normal company logo, while link features may exhibit high-risk characteristics. Based on these inconsistencies, the fused feature vector generated by the multimodal fusion module as a whole exhibits high-risk characteristics.

[0171] In the intent recognition stage, the intent recognition module classifies the intent of the target email based on the fused multimodal feature vector, identifies the intent type of the target email as "phishing scam", and outputs the corresponding confidence score of 0.92.

[0172] During the risk assessment phase, the risk assessment module combines the intent type and confidence score to assess the risk of the target email, calculates a risk score of 85 for the target email, and assigns a "high-risk" risk label to the target email based on the risk score.

[0173] In another specific embodiment, the target email is a normal business communication email. The target email contains multiple modalities, including a text email with the subject "Project Progress Report," a body email containing a normal description of the current progress of the project, and an attachment consisting of a project report attached to the email, which is a PDF file.

[0174] In this embodiment, the multimodal content extraction module first performs multimodal content extraction on the target email to obtain the text information and attachment information in the target email. Subsequently, the feature encoding module performs feature encoding processing on the extracted different modalities of content respectively.

[0175] During the text feature encoding process, the text encoding unit performs semantic analysis on the email subject and body content, identifies that the text content conforms to the semantic features of normal business communication, and generates a text feature vector that represents normal business features.

[0176] During the attachment feature encoding process, the attachment encoding unit parses the PDF format attachment, detects that the attachment file type is legal and the file format is normal, and finds no abnormal file features. The generated attachment feature vector shows normal features.

[0177] In the multimodal feature fusion stage, the multimodal fusion module fuses the text feature vector and the attachment feature vector. The fusion results show that the consistency between the modal features is good, no abnormal or conflicting features between modalities are detected, and the generated fused feature vector as a whole presents low-risk features.

[0178] During the intent recognition phase, the intent recognition module classifies the target email based on the fused multimodal feature vector, identifies the intent type of the target email as "normal business communication", and outputs the corresponding confidence score of 0.95.

[0179] During the risk assessment phase, the risk assessment module assesses the risk of the target email based on the intent type and its confidence score, calculates a risk score of 10 for the target email, and assigns a "normal" risk label to the target email accordingly.

[0180] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the above methods.

[0181] This application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of any of the above methods.

[0182] The computer program product may be a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the computer program product of the embodiments of this application is not limited thereto, and the computer program product may be any combination of one or more computer-readable media.

[0183] See Figure 3 , Figure 3 This is a structural block diagram of a computer device provided in an embodiment of this application.

[0184] This application also provides a computer device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of any of the above methods.

[0185] The embodiments of this application do not limit the computer device, which may be, for example, a local computer device, a cloud computer device, a distributed computer device, etc.

[0186] The computer device may include: a memory 110, a processor 120, and a communication interface 130. The memory 110, the processor 120, and the communication interface 130 are connected through internal connection paths.

[0187] The memory 110 is used to store computer programs, which in some implementations may include code for implementing the methods of the embodiments of this application.

[0188] The processor 120 executes the computer program stored in the memory 110 to control the communication interface 130 to receive input data and information, and output operation results and other data. In some implementations, when the solutions of the embodiments of this application are implemented by software or firmware, the computer program used to implement the solutions of the embodiments of this application can be stored in the processor 120 and executed by the processor 120.

[0189] The memory 110 may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory may be random access memory (RAM). It should be noted that the memory 110 described herein is intended to include, but is not limited to, any memory of these and other suitable types. As an example, the memory 110 includes random access memory (RAM), cache memory, and read-only memory (ROM). The memory 110 stores a computer program that can be executed by processor 120, causing processor 120 to implement the steps of any of the methods described above.

[0190] The processor 120 can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor, or the processor 120 can be any conventional processor.

[0191] In implementation, each step of the above method can be completed by the integrated logic circuitry of the hardware in the processor 120 or by instructions in software form. The method disclosed in the embodiments of this application can be directly implemented by the hardware processor, or by a combination of hardware and software modules in the processor 120. The software modules can be located in mature storage media in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in the memory 110, and the processor 120 reads the information in the memory 110 and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, detailed descriptions are not provided here.

[0192] In some implementations, in addition to the hardware units described above, computer devices may also include software modules, such as operating systems, basic input / output systems (BIOS), and application software.

[0193] An operating system is used to manage the hardware and / or software resources of a computer device; it is the kernel and foundation of the computer. The operating system handles fundamental tasks such as managing and configuring memory, determining the priority of system resource allocation, controlling input and output devices, operating the network, and managing the file system. To facilitate user operation, most operating systems provide a user interface for interaction with the system.

[0194] The BIOS is used to perform hardware initialization during the power-on boot phase and to provide runtime services for the operating system and applications. In some implementations, the BIOS can also monitor and display processor temperature and execute temperature protection strategies.

[0195] Application software, also known as an application program, can be understood as software written for a specific user application purpose, and is one of the main categories of computer software. For example, application software can be a program used to achieve purposes such as power control and temperature management.

[0196] It is understood that the specific examples in this application are only intended to help those skilled in the art better understand the implementation of this application, and are not intended to limit the scope of protection of this application.

[0197] It is understood that in the various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of this application.

[0198] It is understood that the various implementation methods described in this application can be implemented individually or in combination, and this application does not limit them.

[0199] Unless otherwise stated, all technical and scientific terms used in this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0200] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0201] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the embodiments described above can be referred to the corresponding processes in other embodiments, and will not be repeated here.

[0202] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0203] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the technical solution in this application, depending on actual needs.

[0204] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0205] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, essentially, or the part that contributes to related technologies, or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0206] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An email processing method, characterized in that, include: Multimodal feature extraction is performed based on the target email to obtain multimodal feature information corresponding to the target email; The target email includes at least two of the following: text information, image information, attachment information, and link information; the multimodal feature information includes at least two of the following: text feature information, image feature information, attachment feature information, and link feature information. Cross-modal interaction is performed based on the multimodal feature information to generate cross-modal interaction information for the target email; Intent classification is performed based on the cross-modal interaction information to obtain the intent type of the target email and its corresponding confidence level; the intent type includes at least one of business communication, financial transfer request, data breach, phishing scam, malware propagation and social engineering attack; Based on the intent type and its corresponding confidence level, and combined with historical email information related to the target email, the risk of the target email under the intent type is assessed to obtain a risk score for the target email. The risk label of the target email is determined based on the risk score; The risk label is used to characterize the risk level of the target email under the intent type.

2. The email processing method according to claim 1, characterized in that, The multimodal feature extraction based on the target email includes: Multimodal content extraction is performed on the target email to obtain at least two of the following: text content, image content, attachment content, and link content corresponding to the target email. At least two of the text content, image content, attachment content, and link content are respectively feature-encoded to obtain at least two of the text feature information, image feature information, attachment feature information, and link feature information.

3. The email processing method according to claim 1, characterized in that, The cross-modal interaction based on the multimodal feature information includes: Correlation assessment is performed between feature information of different modes in the multimodal feature information to obtain corresponding correlation assessment information; Based on the relevance assessment information, feature fusion is performed on the feature information of different modalities in the multimodal feature information to generate cross-modal interaction information of the target email.

4. The email processing method according to claim 3, characterized in that, The correlation assessment between feature information of different modalities in the multimodal feature information includes: An attention model is used to evaluate the correlation between features of different modalities in the multimodal feature information, and a correlation weight matrix between features of different modalities is calculated; the correlation weight matrix is ​​used as the correlation evaluation information.

5. The email processing method according to claim 1, characterized in that, The process of determining the risk label of the target email based on the risk score includes: Based on the risk score and multiple preset risk score thresholds, the risk label of the target email is determined; the multiple risk score thresholds are used to divide the value range of the risk score into multiple numerical intervals.

6. The email processing method according to claim 1, characterized in that, The risk label includes at least one of high risk, medium risk, and low risk, and the method further includes: Based on the intent type of the target email and at least one of its corresponding confidence level, risk score, and risk label, a processing strategy model is used to determine the processing strategy corresponding to the target email; the processing strategy includes at least one of interception, isolation, marking, encryption, deletion, forwarding, and notification. The system receives feedback information regarding the processing strategy and, based on the feedback information and at least one of the target email's intent type and its corresponding confidence level, risk score, and risk label, forms training data for training the processing strategy model.

7. The email processing method according to claim 1, characterized in that, The text information includes at least one of email subject, email body, email header fields, and email signature; the email header fields include at least one of sender address, recipient address, sending time, and CC information; and / or, The attachment information includes at least one of the following: file type, file size, and file content characteristics. The file content characteristics include at least one of the following: file header information and file hash information; and / or, The link information includes URL links contained in the email body and the domain information pointed to by the URL links. The domain information includes at least one of the following: domain string length, registration time information, certificate information, path information, and historical reputation information; and / or, The historical email information includes at least one of the following: sender's historical email behavior information, recipient's historical email interaction information, and email time sequence information related to the target email.

8. The email processing method according to claim 1, characterized in that, The risk assessment of the target email under the stated intent type includes: Based on the target email and historical email information related to the target email, risk factors of multiple dimensions are extracted, and a risk value is assigned to each risk factor. Based on the intent type and its corresponding confidence level, the importance of each risk factor is assessed to determine the weight parameter of each risk factor; The risk values ​​of multiple risk factors are weighted and fused according to their corresponding weight parameters to obtain the risk score of the target email.

9. The email processing method according to claim 1, characterized in that, The process of identifying the target email includes: If a preset trigger event is detected for at least one email, each email is identified as the target email; wherein the preset trigger event includes at least one of the following: a new email arrives; a user initiates a manual review request for an email; or the emails scanned by a timed scanning task are not empty.

10. An email processing system, characterized in that, include: The feature extraction module is used to extract multimodal features based on the target email to obtain multimodal feature information corresponding to the target email; The target email includes at least two of the following: text information, image information, attachment information, and link information; the multimodal feature information includes at least two of the following: text feature information, image feature information, attachment feature information, and link feature information. A multimodal fusion module is used to perform cross-modal interaction based on the multimodal feature information to generate cross-modal interaction information of the target email; An intent recognition module is used to classify intents based on the cross-modal interaction information to obtain the intent type of the target email and its corresponding confidence level; the intent type includes at least one of business communication, financial transfer request, data breach, phishing scam, malware propagation and social engineering attack; The risk assessment module is used to assess the risk of the target email under the intent type based on the intent type and its corresponding confidence level, combined with historical email information related to the target email, so as to obtain a risk score for the target email. The risk label of the target email is determined based on the risk score; The risk label is used to characterize the risk level of the target email under the intent type.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the email processing method according to any one of claims 1 to 9.

12. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the email processing method according to any one of claims 1 to 9.

Images