Junk mail identification method and system, computer equipment and storage medium

Abstract

The invention relates to the technical field of junk mail recognition, and discloses a junk mail recognition method and system, computer equipment and a storage medium, and the method comprises the steps: inputting a to-be-recognized mail into a preset mail recognition model for junk mail recognition, and obtaining a recognition result of the to-be-recognized mail; obtaining feedback data of the user for the identification result, and triggering an online learning mechanism according to the feedback data; collecting sample data from a preset vector database to form a training sample, and updating model parameters of the mail recognition model by adopting a variational Bayesian online learning algorithm according to the training sample; and performing junk mail identification by using the updated mail identification model. According to the method, the mail recognition model based on the twin network structure and real-time online learning updating is constructed, so that accurate and efficient junk mail recognition is realized, timely response to user feedback is realized, and personalized requirements of users are met.

Description

Technical Field

[0001] The present invention relates to the technical field of spam identification, and in particular to a spam identification method, system, computer equipment and storage medium. Background Art

[0002] Spam identification is a core issue in the field of email security. With the continuous iteration and update of artificial intelligence and deep learning, this field has also made great progress. However, in actual applications, spam identification is a dynamic game process. Senders will try various ways to bypass the detection of the current model. Various spam variants emerge in an endless stream, which puts high demands on the update of the detection model.

[0003] In the current conventional neural network-based spam identification method, the model is updated after enough new samples are collected within a period of time, and then the old model is retrained and replaced. Therefore, there are problems such as long cycle, slow response, and more manual intervention. In addition, some emails are in a gray area, and their judgment is also related to the user's subjective preference. The current conventional single model obviously cannot meet the personalized needs of users. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a spam identification method, system, computer device and storage medium, which can solve the problems of long update cycle and untimely response of the existing methods, and achieve the effect of accurately and efficiently identifying spam and meeting the personalized needs of users.

[0005] In a first aspect, the present invention provides a method for identifying spam, the method comprising: Inputting the to-be-identified email into a preset email identification model for spam identification, and obtaining an identification result of the to-be-identified email, wherein the email identification model comprises a feature extraction module, a feature identification sub-model, and a result fusion module connected in series in sequence, wherein the feature identification sub-model comprises a basic identification sub-model, a personalized identification sub-model, and a weight allocation sub-model connected in parallel, and the basic identification sub-model and the personalized identification sub-model have the same structure; Obtaining user feedback data on recognition results, and triggering an online learning mechanism based on the feedback data; Collecting sample data from a preset vector database to form training samples, and updating the model parameters of the email recognition model based on the training samples and using a variational Bayesian online learning algorithm; The updated email identification model is used to perform spam identification.

[0006] Furthermore, the feature extraction module includes a text feature extraction module, a non-text feature extraction module and a feature splicing module, the text feature extraction module is constructed based on a lightweight pre-training model, and the non-text feature extraction module is constructed based on a variational autoencoder; The basic recognition sub-model and the personalized recognition sub-model both include a feature interaction layer, a multi-layer perceptron and a normalization layer connected in series in sequence; The weight allocation sub-model is constructed based on a latent variable model.

[0007] Furthermore, the step of inputting the to-be-identified email into a preset email identification model for spam identification includes: Inputting the mail to be identified into the text feature extraction module and the non-text feature extraction module respectively for feature extraction, and inputting the extracted text feature vector and non-text feature vector into the feature splicing module for feature splicing to obtain a feature vector; The feature vectors are respectively input into the basic recognition sub-model and the personalized recognition sub-model for feature interaction and feature recognition to obtain a basic recognition result and a personalized recognition result; Inputting the feature vector into the weight allocation sub-model to perform potential feature analysis to obtain a basic recognition weight and a personalized recognition weight; The basic recognition result, the personalized recognition result, the basic recognition weight and the personalized recognition weight are input into the result fusion module for weighted summation to obtain the recognition result of the mail to be recognized.

[0008] Furthermore, the steps of feature interaction and feature recognition of the basic recognition sub-model and the personalized recognition sub-model include: The feature interaction layer performs feature interaction on the text feature vector and the non-text feature vector, and the multi-layer perceptron and the normalization layer perform feature recognition on the interactive feature vector obtained by the interaction.

[0009] Furthermore, the step of performing feature interaction on the text feature vector and the non-text feature vector through the feature interaction layer includes: Performing linear transformation on the text feature vector and the non-text feature vector respectively to obtain a first column vector and a second column vector; Constructing a second-order interaction matrix according to the first column vector and the second column vector, and concatenating each element in the second-order interaction matrix to obtain an interaction column vector; The text feature vector, the non-text feature vector and the interactive column vector are concatenated to obtain an interactive feature vector.

[0010] Furthermore, the training step of the preset email recognition model includes: Training the feature extraction module and the basic recognition sub-model respectively; Using the trained model parameters of the basic recognition sub-model as the initial model parameters of the personalized recognition sub-model; The initial model parameters of the weight allocation sub-model are determined by random initialization method.

[0011] Furthermore, the step of updating the model parameters of the email recognition model based on the training sample and using a variational Bayesian online learning algorithm includes: Based on variational Bayesian inference, a variational lower bound of the likelihood function of the feature recognition sub-model is constructed, and by introducing a Gaussian prior into the basic recognition sub-model, the variational lower bound is reconstructed to obtain an optimized objective function; According to the training samples, the optimization objective function is iteratively calculated to update the parameters of the feature recognition sub-model.

[0012] In a second aspect, the present invention provides a spam identification system, the system comprising: The mail identification module is used to input the mail to be identified into a preset mail identification model to identify spam mails and obtain the identification result of the mail to be identified. The mail identification model includes a feature extraction module, a feature identification sub-model and a result fusion module connected in series in sequence. The feature identification sub-model includes a basic identification sub-model, a personalized identification sub-model and a weight allocation sub-model connected in parallel, and the basic identification sub-model and the personalized identification sub-model have the same structure; An online learning trigger module is used to obtain user feedback data on recognition results and trigger an online learning mechanism based on the feedback data; A model updating module, used to collect sample data from a preset vector database to form training samples, and update the model parameters of the email recognition model based on the training samples and using a variational Bayesian online learning algorithm; The email identification module is further used to use the updated email identification model to perform spam identification.

[0013] In a third aspect, an embodiment of the present invention further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0014] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.

[0015] The present invention provides a method, system, computer device and storage medium for identifying spam. By constructing an email identification model based on a twin network structure and real-time online learning and updating, the present invention can not only accurately and efficiently dynamically identify various types of spam and their variants, but also can adjust the model in real time according to user feedback, thereby meeting the personalized needs of users. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a schematic flow chart of a method for identifying spam in an embodiment of the present invention; Figure 2 is a schematic diagram of the structure of a mail identification model in an embodiment of the present invention; Figure 3 yes Figure 2 Schematic diagram of the structure of the feature extraction module; Figure 4 yes Figure 2 Schematic diagram of the structure of the basic identification sub-model; Figure 5 is a schematic diagram of the process of online learning of the email recognition model in an embodiment of the present invention; Figure 6 is a schematic diagram of the structure of a spam identification system in an embodiment of the present invention; Figure 7 It is a diagram of the internal structure of a computer device in an embodiment of the present invention. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] See also Figure 1 The first embodiment of the present invention provides a method for identifying spam, which includes steps S10 to S40: Step S10, inputting the to-be-identified email into a preset email identification model for spam identification, and obtaining an identification result of the to-be-identified email, wherein the email identification model comprises a feature extraction module, a feature identification sub-model, and a result fusion module connected in series in sequence, wherein the feature identification sub-model comprises a basic identification sub-model, a personalized identification sub-model, and a weight allocation sub-model connected in parallel, and the basic identification sub-model and the personalized identification sub-model have the same structure; Step S20, obtaining user feedback data on the recognition result, and triggering an online learning mechanism according to the feedback data; Step S30, collecting sample data from a preset vector database to form training samples, and updating the model parameters of the email recognition model based on the training samples and using a variational Bayesian online learning algorithm; Step S40: Use the updated email recognition model to perform spam recognition.

[0019] In the spam identification method provided by the present invention, an email identification model based on a twin network structure and an online learning mechanism is constructed to analyze and identify the input email content. The model architecture is as follows: Figure 2 As shown, the email recognition model of the present invention includes a feature extraction module, a feature recognition sub-model and a result fusion module connected in series in sequence, wherein the feature recognition sub-model includes a basic recognition sub-model, a personalized recognition sub-model and a weight allocation sub-model connected in parallel, and the structures of the basic recognition sub-model and the personalized recognition sub-model are consistent.

[0020] See also Figure 3 In a preferred embodiment, the feature extraction module includes a text feature extraction module, a non-text feature extraction module and a feature splicing module. The text feature extraction module is connected in parallel with the non-text feature extraction module and then in series with the feature splicing module. The text feature extraction module is constructed based on a lightweight pre-training model, and the non-text feature extraction module is constructed based on a variational autoencoder.

[0021] In this embodiment, the text features in the email are extracted by the lightweight pre-trained model tinyBERT. This model has a small model size and low computational complexity while maintaining high performance. The non-text features in the email are extracted by the variational autoencoder VAE. The non-text features here include whether the sender and sender are in the same domain, whether the URL is carried, the proportion of HTML tags, etc., which can be customized according to actual needs. The variational autoencoder includes an encoder and a decoder. The encoder is used to map non-text features to feature space, and the decoder is responsible for restoring non-text feature vectors to original features. Since the non-text feature vector does not need to be restored in this embodiment, after the VAE training is completed, it is only necessary to retain the feature vector used by the encoder to output non-text features. For the extracted text feature vector and non-text feature vector, they are spliced ​​together through the feature splicing layer as the feature vector of the email to be identified.

[0022] For the feature vector output by the feature extraction module, in conventional spam identification methods, most of them use a single model to analyze and identify the feature vector, but this method cannot meet the personalized needs of users. For example, university customers have a higher tolerance for training promotion emails and generally consider them to be normal emails, but corporate users are not like this and will consider them to be spam. Therefore, using a single model to give the same judgment to all customers is obviously unable to meet the diverse needs of users.

[0023] In order to solve this problem, in this embodiment, a basic recognition sub-model and a personalized recognition sub-model are constructed based on the twin network architecture. That is, the model structures of the basic recognition sub-model and the personalized recognition sub-model in this embodiment are the same. Taking the basic recognition sub-model as an example, its sub-model structure is as follows: Figure 4 As shown in FIG. 1 , the basic recognition sub-model includes a feature interaction layer, a multi-layer perceptron and a normalization layer connected in series. The feature interaction layer is mainly used to perform feature interaction on the text features and non-text features in the input feature vector. The feature interaction layer is added before the multi-layer perceptron because in actual applications, it is impossible to determine whether an email is spam based on text features alone. For example, if the sender and receiver of a subsidy email are in the same domain, it is likely to be a normal internal company notification email; but if it is an external domain email, it is likely to be a subsidy phishing email, which requires a comprehensive judgment of text features and non-text features.

[0024] In fact, a simple twin network structure does not have the function of personalized customization, so the conventional twin neural network cannot be directly applied to the present invention. In order to make the model meet the personalized needs of users, this embodiment adds a weight allocation sub-model on the basis of the twin network structure. The weight allocation sub-model is constructed based on the latent variable model. The weights of the recognition results of the two twin recognition sub-models are balanced by introducing latent variables, thereby achieving the robustness and flexibility of the model.

[0025] Based on the above model structure and the functions of each part of the model, the specific steps of spam identification through the email identification model include: Inputting the mail to be identified into the text feature extraction module and the non-text feature extraction module respectively for feature extraction, and inputting the extracted text feature vector and non-text feature vector into the feature splicing module for feature splicing to obtain a feature vector; The feature vectors are respectively input into the basic recognition sub-model and the personalized recognition sub-model for feature interaction and feature recognition to obtain a basic recognition result and a personalized recognition result; Inputting the feature vector into the weight allocation sub-model to perform potential feature analysis to obtain a basic recognition weight and a personalized recognition weight; The basic recognition result, the personalized recognition result, the basic recognition weight and the personalized recognition weight are input into the result fusion module for weighted summation to obtain the recognition result of the mail to be recognized.

[0026] Based on the above recognition steps and model structure, it can be known that in the feature recognition sub-model, feature interaction is performed on the text feature vector and the non-text feature vector through the feature interaction layer, and feature recognition is performed on the interactive feature vector obtained through the multi-layer perceptron and the normalization layer.

[0027] At present, the commonly used methods for the interaction processing of the two types of features include increasing the number of layers of the neural network to implicitly capture the second-order interaction, and constructing the second-order interaction based on the factor decomposition machine direct point multiplication. However, the method of implicitly capturing the second-order interaction by increasing the number of layers of the neural network will increase the amount of calculation, and the interaction effect is limited, while the method based on the factor decomposition machine point multiplication can only obtain a scalar, which will lose a lot of information. Therefore, in this embodiment, the above conventional method is not used for feature interaction, but a method for constructing an interactive feature vector with low calculation amount and high information preservation is provided, and its specific steps include: Performing linear transformation on the text feature vector and the non-text feature vector respectively to obtain a first column vector and a second column vector; Constructing a second-order interaction matrix according to the first column vector and the second column vector, and concatenating each element in the second-order interaction matrix to obtain an interaction column vector; The text feature vector, the non-text feature vector and the interactive column vector are concatenated to obtain an interactive feature vector.

[0028] In this embodiment, assuming that the text feature vector is x1, whose dimension is d1, and the non-text feature vector is x2, whose dimension is d2, firstly, the preset matrices W1 and W2 are used to multiply x1 and x2 respectively for linear transformation, thereby mapping into two column vectors with dimensions d', i.e., the first column vector W1x1 and the second column vector W2x2, wherein d' is a hyperparameter and can be set as appropriate. A larger value means more information retention, and a smaller value means lower computational complexity. Preferably, d' is set to: Among them, the operator Indicates rounding down.

[0029] Multiply the first column vector W1x1 and the second column vector W2x2 by transposing them to obtain a d`-dimensional second-order interaction matrix: Then the element information in the second-order interaction matrix is ​​concatenated into a column vector to obtain the interaction column vector, denoted as x3, and its dimension is denoted as d3: Finally, the text feature vector x1, the non-text feature vector x2 and the interaction column vector x3 are concatenated into a column vector as the interaction feature vector, whose dimension is d1+d2+d3. The interaction feature vector is the input of the multi-layer perceptron MLP.

[0030] In the feature interaction layer of this embodiment, the text feature vector x1 and the non-text feature vector x2 will not perform second-order interaction with themselves. This is because the relevant information has been fully interacted through the feature extraction module. Therefore, not performing self-interaction in the feature interaction layer will not affect the subsequent recognition effect. At the same time, not performing self-interaction also reduces the amount of calculation.

[0031] MLP is a traditional feedforward neural network. In this embodiment, since its input data has been interactively retained with effective features, MLP itself does not need many layers, such as two or three layers to meet the recognition requirements, which also makes the overall parameter amount and calculation amount of the model small. The final normalization layer uses the softmax function to give the probability of the email being a preset category, such as normal, ordinary spam, phishing email, etc. For the multi-layer perceptron and normalization layer, refer to the conventional structure, which will not be repeated here.

[0032] In this embodiment, the weight allocation sub-model is constructed using a latent variable model, which is a neural network that models the potential features between data by introducing latent variables (invisible virtual variables). Common latent variable models include hidden Markov models, Boltzmann machines, and diffusion models. The input of the weight allocation sub-model is the feature vector extracted by the feature extraction model, and it outputs a probability, which is used to fine-tune and control the coupling weights of the basic recognition sub-model and the personalized recognition sub-model, and is used to quantitatively characterize the user's preferences.

[0033] Generally speaking, the basic recognition sub-model and the personalized recognition sub-model form a twin structure. The two are coupled with each other. Under the control of the weight allocation sub-model, the final recognition result is output through the weighted summation of the result fusion module.

[0034] Based on the above model structure, the present invention adopts a staged training method to train the model. First, the model is initially trained, that is, offline training, and the trained model is used to perform email recognition. The steps of the initial training include: Training the feature extraction module and the basic recognition sub-model respectively; Using the trained model parameters of the basic recognition sub-model as the initial model parameters of the personalized recognition sub-model; The initial model parameters of the weight allocation sub-model are determined by random initialization method.

[0035] In this embodiment, the tinyBERT in the feature extraction module is a trained model and does not need to be trained again. After VAE is trained once, its parameters will not change in the subsequent process and it is used as a fixed non-text feature extractor. The basic recognition sub-model can be trained separately using conventional back propagation in the initial training stage. The model parameters of the basic recognition sub-model after training are used as the initial model parameters of the personalized recognition sub-model. At the same time, the weight allocation sub-model does not need to be trained at this stage, and its parameters can be determined by random initialization. This is because in the initial training stage, the basic recognition sub-model and the personalized recognition sub-model are models with exactly the same structure and parameters, so their weights will not affect the final output results.

[0036] Then, the email recognition model that has completed the initial training is provided to the user for spam identification, and the user's feedback information on the identification results is obtained. The feedback information is used to trigger the next stage of training. In this embodiment, the second stage of training adopts an online learning mechanism. The feedback information here refers to the feedback from the user during use. For example, an email will enter the user's inbox after being judged as normal by the model. If the user thinks that the email is spam or does not want to receive similar emails again due to preference issues, it will be moved to the trash can, thereby obtaining training samples and labels. Of course, more sample data cannot be obtained only through user feedback information. Therefore, in this embodiment, two sources of online learning samples are provided, that is, in addition to user feedback information, a vector database is also established. The vector database is a database that stores input feature vectors x and labels y, which is shared by all users and updated and maintained by the anti-spam service provider in an asynchronous manner.

[0037] By introducing and maintaining the vector database, this embodiment can conveniently synchronize the latest monitored objective misjudgment and missed judgment cases to each user. At the same time, since the database stores vectors instead of original emails, privacy issues are also avoided. It should be noted that the vector database in this embodiment is used in the learning stage, and its update and maintenance process is asynchronous with the online learning process.

[0038] When the online learning mechanism is triggered, training samples are formed by collecting sample data from the preset vector database. The collection methods include: based on cosine similarity, matching samples in the vector database that are similar to user feedback samples, adding them to the training sample set for data enhancement; and randomly sampling samples in the vector database, adding them to the training sample set to ensure data diversity. The total number of training samples can be determined according to resource conditions, and there is no excessive restriction here.

[0039] Since the latent variable model is introduced as the weight allocation submodel in the present invention, a variational Bayesian online learning algorithm is used in the online learning stage of the model to optimize and update the parameters of the model, and the specific steps include: Based on variational Bayesian inference, a variational lower bound of the likelihood function of the feature recognition sub-model is constructed, and by introducing a Gaussian prior into the basic recognition sub-model, the variational lower bound is reconstructed to obtain an optimized objective function; According to the training samples, the optimization objective function is iteratively calculated to update the parameters of the feature recognition sub-model.

[0040] In this embodiment, variational Bayes is an optimization algorithm for latent variable models, which indirectly maximizes the likelihood function by optimizing the variational lower bound. For the above model, the present invention constructs a corresponding online iterative optimization algorithm based on variational Bayes. Specifically, assuming that the sample input feature is x, the sample label is y, the latent variable in the weight allocation submodel is z, and the parameters of the weight allocation submodel, the basic recognition submodel, and the personalized recognition submodel are respectively recorded as , so the parameters of the feature recognition sub-model are recorded as , the parameters obtained by training the basic recognition sub-model in the first stage are recorded as , then the model parameters of the basic recognition sub-model and the personalized recognition sub-model in the first stage are .

[0041] The likelihood function of the feature recognition sub-model can be expressed as: in, is the likelihood function of the weight distribution sub-model. When z is 1, it corresponds to the weight of the basic recognition sub-model. When z is 2, it corresponds to the weight of the personalized recognition sub-model. Accordingly, the likelihood function of the twin network is expressed as: in, They represent the likelihood functions of the basic recognition sub-model and the personalized recognition sub-model respectively.

[0042] Parameters of the basic identification submodel The prior distribution of is expressed as: Where N represents the Gaussian distribution function, that is The mean is , the variance is Gaussian distribution of. Among them, I is The identity matrix of the same dimension, is a hyperparameter used to control the degree of deviation of the basic model during online learning. It should be noted that, in this embodiment, p(*) represents the probability density function of the corresponding variable, and q(*) represents the approximate distribution of the latent variable z.

[0043] Due to the existence of hidden variables, it is necessary to construct a learning algorithm based on variational Bayes to construct the model. Considering that the feature vector x only appears in the conditional term, for the sake of simplicity, x is omitted in the following description.

[0044] Introduce the approximate distribution function q(z) of the latent variable z to rewrite the log-likelihood function , we can get: It should be noted that the parameter q appearing in the formula in this embodiment and the function q in the formula description all refer to the approximate distribution function q(z).

[0045] Find the expectation of the approximate distribution function q(z) on both sides of the equation , the left side can be expressed as: The right side can be expressed as: Since the second term on the right side of the equation is in the form of KL divergence, the expected value on the right side can be equivalent to the function The sum of the KL divergence term and the KL divergence term, and since the KL divergence term is always greater than or equal to 0, the log-likelihood function can be expressed as: It can be seen that the above formula is about the function q(z) and the parameter So the function that maximizes It is a functional optimization problem. By solving such a functional optimization problem, the log-likelihood function can be indirectly maximized. .

[0046] Furthermore, we consider the prior distribution of the parameters of the basic identification submodel , reconstruct the optimization objective as: in, With hyperparameters Related constants, operators Indicates the calculation of the second norm. From the above derivation, we can see that introducing the Gaussian prior is equivalent to Perform L2 regularization.

[0047] From another perspective, considering: When optimized, right is a constant and can be ignored, and let The optimization objective constructed above can be obtained. That is, the above optimization objective is also equivalent to the parameters of the weight distribution sub-model and the personalized recognition sub-model. Take the uninformative prior and the parameters of the basic identification sub-model Maximum a posteriori estimation (MAP) with Gaussian prior.

[0048] Maximize the objective function using the coordinate ascent method , we can get the function q and parameter The iterative calculation formula is as follows. Assuming that the maximum number of iterations is n, taking the calculation of the i+1 step from the i-th step as an example, the calculation steps include: (1) Fixed for , optimize the function q: We can get: (2) Fixed q is ,optimization : We can get: Use gradient ascent to iterate and optimize the above three expressions for m iterations to get: Taking the calculation of step j+1 from step j as an example (j=0,...,m-1), we can get: in, is the step length.

[0049] Based on the above steps, the overall process of online learning in this embodiment is as follows: Figure 5 As shown, by optimizing the approximate distribution of latent variables, the model parameters of the feature recognition sub-model are iteratively optimized. When the calculation converges or reaches the maximum number of iterations, the optimal model parameters can be obtained. It can be seen that after the parameters of the feature extraction module are trained in the first stage, they can be fixed and no second stage training is required. In the second stage of online learning training, the model parameters of the feature recognition sub-model, namely the basic recognition sub-model, the personalized recognition sub-model and the weight allocation sub-model are mainly optimized to obtain an updated email recognition model.

[0050] The updated email recognition model is then used to identify spam until the model's online learning mechanism is triggered again, and the model is updated again through the second stage of online learning. The online learning mechanism of this embodiment can achieve timely response to user feedback and continuous model updates, thereby ensuring the accuracy, timeliness and flexibility of the email recognition model.

[0051] The present invention provides a spam identification method which is optimized in terms of network module design, model architecture, online learning sample generation, training method, and online optimization algorithm. It not only realizes accurate and efficient spam identification, but also realizes timely response to user feedback, thus meeting the personalized needs of users.

[0052] Specifically, in terms of network module design, the present invention uses tinyBERT to extract text features and VAE to extract non-text features, and proposes a second-order interactive calculation method for text features and non-text features, which effectively improves the accuracy and computational efficiency of subsequent recognition.

[0053] In terms of model architecture, the present invention proposes a twin network architecture of the basic recognition sub-model and the personalized recognition sub-model, and introduces a hidden variable model to finely control the coupling between the two. Under this architectural design, the basic recognition sub-model, as a benchmark model, reflects the standards of all users for email judgment at a macro level, and provides a stable output for the entire model by introducing the Gaussian prior of the parameters during online learning, so that in the face of possible erroneous feedback and misoperation by users, the model collapse is avoided and the robustness of the entire model is maintained; the personalized recognition sub-model, as a personalized model, improves the objectively existing false omissions through online learning, and at the same time meets the personalized needs of users, thereby effectively improving the user experience; the weight distribution sub-model, as the central controller of the basic recognition sub-model and the personalized recognition sub-model, plays the role of model coupling, and at the same time, through the second stage of online learning, the personalized characteristics of users are quantified to meet the diverse needs of users.

[0054] In terms of sample generation, the present invention constructs a vector database and combines it with an online sample generation method based on user feedback. This ensures sample diversity and timeliness while taking privacy into account, thus effectively supporting the online learning of the model.

[0055] In terms of training method and optimization algorithm, the present invention is based on the design of twin network architecture. In the first stage, only the basic recognition sub-model needs to be trained without training the weight allocation sub-model, thereby realizing the cold start of the weight allocation sub-model in the second stage. Moreover, on the basis of variational Bayes, Gaussian prior is introduced only to the basic recognition sub-model and the variational lower bound is reconstructed to constrain the update amplitude of the basic recognition sub-model, thereby ensuring the robustness of the model.

[0056] See also Figure 6 Based on the same inventive concept, a spam identification system proposed in the second embodiment of the present invention includes: The mail identification module 10 is used to input the mail to be identified into a preset mail identification model to identify spam mails and obtain the identification result of the mail to be identified. The mail identification model includes a feature extraction module, a feature identification sub-model and a result fusion module connected in series in sequence. The feature identification sub-model includes a basic identification sub-model, a personalized identification sub-model and a weight allocation sub-model connected in parallel, and the basic identification sub-model and the personalized identification sub-model have the same structure. The online learning trigger module 20 is used to obtain user feedback data on the recognition result and trigger the online learning mechanism according to the feedback data; A model updating module 30, for collecting sample data from a preset vector database to form training samples, and updating the model parameters of the email recognition model according to the training samples and using a variational Bayesian online learning algorithm; The email identification module 10 is further configured to use the updated email identification model to perform spam identification.

[0057] The technical features and technical effects of the spam identification system proposed in the embodiment of the present invention are the same as those of the method proposed in the embodiment of the present invention, and are not described in detail here. Each module in the above spam identification system can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each of the above modules.

[0058] In addition, an embodiment of the present invention further provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0059] See also Figure 7, an internal structure diagram of a computer device in an embodiment, the computer device can specifically be a terminal or a server. The computer device includes a processor, a memory, a network interface, a display and an input device connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a spam identification method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0060] It can be understood by those skilled in the art that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computing device may include more or fewer components than those shown in the figure, or combine certain components, or have the same component arrangement.

[0061] In addition, an embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are implemented.

[0062] In summary, the embodiments of the present invention propose a method, system, computer device and storage medium for identifying spam. The method obtains an identification result of the email to be identified by inputting the email to be identified into a preset email identification model, wherein the email identification model includes a feature extraction module, a feature identification sub-model and a result fusion module connected in series in sequence, and the feature identification sub-model includes a basic identification sub-model, a personalized identification sub-model and a weight allocation sub-model connected in parallel, and the structure of the basic identification sub-model and the personalized identification sub-model are consistent; user feedback data on the identification result is obtained, and an online learning mechanism is triggered according to the feedback data; sample data is collected from a preset vector database to form a training sample, and the model parameters of the email identification model are updated according to the training sample and a variational Bayesian online learning algorithm; and the updated email identification model is used to identify spam. The present invention performs spam identification through an email identification model based on a twin network architecture, and updates the model based on a variational Bayesian online learning algorithm. It not only achieves accurate and efficient spam identification, but also achieves timely response to user feedback, meets the personalized needs of users, and further ensures the accuracy, efficiency, flexibility and stability of spam identification.

[0063] Each embodiment in this specification is described in a progressive manner, and the same or similar parts of each embodiment can be directly referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. It should be noted that the technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above-mentioned embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0064] The above-mentioned embodiments only express several preferred implementation modes of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in the technical field, several improvements and substitutions can be made without departing from the technical principles of the present invention, and these improvements and substitutions should also be regarded as the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be based on the protection scope of the claims.

Claims

1. A method for identifying spam, characterized in that: include: Inputting the to-be-identified email into a preset email identification model for spam identification, and obtaining an identification result of the to-be-identified email, wherein the email identification model comprises a feature extraction module, a feature identification sub-model, and a result fusion module connected in series in sequence, wherein the feature identification sub-model comprises a basic identification sub-model, a personalized identification sub-model, and a weight allocation sub-model connected in parallel, and the basic identification sub-model and the personalized identification sub-model have the same structure; Obtaining user feedback data on recognition results, and triggering an online learning mechanism based on the feedback data; Collecting sample data from a preset vector database to form training samples, and updating the model parameters of the email recognition model based on the training samples and using a variational Bayesian online learning algorithm; The updated email identification model is used to perform spam identification.

2. The method for identifying spam according to claim 1, characterized in that: The feature extraction module includes a text feature extraction module, a non-text feature extraction module and a feature splicing module, wherein the text feature extraction module is constructed based on a lightweight pre-training model, and the non-text feature extraction module is constructed based on a variational autoencoder; The basic recognition sub-model and the personalized recognition sub-model both include a feature interaction layer, a multi-layer perceptron and a normalization layer connected in series in sequence; The weight allocation sub-model is constructed based on a latent variable model.

3. The method for identifying spam according to claim 2, characterized in that: The step of inputting the to-be-identified email into a preset email identification model for spam identification comprises: Inputting the mail to be identified into the text feature extraction module and the non-text feature extraction module respectively for feature extraction, and inputting the extracted text feature vector and non-text feature vector into the feature splicing module for feature splicing to obtain a feature vector; The feature vectors are respectively input into the basic recognition sub-model and the personalized recognition sub-model for feature interaction and feature recognition to obtain a basic recognition result and a personalized recognition result; Inputting the feature vector into the weight allocation sub-model to perform potential feature analysis to obtain a basic recognition weight and a personalized recognition weight; The basic recognition result, the personalized recognition result, the basic recognition weight and the personalized recognition weight are input into the result fusion module for weighted summation to obtain the recognition result of the mail to be recognized.

4. The method for identifying spam according to claim 3, characterized in that: The steps of feature interaction and feature recognition of the basic recognition sub-model and the personalized recognition sub-model include: The feature interaction layer performs feature interaction on the text feature vector and the non-text feature vector, and the multi-layer perceptron and the normalization layer perform feature recognition on the interactive feature vector obtained by the interaction.

5. The method for identifying spam according to claim 4, characterized in that: The step of performing feature interaction on the text feature vector and the non-text feature vector through the feature interaction layer comprises: Performing linear transformation on the text feature vector and the non-text feature vector respectively to obtain a first column vector and a second column vector; Constructing a second-order interaction matrix according to the first column vector and the second column vector, and concatenating each element in the second-order interaction matrix to obtain an interaction column vector; The text feature vector, the non-text feature vector and the interactive column vector are concatenated to obtain an interactive feature vector.

6. The method for identifying spam according to claim 2, characterized in that: The training steps of the preset email recognition model include: Training the feature extraction module and the basic recognition sub-model respectively; Using the trained model parameters of the basic recognition sub-model as the initial model parameters of the personalized recognition sub-model; The initial model parameters of the weight allocation sub-model are determined by random initialization method.

7. The method for identifying spam according to claim 2, characterized in that: The step of updating the model parameters of the email recognition model according to the training sample and using the variational Bayesian online learning algorithm includes: Based on variational Bayesian inference, a variational lower bound of the likelihood function of the feature recognition sub-model is constructed, and by introducing a Gaussian prior into the basic recognition sub-model, the variational lower bound is reconstructed to obtain an optimized objective function; According to the training samples, the optimization objective function is iteratively calculated to update the parameters of the feature recognition sub-model.

8. A spam identification system, characterized in that: include: The mail identification module is used to input the mail to be identified into a preset mail identification model to identify spam mails and obtain the identification result of the mail to be identified. The mail identification model includes a feature extraction module, a feature identification sub-model and a result fusion module connected in series in sequence. The feature identification sub-model includes a basic identification sub-model, a personalized identification sub-model and a weight allocation sub-model connected in parallel, and the basic identification sub-model and the personalized identification sub-model have the same structure; An online learning trigger module is used to obtain user feedback data on recognition results and trigger an online learning mechanism based on the feedback data; A model updating module, used to collect sample data from a preset vector database to form training samples, and update the model parameters of the email recognition model based on the training samples and using a variational Bayesian online learning algorithm; The email identification module is further configured to use the updated email identification model to perform spam identification.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images