Training method for short message content auditing model, electronic equipment, computer program product and short message auditing method

Abstract

The invention provides a training method for a short message content auditing model, electronic equipment, a computer program product and a short message auditing method. The training method for the short message content auditing model comprises the steps of obtaining sample data based on historical data; and based on the data volume of the sample data, selecting a vector database, a logistic regression training model or a naive Bayes training model to process the corresponding sample data so as to obtain a first training model, a second training model and a third training model, and obtaining an auditing model based on the first training model, the second training model and the third training model. The purposes of reducing the maintenance cost, improving the flexibility, accurately and quickly processing complex contents and avoiding omission are achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of SMS review, and in particular, relates to a training method for a SMS content review model, an electronic device, a computer program product, and a SMS review method. Background Art

[0002] With the development of mobile communication technology, text messages have become one of the important ways of communication in people's daily lives. However, as the number of text messages increases, some inappropriate content is also included, such as spam and fraudulent information. Traditional audit methods often rely on setting rules and keywords in advance in the program for text audit or manual review, which has high maintenance costs, poor flexibility, difficulty in handling complex content, low efficiency, and easy omissions. Summary of the invention

[0003] In response to the problems existing in the prior art, the present invention provides a training method, electronic device, computer program product and SMS review method for a text message content review model, which at least partially solve the problems existing in the prior art such as high maintenance cost, poor flexibility, difficulty in handling complex content, low efficiency and easy omissions.

[0004] In a first aspect, an embodiment of the present disclosure provides a training method for a text message content review model, comprising: Get sample data based on historical data; Acquire a first data volume of sample data, and if the first data volume is less than a first set data volume, process the sample data using a vector database to obtain a first training model; If the first data volume is not less than the first set data volume, the sample data is subjected to similarity deduplication processing to obtain a second data volume after similarity deduplication; if the second data volume is greater than the second set data volume, the sample data after similarity deduplication is processed using a logistic regression training model to obtain a second training model; If the second data volume is not greater than the second set data volume, the naive Bayes training model is used to process the sample data before similarity deduplication, thereby obtaining a third training model; An audit model is obtained based on the first training model, the second training model and the third training model.

[0005] Optionally, obtaining sample data based on historical data includes: Load the historical data within the set time period as training samples. After loading, determine the ratio of positive and negative samples. If the ratio is unbalanced, continue to load the data before the set time period.

[0006] Optionally, obtaining sample data based on historical data includes: excluding junk text messages in the historical data; that is, negative samples of the historical data include rejected text message data but do not include junk text message data.

[0007] Optionally, obtaining sample data based on historical data includes: excluding SMS data with at least two different review results for the same SMS.

[0008] Optionally, before the step of using a logistic regression training model to process the sample data after similarity deduplication, the step includes determining whether the positive and negative samples of the sample data after similarity deduplication meet the requirements. If not, oversampling or undersampling is selected to process the sample data after similarity deduplication.

[0009] Optionally, the step of using the naive Bayes training model to process the sample data before similarity deduplication includes: judging whether the positive and negative samples of the sample data before similarity deduplication meet the requirements, and if not, selecting undersampling to process the sample data before similarity deduplication.

[0010] Optionally, after the step of obtaining the audit model based on the first training model, the second training model and the third training model, the step further includes updating the obtained audit model, backing up the audit model before the update during the update, and during the update time period, when data is received, determining whether the current audit model status is available, and if not available, using the backed-up audit model before the update to process the received data.

[0011] In a second aspect, an embodiment of the present disclosure further provides an electronic device, the electronic device comprising: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any training method for a text message content review model described in the first aspect.

[0012] In a third aspect, an embodiment of the present disclosure further provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements any training method for a text message content review model as described in the first aspect.

[0013] In a fourth aspect, the embodiments of the present disclosure also provide a text message review method, using a review model obtained by any of the training methods described in the first aspect, and the review results of the review model are stored in a cache. When new review data is received, the new review data is first compared with the review results in the cache. If the review results in the cache exist in the same data as the new review data, the review results of the new review data are directly obtained based on the review results in the cache.

[0014] The present invention provides a training method for a text message content review model, an electronic device, a computer program product, and a text message review method. The training method for a text message content review model selects a corresponding training model according to the amount of sample data to obtain a review model, so that the model training is fast and accurate, and the trained review model is used to review text messages, thereby achieving the purpose of reducing maintenance costs, improving flexibility, accurately and quickly processing complex content, and avoiding omissions. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The above and other objects, features and advantages of the present disclosure will become more apparent through a more detailed description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, wherein like reference numerals generally represent like components throughout the exemplary embodiments of the present disclosure.

[0016] Figure 1 A flowchart of a training method for a text message content review model provided in an embodiment of the present disclosure; Figure 2 A flowchart of a text message review provided in an embodiment of the present disclosure; Figure 3 A functional block diagram of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0017] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0018] It should be clear that the following embodiments of the present disclosure are described by specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other in the absence of conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present disclosure.

[0019] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present disclosure, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein may be used to implement this device and / or practice this method.

[0020] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. The illustrations only show components related to the present disclosure rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0021] Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects described may be practiced without these specific details.

[0022] Faiss is a library developed by Facebook AI Research that is specifically designed for efficient similarity search and dense vector clustering. Faiss is very efficient in dealing with large-scale vector search problems and is a very important tool in the field of machine learning and big data.

[0023] Main features: Efficiency: Faiss provides a variety of algorithms and data structures to accelerate the vector search process, including CPU and GPU-based implementations.

[0024] Scalability: It can process hundreds of millions of vectors and supports distributed search.

[0025] Versatility: Supports multiple distance calculation methods, such as Euclidean distance, inner product, etc.

[0026] Multi-language interface: Although Faiss is written in C++, it provides interfaces for languages ​​such as Python and MATLAB.

[0027] Main scenarios and problems solved: Recommendation system: Problem: In the recommendation system, we need to find a set of items that are most similar to the user's historical behavior or interests. Solution: Faiss can quickly find the most similar items in a large number of item vectors, thereby providing personalized recommendations for users.

[0028] Image Recognition and Retrieval: Problem: In an image retrieval system, we need to find similar images in the database based on the query image. Solution: By converting the image into a vector and using Faiss for similarity search, we can quickly find matching images.

[0029] Speech Recognition: Problem: In speech recognition, speech signals need to be converted into text, which usually involves finding matches in a large number of speech feature vectors. Solution: Faiss can be used to speed up this process and improve the efficiency of speech recognition.

[0030] Natural Language Processing (NLP): Problem: In NLP tasks, such as text similarity analysis or question answering systems, we need to find a piece of text that is most similar to the query text. Solution: This problem can be solved efficiently by converting the text into a vector representation and using Faiss for similarity search.

[0031] Security monitoring: Problem: In a video surveillance system, multiple targets need to be identified and tracked. Solution: Through feature vector matching, Faiss can help the system quickly identify the targets in the surveillance image.

[0032] Gene Data Analysis: Problem: In gene sequence analysis, it is necessary to find similar sequences in a large number of gene expression vectors. Solution: Faiss can handle high-dimensional data and is suitable for efficient searching in gene databases.

[0033] Audio Fingerprint Technology: Problem: In audio recognition, it is necessary to identify and match audio clips. Solution: By converting the audio into a vector and searching with Faiss, matching audio can be found quickly.

[0034] Faiss has been widely used in machine learning and big data due to its efficiency and versatility. By providing fast and accurate vector search capabilities, Faiss has greatly promoted the development of applications that require large-scale similarity search.

[0035] The Naive Bayes classifier is a simple probabilistic classifier based on Bayes' theorem, which assumes that each feature is independent of each other. Although this assumption is often not true in reality, the Naive Bayes classifier still shows amazing effectiveness and efficiency in many practical applications.

[0036] Basic principle: Bayes' theorem describes the probability of another event occurring given that another event has occurred. For classification problems, the naive Bayes classifier uses the following formula to calculate the probability of the target variable yy given the input feature xx:

[0037] P(y∣x1,x2,...,xn)=P(y)∏i=1nP(xi∣y)P(x1,x2,...,xn)P(y∣x1,x2,...,xn)=P(x1,x2,...,xn)P(y)∏i=1nP(xi∣y), Where: P(y)P(y) is the prior probability of class label yy. P(xi|y)P(xi|y) is the conditional probability of feature xixi given class label yy. P(x1,x2,...,xn)P(x1,x2,...,xn) is the joint probability of all features, which is usually ignored in classification because it is constant for all classes.

[0038] Features: Simple: The model of the naive Bayes classifier is simple, easy to understand and implement.

[0039] Efficient: When the feature independence assumption holds, the computation is very efficient.

[0040] Requires fewer samples: It can perform well even with less data.

[0041] Application scenarios: Naive Bayes classifier is suitable for text classification, spam detection, disease prediction and other fields.

[0042] Advantages and Disadvantages: Advantages: The algorithm is simple and easy to implement. It requires a small number of samples and can handle multi-classification problems. It has a fast prediction speed and is suitable for real-time systems.

[0043] Disadvantages: The feature independence assumption limits its accuracy. It is very sensitive to the expression form of the input data.

[0044] Logistic Regression is a predictive analysis method widely used in statistics and machine learning, especially in binary classification problems. Despite the name "regression", it is actually a classification algorithm used to predict whether an event will occur or not.

[0045] Basic principle: Logistic regression predicts the probability of an event by mapping the output of the linear regression model to between 0 and 1 using a logistic function (usually the Sigmoid function). The formula for the Sigmoid function is:

[0046] σ(z)=11+e−zσ(z)=1+e−z1, Among them, zz is the linear combination of the model, that is, z=β0+β1x1+β2x2+...+βnxnz=β0+β1x1+β2x2+...+βnxn, ββ is the coefficient, and xx is the feature.

[0047] Features: Output interpretation: The output value can be interpreted as the probability of belonging to a certain class. Binary classification: Mainly used for binary classification problems, but can also be extended to multi-classification problems. Output range: The output value is between 0 and 1.

[0048] Application scenarios: Logistic regression is applicable to a variety of binary classification problems, such as: Credit scoring models: predicting whether an individual is at risk of defaulting. Medical diagnosis: predicting the presence or absence of a disease. Spam detection: determining whether an email is spam. Customer churn prediction: predicting whether a customer will stop using a service.

[0049] Advantages: The model is simple, easy to implement and interpret. The output is easy to interpret as probabilities. It is computationally efficient.

[0050] Disadvantages: It cannot handle nonlinear problems well. When the feature space is large, the performance may degrade. For nonlinear relationships, feature engineering is required to extract useful features.

[0051] 3. Oversampling and undersampling: Oversampling and undersampling are two common techniques for dealing with imbalanced datasets.

[0052] Oversampling is a method of balancing a dataset by increasing the number of minority class samples. Common oversampling methods include:

[0053] Random oversampling: Increasing the number of minority class samples by randomly copying them, but may lead to overfitting because it does not introduce new information.

[0054] SMOTE (Synthetic Minority Over-sampling Technique): Generates new samples by interpolating between minority class samples, thereby reducing the risk of overfitting.

[0055] ADASYN: Generates synthetic samples adaptively according to the classification difficulty of samples, and generates more new samples for samples with high classification difficulty.

[0056] The advantages of oversampling include the ability to increase the size of the dataset, improve the performance of the model, solve the problem of class imbalance in the dataset, avoid information loss, and improve the performance of the model. But it may also lead to overfitting, because the generated new samples may introduce noise, may cause a shift in the sample distribution, and may increase the requirements for computing time and storage space.

[0057] Undersampling is a method of balancing a dataset by reducing the number of samples in the majority class. Common undersampling methods include:

[0058] Random undersampling: Randomly delete majority class samples to reduce their number to the same or similar to that of the minority class, but a lot of useful information may be lost, resulting in a decrease in model performance.

[0059] Cluster Centroids: Cluster the majority class samples into several clusters through a clustering algorithm, and replace the original samples with the centers of the clusters, thereby reducing the number of majority class samples.

[0060] Tomek Links: This method reduces the number of majority class samples by removing the pairs of majority class samples that are closest to minority class samples (Tomek links). This method not only reduces the majority class samples but also removes the noise samples in the dataset.

[0061] The advantage of undersampling is that it can reduce the size of the data set, thereby reducing the need for computing time and storage space, reducing the complexity of the model, avoiding overfitting, balancing the category distribution, and thus improving the accuracy of the model. However, it may also lead to information loss. When the number of minority category samples in the data set is too small, undersampling may lead to underfitting and may introduce randomness, resulting in different results obtained by different undersampling methods.

[0062] Whether to choose oversampling or undersampling depends on the specific dataset and problem. If there are very few minority class samples, directly deleting majority class samples will result in too much information loss, so oversampling is suitable. If there are very many majority class samples, randomly copying minority class samples will result in overfitting, so undersampling is suitable. Sometimes, oversampling and undersampling can be combined to further improve model performance.

[0063] Existing SMS review systems generally adopt rule-based text review solutions and manual review. They mainly rely on predefined rules and keyword lists to detect inappropriate content. When encountering a large amount of complex SMS content that is difficult to handle, manual review is required, which is a very heavy workload.

[0064] shortcoming: (1) High maintenance cost: As language changes and develops, rules and keyword lists need to be updated frequently to adapt to new forms of inappropriate content.

[0065] (2) Poor flexibility: Rules usually target known forms of bad content and may not be able to effectively detect newly emerging forms.

[0066] (3) False positives and false negatives: Limited understanding of variants, synonyms, and context may lead to false positives (incorrectly flagging normal content) or false negatives (failing to identify objectionable content).

[0067] (4) Difficulty in handling complex content: Rule-based methods may have difficulty accurately judging language content that contains complex context, metaphors, or irony.

[0068] like Figure 1 As shown, this embodiment discloses a training method for a text message content review model, including: Get sample data based on historical data; Historical data comes mainly from two places: 1. The review record table stores the content and status (review passed, review rejected) of all the manually reviewed text messages in history, which are used as labeled positive and negative samples for training.

[0069] 2. Historically accumulated spam messages such as fraud and illegal content.

[0070] Because of the large amount of data, the data is not loaded from MySQL every time training is performed. Instead, the data is queried from the database in increments and placed on the server at regular intervals. Therefore, each time training is performed, the text data accumulated on the server is used (the reading speed is very fast). Compared with directly querying from the MySQL database, the speed is increased by dozens or hundreds of times.

[0071] During the training, SMS content is mainly divided into the following types: industry, marketing, light reminder, heavy reminder, mutual finance, finance, spam SMS, etc. It supports the expansion of more types and can be easily expanded to add more types.

[0072] Depending on the type, different types of machine learning models will be trained to ensure higher accuracy.

[0073] Acquire a first data volume of sample data, and if the first data volume is less than a first set data volume, process the sample data using a vector database to obtain a first training model; In specific application scenarios, if the amount of data in a certain type of SMS is too small, at or below 10,000, a vector database such as the Faiss vector database can be directly used. The types include industry, marketing, light reminder, heavy reminder, mutual finance, finance, junk SMS, etc.

[0074] If the first data volume is not less than the first set data volume, the sample data is subjected to similarity deduplication processing to obtain a second data volume after similarity deduplication; if the second data volume is greater than the second set data volume, the sample data after similarity deduplication is processed using a logistic regression training model to obtain a second training model; If the second data volume is not greater than the second set data volume, the naive Bayes training model is used to process the sample data before similarity deduplication, thereby obtaining a third training model; The amount of data after similarity removal is not large enough, at the level of hundreds of thousands, and the model will choose the naive Bayes training model, otherwise it will use the logistic regression training model.

[0075] When using Naive Bayes to train the model, samples with similarity-deduplication are used for training, and logistic regression uses sample data without similarity-deduplication.

[0076] Naive Bayes and logistic regression are chosen because these two models are faster to train and can be trained and updated faster.

[0077] An audit model is obtained based on the first training model, the second training model and the third training model.

[0078] Optionally, obtaining sample data based on historical data includes: Load the historical data within the set time period as training samples. After loading, determine the ratio of positive and negative samples. If the ratio is unbalanced, continue to load the data before the set time period.

[0079] Optionally, obtaining sample data based on historical data includes: excluding junk text messages in the historical data; that is, negative samples of the historical data include rejected text message data but do not include junk text message data.

[0080] When loading data, all historical data is not loaded. Too much historical data has accumulated, so data within a certain period of time is loaded as training samples. After loading is completed, the ratio of positive and negative samples will be judged. If it is unbalanced, insufficient samples will be loaded forward along the timeline. For example, if there are not enough text messages rejected by the review, the data rejected by the review will continue to be loaded forward outside the loaded time range until the earliest time limit allowed for loading is reached or there is no data. When loading data, the model of the prior art will merge and load spam text messages as negative samples. However, this embodiment excludes spam text messages, thereby making the sample data more accurate.

[0081] Optionally, the acquisition of sample data based on historical data includes: excluding SMS data with at least two different review results for the same SMS. For example, the content of the same message is exactly the same, but one reviewer determines that the SMS may violate the rules and should not be sent, while another reviewer mis-reviews it and approves it or thinks it can be approved. Such SMS with inconsistent review status will be cleared from the training samples, so as to make the training more accurate.

[0082] Optionally, before the step of using a logistic regression training model to process the sample data after similarity deduplication, the step includes determining whether the positive and negative samples of the sample data after similarity deduplication meet the requirements. If not, oversampling or undersampling is selected to process the sample data after similarity deduplication.

[0083] Similarity deduplication will match SMS messages that are similar to the SMS message. For example, the main content is the same, but the link or some part of the name is different. These highly similar SMS messages should also be removed from the training data. At this time, a deduplication calculation is performed based on the similarity of the loaded data. Similar SMS messages will only retain one similar SMS message, not the same SMS message.

[0084] Optionally, the step of using the naive Bayes training model to process the sample data before similarity deduplication includes: judging whether the positive and negative samples of the sample data before similarity deduplication meet the requirements, and if not, selecting undersampling to process the sample data before similarity deduplication.

[0085] After data preprocessing and similarity removal, before selecting training samples and models to start training, if a machine learning model is used instead of a vector database, and it is found that the positive and negative samples of the data are unbalanced, another process will be performed: In case of logistic regression, oversampling or undersampling is chosen based on the configuration.

[0086] If it is Naive Bayes, undersampling is used, which has been tested and verified to have the highest accuracy.

[0087] Undersampling uses the random undersampling method, and oversampling uses the SMOTE (Synthetic Minority Oversampling Technique) method.

[0088] Use the training sample data and the selected training model for training.

[0089] The training sample data must be vectorized before training and prediction.

[0090] Vectorization tools for samples: Logistic regression uses: CountVectorizer (term frequency (TF) vectorizer). The vector database of Naive Bayes uses: TfidfVectorizer (term frequency-inverse document frequency (TF-IDF) vectorizer). After testing, this has a higher accuracy in the current training samples.

[0091] After the model training is completed, the vectorizer used for training, the trained model, and related model information (including SMS type and other information) will be saved.

[0092] When using a vector database, the process is basically the same as above, and the constructed Faiss index object and related model information will also be saved.

[0093] Optionally, after the step of obtaining the audit model based on the first training model, the second training model and the third training model, the step further includes updating the obtained audit model, backing up the audit model before the update during the update, and during the update time period, when data is received, determining whether the current audit model status is available, and if not available, using the backed-up audit model before the update to process the received data.

[0094] Data is loaded and new models are trained regularly every day. After training is completed, the registered prediction service instance is obtained from the Nacos registration center, and the intranet IP and port information exposed by the relevant instance is obtained, and the trained relevant model data (model, vectorizer, model information) is distributed to the prediction service instance.

[0095] Distribution is complete, all instances are notified, and the model is updated.

[0096] The updating of the model is a smooth process, as follows: 1. The prediction service receives notification of model update, 2. The prediction service loads the corresponding model file, vectorizer, and model information from the local machine. Different models are loaded for different SMS types. 3. After loading is complete, set the current model object and vectorized object to the previous generation model, and point the current model to the latest loaded model. 4. How to ensure that even if any accidents occur during the update and the initialization process of the model, the prediction will not be affected. This is because the previous model object will be retained. When predicting, first determine whether the current model status is available. For example, when updating, if the status is not available (including any unavailability of the model or the vectorizer), then use the previous model object and vectorizer for prediction first.

[0097] The corresponding systems for the training method include: data collection system, machine audit model training system, prediction system, and machine audit background management system.

[0098] Data collection system: responsible for obtaining original SMS data from the existing SMS system and sending the data to the machine review model training system through the message middleware.

[0099] Machine audit model training system: After receiving the data processed by the data acquisition system, the system first performs data preprocessing, data similarity deduplication and model selection on the data, and then performs model training on the selected model and sample data.

[0100] Prediction system: Receives trained model files, vectorizers, and model information. At the same time, it receives http requests sent by the client. When predicting the content of the SMS, it will first predict spam SMS. The vector database will prioritize similarity search. If there is a match, it will be directly rejected. If it is not a spam SMS, it will select the corresponding type of model to predict whether the SMS is approved or rejected according to the SMS type of the sender.

[0101] Machine Audit Backend Management System: Provides a graphical interface for auditors to operate, including viewing audit results, manual audit, adjusting audit strategies, etc.

[0102] This embodiment applies as follows: SMS content filtering technology: In order to prevent the spread of spam and bad information, the SMS platform uses text analysis, keyword matching, machine learning and other algorithms to detect and filter SMS content in real time. By establishing a huge SMS content library and blacklist database, the platform can identify and intercept SMS containing sensitive words, advertising information, illegal content, etc., thereby effectively purifying the SMS environment and protecting the legitimate rights and interests of users.

[0103] SMS content machine review technology: To ensure that the SMS content complies with the requirements of different industries, the platform has established an SMS content review system. This system combines manual review and automated review to strictly review the content of the SMS sent. The SMS content is roughly divided into the following industries: industry, marketing, light reminder, heavy reminder, mutual finance, finance, spam SMS, etc. Since the SMS content required to be sent by different industries is different, models for different industries are trained according to different industries to improve the accuracy of automatic review. At the same time, manual review is performed on SMS messages that are unknown to the automatic review.

[0104] Foreign IP identification in SMS content: If there is a foreign IP in the SMS content, the probability of illegal content will be greatly increased, and foreign IPs will be blocked. Use IP address location query services, such as IP databases (such as IP2Location, MaxMind GeoIP, etc.), to determine the geographical location of the IP address, match the IP address in the SMS content with the IP database, obtain the IP address location information, and determine whether the IP address location is foreign. If it is foreign, it will be blocked; if it is domestic, it can be excluded.

[0105] This embodiment has the following advantages: the audit scheme based on machine learning can automatically learn new features and patterns from a large amount of data, so as to better adapt to the ever-changing language environment.

[0106] Machine learning-based methods can adapt to these emerging content through continuous learning.

[0107] Machine learning models can learn the connections between synonyms and variations through training, thereby more accurately identifying bad content and more accurately distinguishing between normal and bad content, thereby reducing false positives and negatives.

[0108] Machine learning-based methods can better understand the context of text, and thus perform better when dealing with content containing metaphors, sarcasm, or other complex language structures. Machine learning models can also better handle ambiguous concepts and polysemous words, thereby improving the accuracy of review.

[0109] Automatically learn new features: The machine learning-based review solution can automatically learn new features and patterns from a large amount of reviewed SMS data, thereby better adapting to the ever-changing SMS content.

[0110] Higher accuracy: Different types of machine review models are trained according to different types of SMS content, and the accuracy is higher when the corresponding type of model is used for machine review of a certain type of SMS content.

[0111] Higher efficiency: The audit method based on machine learning can achieve faster processing speed and is suitable for processing large amounts of data and real-time auditing needs.

[0112] Better customizability: Machine learning-based audit solutions can customize different audit strategies by adjusting model parameters or training data to meet the specific content requirements of different types of SMS content audit results.

[0113] The present embodiment also discloses a text message review method, which uses an review model obtained by all or part of the steps of the training method disclosed in the present embodiment. The review results of the review model are stored in a cache. When new review data is received, the new review data is first compared with the review results in the cache. If the review results in the cache exist in the same data as the new review data, the review results of the new review data are directly obtained based on the review results in the cache.

[0114] like Figure 2 As shown, when predicting text messages, spam text message prediction will be used first, and the vector database will prioritize similarity search. If there is a match, it will be directly reviewed and rejected.

[0115] If it is not a spam message, the corresponding type of model will be selected for prediction based on the sender's message type. Although this implementation uses different types of models and vector databases, it encapsulates a unified training and prediction interface.

[0116] The result of the prediction by the model and vector database is a score. According to the configuration, the score threshold is subsequently processed to determine whether the message is passed or rejected. A small number of text messages at the edge of the threshold will enter the manual review process. This will improve the accuracy, reduce the misjudgment rate, and avoid prediction errors that may cause text messages that should not be sent to be sent.

[0117] Predicted or reviewed SMS messages will be cached for 48 hours. The cache time is set according to the device. The next time a new SMS message comes in, it will be checked from the cache first. If the SMS content is the same, the result will be returned directly. If it is not in the cache, it will be automatically reviewed by the machine.

[0118] In SMS review, because a small amount of data will be manually reviewed, within a few hours before the new model (trained with the latest samples) is updated, the incremental data that is rejected by the review is matched for similarity using the vector database. Before prediction, the spam SMS model is used to predict, and then the model trained by the training method disclosed in this embodiment is used to predict whether to reject. If not rejected, the corresponding training model in the corresponding model is used for prediction.

[0119] The electronic device disclosed in this embodiment includes a memory and a processor. The memory is used to store non-temporary computer-readable instructions. Specifically, the memory may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc.

[0120] The processor may be a central processing unit (CPU) or other forms of processing units with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions. In one embodiment of the present disclosure, the processor is used to run the computer-readable instructions stored in the memory, so that the electronic device performs all or part of the steps of the training method for the SMS content review model in the aforementioned embodiments of the present disclosure.

[0121] Those skilled in the art should be able to understand that in order to solve the technical problem of how to obtain a good user experience, the present embodiment may also include well-known structures such as a communication bus and an interface, and these well-known structures should also be included in the protection scope of the present disclosure.

[0122] like Figure 3 The present invention provides a schematic diagram of the structure of an electronic device according to an embodiment of the present invention, which is suitable for implementing the electronic device in the embodiment of the present invention. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0123] like Figure 3 As shown, the electronic device may include a processing device (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage device into a random access memory (RAM). In the RAM, various programs and data required for the operation of the electronic device are also stored. The processing device, ROM, and RAM are connected to each other through a bus. The input / output (I / O) interface is also connected to the bus.

[0124] Typically, the following devices can be connected to the I / O interface: input devices such as sensors or visual information acquisition devices; output devices such as display screens; storage devices such as tapes, hard disks, etc.; and communication devices. The communication device allows the electronic device to communicate with other devices (such as edge computing devices) wirelessly or by wire to exchange data. Figure 3 An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0125] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processing device, all or part of the steps of the training method for a text message content review model of an embodiment of the present disclosure are executed.

[0126] For detailed description of this embodiment, reference may be made to the corresponding descriptions in the aforementioned embodiments, which will not be repeated here.

[0127] The computer-readable storage medium disclosed in this embodiment stores non-transitory computer-readable instructions. When the non-transitory computer-readable instructions are executed by a processor, all or part of the steps of the training method for the SMS content review model of each embodiment of the present disclosure are executed.

[0128] The above-mentioned computer-readable storage media include, but are not limited to: optical storage media (e.g., CD-ROM and DVD), magneto-optical storage media (e.g., MO), magnetic storage media (e.g., magnetic tape or mobile hard disk), media with built-in rewritable non-volatile memory (e.g., memory card) and media with built-in ROM (e.g., ROM box).

[0129] For detailed description of this embodiment, reference may be made to the corresponding descriptions in the aforementioned embodiments, which will not be repeated here.

[0130] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0131] In the present disclosure, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. The block diagrams of the devices, devices, equipment, and systems involved in the present disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably with them.

[0132] Additionally, as used herein, "or" used in a list of items beginning with "at least one" indicates a separate list, so that, for example, a list of "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not mean that the example described is preferred or better than other examples.

[0133] It should also be noted that in the system and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0134] Various changes, substitutions, and modifications of the techniques described herein may be made without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of the present disclosure is not limited to the specific aspects of the processes, machines, manufactures, compositions of events, means, methods, and actions described above. Currently existing or later to be developed processes, machines, manufactures, compositions of events, means, methods, or actions that perform substantially the same functions or achieve substantially the same results as the corresponding aspects described herein may be utilized. Thus, the appended claims include such processes, machines, manufactures, compositions of events, means, methods, or actions within their scope.

[0135] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0136] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A training method for a text message content review model, characterized in that: include: Get sample data based on historical data; Acquire a first data volume of sample data, and if the first data volume is less than a first set data volume, process the sample data using a vector database to obtain a first training model; If the first data volume is not less than the first set data volume, the sample data is subjected to similarity deduplication processing to obtain a second data volume after similarity deduplication; if the second data volume is greater than the second set data volume, the sample data after similarity deduplication is processed using a logistic regression training model to obtain a second training model; If the second data volume is not greater than the second set data volume, the naive Bayes training model is used to process the sample data before similarity deduplication, thereby obtaining a third training model; An audit model is obtained based on the first training model, the second training model and the third training model.

2. The training method for a text message content review model according to claim 1, characterized in that: The acquiring of sample data based on historical data includes: Load the historical data within the set time period as training samples. After loading, determine the ratio of positive and negative samples. If the ratio is unbalanced, continue to load the data before the set time period.

3. The training method for a text message content review model according to claim 2, characterized in that: The obtaining of sample data based on historical data includes: excluding junk text messages in the historical data; that is, the negative samples of the historical data include rejected text message data, but do not include junk text message data.

4. The training method for a text message content review model according to claim 1, characterized in that: The obtaining of sample data based on historical data includes: excluding SMS data with at least two different review results for the same SMS.

5. The training method for a text message content review model according to claim 1, characterized in that: The step of using a logistic regression training model to process the sample data after similarity deduplication includes determining whether the positive and negative samples of the sample data after similarity deduplication meet the requirements. If not, oversampling or undersampling is selected to process the sample data after similarity deduplication.

6. The training method for a text message content review model according to claim 1, characterized in that: The step of using the naive Bayes training model to process the sample data before similarity deduplication includes: judging whether the positive and negative samples of the sample data before similarity deduplication meet the requirements, and if not, selecting undersampling to process the sample data before similarity deduplication.

7. The training method for a text message content review model according to claim 1, characterized in that: After the step of obtaining the audit model based on the first training model, the second training model and the third training model, the method also includes updating the obtained audit model, backing up the audit model before the update during the update, and determining whether the current audit model status is available when data is received during the update time period. If not, use the backed-up audit model before the update to process the received data.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the training method for the SMS content review model described in any one of claims 1-7.

9. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the training method for a text message content review model described in any one of claims 1-7 is implemented.

10. A text message audit method, using the audit model obtained by the training method according to any one of claims 1 to 7, characterized in that: The audit results of the audit model are saved in the cache. When new audit data is received, the new audit data is first compared with the audit results in the cache. If the audit results in the cache are the same as the new audit data, the audit results of the new audit data are directly obtained based on the audit results in the cache.

Images