A service SMS monitoring method and system based on big data

Through the service SMS monitoring method based on big data, using technologies such as BERT model and LSTM model, the monitoring strategy is dynamically adjusted, and the problems of inefficient efficiency and insufficient identification accuracy in the existing technology are solved, and efficient and accurate service SMS monitoring and Resource optimization.

CN118474682BActive Publication Date: 2024-11-01ZHEJIANG SANZI ZHILIAN TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410941226.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-15
Publication Date
2024-11-01
Estimated Expiration
2044-07-15

AI Technical Summary

Technical Problem

The existing service SMS monitoring methods are inefficient in processing huge data volumes, cannot monitor in real time, insufficient analysis of diverse and complex SMS content, low recognition accuracy, fixed monitoring strategies, and cannot be dynamically adjusted, resulting in waste of resources and inability to respond to them. timely.

Method used

Using a service SMS monitoring method based on big data, obtain historical monitoring data, filter and classify node characteristics, and implement different monitoring strategies based on classification results. Using technologies such as BERT model, LSTM model and non-negative matrix decomposition to extract and predict data features , dynamically adjust monitoring strategies.

Benefits of technology

It improves the efficiency and accuracy of service SMS monitoring, realizes intelligent analysis and prediction of node characteristics, ensures efficient utilization of resources and timely monitoring, and continuously improves and improves the performance of the monitoring system.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118474682B_ABST
    Figure CN118474682B_ABST
Patent Text Reader

Abstract

The present invention discloses a service SMS monitoring method and system based on big data, which relates to the technical field of service SMS monitoring, including: obtaining monitoring data of historical service SMS; screening monitored nodes and classifying node features; and executing different monitoring strategies according to the classification results. It is capable of efficiently processing and analyzing massive service SMS data, and improving the efficiency and accuracy of monitoring. Intelligent analysis and prediction of node features can identify and predict the characteristics and behaviors of service SMS in real time. The monitoring strategy has the ability to be dynamically adjusted, and different nodes are classified and monitored according to service feature scores to ensure efficient use of resources and timeliness of monitoring. By updating historical data in real time and optimizing monitoring strategies, the performance of the monitoring system can be continuously improved and enhanced to ensure the stability and reliability of the system.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of service short message monitoring, and in particular to a service short message monitoring method and system based on big data. Background Art

[0002] In modern communication networks, service SMS (such as notification SMS, marketing SMS, warning SMS, etc.) plays a vital role in daily life and business activities. With the development of mobile communication technology and the increase in the number of users, the number of service SMS has exploded. How to efficiently and accurately monitor and manage these service SMS has become an important challenge facing communication operators and related service providers. Summary of the invention

[0003] In view of the above-mentioned problems, the present invention is proposed.

[0004] Therefore, the technical problems solved by the present invention are: the existing service SMS monitoring method is inefficient in processing huge amounts of data and cannot be monitored in real time; the analysis of diverse and complex SMS content is insufficient and the recognition accuracy is low; the monitoring strategy is fixed and cannot be adjusted dynamically, resulting in waste of resources and untimely response.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions: a service SMS monitoring method based on big data, comprising:

[0006] Get monitoring data of historical service SMS;

[0007] Screen the monitored nodes and classify the node characteristics;

[0008] Different monitoring strategies are implemented according to the classification results.

[0009] As a preferred solution of the service SMS monitoring method based on big data described in the present invention, the monitoring data includes SMS content data, SMS sending and receiving node data, SMS receiving node location data, and user behavior data;

[0010] The node data for sending and receiving text messages is the communication code between the sender and the receiver;

[0011] The node location data for receiving SMS is the geographical location of the recipient when receiving the service SMS;

[0012] The user behavior data refers to the recipient's usage behavior in a period of time before receiving the service SMS;

[0013] Among them, service SMS is obtained by obtaining SMS data from different service providers and using interfaces or APIs to capture and import data.

[0014] As a preferred solution of the service SMS monitoring method based on big data described in the present invention, wherein: the screening of monitored nodes and classification of node characteristics include extracting the service characteristics of the receiving node according to the sending records of the service SMS and the service characteristic content notified by the provider;

[0015] Obtain all receiving nodes within the monitoring range, and determine whether they have the service characteristics by predicting the receiving nodes; if the receiving nodes have the probability of occurrence of the service characteristics, the receiving nodes are feature classified; if the receiving nodes do not have the probability of occurrence of the service characteristics, the receiving nodes are screened and eliminated;

[0016] The extracting of the service feature of the receiving node refers to performing feature vectorization according to the sending record of the service SMS, which includes the following steps:

[0017] Use the BERT model to convert SMS text data into high-dimensional vectors;

[0018] Use embedding coding to convert the communication codes of nodes into vector representations;

[0019] Use multidimensional scaling to reduce the latitude and longitude of geographic locations;

[0020] Use standardized methods to process user behavior data;

[0021] All feature vectors are concatenated into a high-dimensional feature vector, and NMF is used to reduce the dimension to obtain the final monitoring data feature vector:

[0022] ;

[0023] in, represents the monitoring data feature vector output by NMF; BERT (C) represents the BERT model vectorizing the SMS content C; Embed (N) represents Embedding vectorizing the node data N; MDS (L) represents multi-dimensional scaling to reduce the dimension of the node location data L; Norm (B) represents the standardized user behavior data B, which is used to perform the i To standardize: , represents the mean of user behavior data, represents the standard deviation of user behavior data; NMF represents non-negative matrix factorization function;

[0024] Using the improved time series clustering algorithm, historical service SMS records are clustered and analyzed to extract service features:

[0025] ;

[0026] Where K represents the number of clusters, represents the historical record characteristic function, represents the weight of the kth cluster, C k represents the kth cluster center, is the covariance matrix of the kth cluster, T represents the transpose operation;

[0027] Extract the service features informed by the provider:

[0028] ;

[0029] Service Characteristics Collection:

[0030] ;

[0031] in, represents the provider characteristic function, Indicates the service characteristics notified by the provider. Represents the semantic parsing results based on the BERT model. represents the features extracted from the domain knowledge graph, S represents the text data of the service feature content informed by the provider, Indicates the service feature content of the i-th feature notified by the provider.

[0032] As a preferred solution of the service SMS monitoring method based on big data described in the present invention, wherein: the occurrence probability of the service feature includes, let B(t) represent the user behavior feature vector at time t, and use the LSTM model to predict the user behavior features in the future;

[0033] ;

[0034] in, The predicted future time is The user behavior feature vector at time n represents the time step of LSTM. Indicates the length of the forecast period;

[0035] Suppose the service characteristics are , the historical data is H, then the occurrence probability of the service feature is expressed as:

[0036] ;

[0037] in, represents the probability of occurrence of a service feature given current and predicted user behavior characteristics and historical data, The value range of is [0,1]. The closer the value is to 1, the higher the probability of occurrence of the service feature is, and the closer the value is to 0, the lower the probability of occurrence is. is the joint probability of current and predicted user behavior characteristics under service characteristics and historical data; is the prior probability of the service feature under historical data; is the marginal probability of current and predicted user behavior features under historical data;

[0038] ;

[0039] Where N represents normal distribution; Indicated in the service feature F service The mean vector under is the expected value of the user behavior characteristics; Indicated in the service feature F service The covariance matrix below is the distribution of user behavior characteristics;

[0040] Calculate the prior probability of service characteristics under historical data :

[0041] ;

[0042] Calculate the marginal probability of current and predicted user behavior characteristics under historical data :

[0043] ;

[0044] in, Indicates the number of times a service feature appears in historical data. Indicates the total number of historical data.

[0045] As a preferred solution of the service SMS monitoring method based on big data described in the present invention, the node feature classification includes the feature vector of the user's current behavior. Whether it meets the service feature set F service :

[0046] ;

[0047] like , then it is classified as the service feature has been triggered; let the service feature occurrence probability threshold be ,like , it is classified as a potential trigger of service features; if , it is classified as a low-risk trigger of service characteristics.

[0048] As a preferred solution of the service SMS monitoring method based on big data described in the present invention, the monitoring strategy includes: for the node classification results of the service feature that has been triggered, continuous monitoring is performed within a complete monitoring cycle; for the classification results other than the service feature that has been triggered, periodic monitoring is performed by calculating the service feature score;

[0049] The service feature scoring includes: And the current user behavior feature vector B(t), weighted calculation of service feature score S:

[0050] ;

[0051] Among them, S represents the service feature score, n represents the number of features, and W i represents the weight of the i-th feature, B i (t) is the behavior feature value of the i-th user at the current time t;

[0052] For the i-th user behavior feature value B at the current time t i (t), we need to calculate the cosine similarity between the user behavior feature and all service features and take the minimum value; let the service feature vector set be , where j is the number of the service feature;

[0053] ;

[0054] Among them, cosine_similarity represents the function of calculating cosine similarity, min j Indicates taking the minimum value of the cosine similarity of all service features;

[0055] According to the service feature score, the monitoring period is defined for the nodes in the potential trigger of the service feature and the low-risk trigger of the service feature respectively:

[0056] nodes that potentially trigger the service feature, ;

[0057] For nodes with low risk triggering of the service feature, ;

[0058] Among them, S q represents the service feature score of the qth node, with a total of K nodes; T represents the completion cycle length of the monitoring cycle; represents the reduction factor of nodes classified as potential triggers of service features, represents the reduction factor of nodes classified as low-risk triggers of service characteristics, and .

[0059] As a preferred solution of the service SMS monitoring method based on big data described in the present invention, wherein: the monitoring strategy is used for continuous monitoring, and when a service SMS is detected in a node, the monitoring data is packaged and put into a historical record, thereby completing the update of the historical service SMS;

[0060] The updated historical service SMS monitoring data is used to update the screening of monitored nodes and the node feature classification process, and finally the monitoring strategy is updated, and the service SMS monitoring is performed according to the updated monitoring strategy.

[0061] A service SMS monitoring system based on big data using the method of the present invention, wherein:

[0062] Information management module, to obtain monitoring data of historical service SMS;

[0063] The analysis module screens the monitored nodes and classifies the node characteristics;

[0064] The execution module executes different monitoring strategies according to the classification results.

[0065] A computer device comprises: a memory and a processor; the memory stores a computer program, wherein the processor implements the steps of any one of the big data-based service SMS monitoring methods described in the present invention when executing the computer program.

[0066] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of any one of the big data-based service SMS monitoring methods described in the present invention.

[0067] Beneficial effects of the present invention: The service SMS monitoring method based on big data provided by the present invention can efficiently process and analyze massive service SMS data, and improve the efficiency and accuracy of monitoring. Intelligent analysis and prediction of node characteristics can identify and predict the characteristics and behaviors of service SMS in real time. The monitoring strategy has the ability to dynamically adjust, classify and monitor different nodes according to the service characteristic scores, and ensure efficient use of resources and timely monitoring. By updating historical data in real time and optimizing monitoring strategies, the performance of the monitoring system can be continuously improved and enhanced to ensure the stability and reliability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0069] Figure 1 An overall flow chart of a service SMS monitoring method based on big data provided for the first embodiment of the present invention. DETAILED DESCRIPTION

[0070] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0071] Example 1, reference Figure 1 , is an embodiment of the present invention, and provides a service SMS monitoring method based on big data, comprising:

[0072] S1: Get the monitoring data of historical service SMS.

[0073] Furthermore, the monitoring data includes SMS content data, SMS sending and receiving node data, SMS receiving node location data, and user behavior data. The SMS sending and receiving node data is the communication code between the sender and the receiver. The SMS receiving node location data is the geographical location of the recipient when receiving the service SMS. The user behavior data is the usage behavior of the recipient in a time period before receiving the service SMS. Among them, the service SMS is obtained by obtaining SMS data from different service providers, and using interfaces or APIs to capture and import data.

[0074] By obtaining monitoring data of historical service SMS, the comprehensiveness and diversity of data are ensured. These data include SMS content, node data, location data and user behavior data, which can provide comprehensive information and help accurately analyze and identify the characteristics of service SMS. Combining SMS content data, node data, location data and user behavior data helps to analyze service SMS from multiple dimensions. This can provide a more comprehensive understanding of the background of SMS sending and receiving and its influencing factors, and improve the accuracy of analysis.

[0075] By collecting historical data, we can provide a basis for model training and algorithm optimization. Historical data can reflect past patterns and trends, which helps to build accurate prediction models and identify abnormal behaviors. Obtaining data from different service providers ensures data diversity and representativeness. Data crawling and importing through interfaces or APIs can achieve automated and efficient data collection, reduce manual intervention, and improve the efficiency and accuracy of data acquisition.

[0076] S2: Screen the monitored nodes and classify the node features.

[0077] The screening of monitored nodes and classification of node characteristics include extracting the service characteristics of the receiving node according to the sending record of the service SMS and the service characteristic content notified by the provider.

[0078] Furthermore, all receiving nodes within the monitoring range are obtained, and whether the receiving nodes have the service characteristics is determined by prediction of the receiving nodes; if the receiving nodes have the probability of occurrence of the service characteristics, the receiving nodes are feature classified; if the receiving nodes do not have the probability of occurrence of the service characteristics, the receiving nodes are screened and eliminated.

[0079] By screening and feature classification of monitored nodes, resources and attention can be focused on nodes with a higher probability of service feature occurrence, thereby improving the efficiency and accuracy of monitoring. Screening and classification can avoid unnecessary monitoring and reduce resource waste. By extracting the service features of the receiving nodes, it is possible to accurately identify which nodes have a higher probability of service SMS occurrence. This helps to focus on monitoring high-risk nodes and promptly discover and respond to potential problems. Based on the sending records of service SMS and the service feature content informed by the provider, the service features of the receiving nodes can be extracted to optimize the monitoring strategy. By classifying the features of the receiving nodes, different monitoring strategies can be formulated to ensure the pertinence and effectiveness of the monitoring strategies. The prediction and judgment of the receiving nodes can be dynamically adjusted based on real-time data. This enables the monitoring system to flexibly respond to changing environments and situations and maintain efficient monitoring results. Node feature extraction based on the sending records of service SMS and service feature content provides data-driven decision support. Data-based judgment and classification can reduce human bias and enhance the objectivity and scientificity of the monitoring system.

[0080] The extracting of the service feature of the receiving node refers to performing feature vectorization according to the sending record of the service SMS, which includes the following steps:

[0081] Use the BERT model to convert SMS text data into high-dimensional vectors.

[0082] Embedding coding is used to convert the communication encoding of nodes into vector representation.

[0083] Use multidimensional scaling to reduce the latitude and longitude of geographic locations.

[0084] Use standardized methods to process user behavior data.

[0085] All feature vectors are concatenated into a high-dimensional feature vector, and NMF is used to reduce the dimension to obtain the final monitoring data feature vector:

[0086] All feature vectors are concatenated into a high-dimensional feature vector, and NMF is used to reduce the dimension to obtain the final monitoring data feature vector:

[0087] ;

[0088] in, represents the monitoring data feature vector output by NMF; BERT (C) represents the BERT model vectorizing the SMS content C; Embed (N) represents Embedding vectorizing the node data N; MDS (L) represents multi-dimensional scaling to reduce the dimension of the node location data L; Norm (B) represents the standardized user behavior data B, which is used to perform the i To standardize: , represents the mean of user behavior data, represents the standard deviation of user behavior data; NMF represents non-negative matrix factorization function;

[0089] Using the improved time series clustering algorithm, historical service SMS records are clustered and analyzed to extract service features:

[0090] ;

[0091] Where K represents the number of clusters, represents the historical record characteristic function, represents the weight of the kth cluster, C k represents the kth cluster center, is the covariance matrix of the kth cluster, T represents the transpose operation;

[0092] Extract the service features informed by the provider:

[0093] ;

[0094] Service Characteristics Collection:

[0095] ;

[0096] in, represents the provider characteristic function, Indicates the service characteristics notified by the provider. Represents the semantic parsing results based on the BERT model. It represents the features extracted from the domain knowledge graph, and S represents the text data of the service feature content informed by the provider, which is the main input for semantic parsing through the deep learning model. The service feature content of the i-th feature informed by the provider is a specific part or key feature of the text data, which is analyzed in detail by combining the features extracted from the domain knowledge graph.

[0097] It should be said that the BERT model, embedding coding, multidimensional scaling (MDS) and standardization methods are all relatively mature data processing technologies with good practical application effects. Non-negative matrix factorization (NMF), as an effective dimensionality reduction method, has been widely used in many fields. The combination of hybrid embedding method and NMF can effectively process various types of data and ensure the effectiveness of feature extraction. The improved time series clustering algorithm can be implemented through existing clustering technology and time series analysis methods, and has high operability.

[0098] Through a variety of advanced data processing technologies, we ensure that the features extracted from the monitoring data have high accuracy and representativeness. Combining the BERT model and domain knowledge graph, we can automatically extract key information from complex service feature content and improve the intelligence level of the system. Through time series clustering and feature vectorization, we can identify patterns and rules in historical data, thereby optimizing monitoring strategies and improving monitoring efficiency and accuracy. Through real-time analysis and prediction of node features, we can dynamically adjust monitoring strategies to ensure that the system can respond to changing situations and needs in a timely manner.

[0099] The occurrence probability of the service feature includes: let B(t) represent the user behavior feature vector at time t, and use the LSTM model to predict the user behavior features in the future;

[0100] ;

[0101] in, The predicted future time is The user behavior feature vector at time n represents the time step of LSTM. Indicates the length of the forecast period;

[0102] Suppose the service characteristics are , the historical data is H, then the occurrence probability of the service feature is expressed as:

[0103] ;

[0104] in, represents the probability of occurrence of a service feature given current and predicted user behavior characteristics and historical data, The value range of is [0,1]. The closer the value is to 1, the higher the probability of occurrence of the service feature is, and the closer the value is to 0, the lower the probability of occurrence is. is the joint probability of current and predicted user behavior characteristics under service characteristics and historical data; is the prior probability of the service feature under historical data; is the marginal probability of current and predicted user behavior features under historical data;

[0105] ;

[0106] Where N represents normal distribution; Indicated in the service feature F service The mean vector under is the expected value of the user behavior characteristics; Indicated in the service feature F service The covariance matrix below is the distribution of user behavior characteristics;

[0107] Calculate the prior probability of service characteristics under historical data :

[0108] ;

[0109] Calculate the marginal probability of current and predicted user behavior characteristics under historical data :

[0110] ;

[0111] in, Indicates the number of times a service feature appears in historical data. Indicates the total number of historical data.

[0112] It should be noted that LSTM (Long Short-Term Memory Network) is a neural network model that can capture long-term dependencies in time series data. Using the LSTM model to predict user behavior characteristics in the future can accurately capture the dynamic changes and trends of user behavior, and provide a reliable data basis for the subsequent calculation of the probability of occurrence of service characteristics. Using Bayes' theorem to calculate the probability of occurrence of service characteristics can comprehensively consider current behavior characteristics, predicted user behavior characteristics and historical data to improve the accuracy and reliability of probability calculation. This method can make full use of existing data and provide scientific probability estimates. Assuming that user behavior characteristics follow Gaussian distribution, its mean vector and covariance matrix can be used to describe the expected value and distribution of behavior characteristics. This method has good mathematical properties and computational efficiency in practical applications.

[0113] Predicting future user behavior characteristics through the LSTM model can provide dynamic data input for the probability calculation of service characteristics, improving the foresight and early warning capabilities of the monitoring system. Combining current behavior characteristics, predicted user behavior characteristics and historical data, the occurrence of service characteristics can be comprehensively analyzed to improve the accuracy and reliability of the monitoring system. Using Bayesian theorem and Gaussian distribution for probability calculation can provide scientific probability estimates for the occurrence of service characteristics and enhance the decision support capabilities of the monitoring system.

[0114] The node feature classification includes the feature vector of the user's current behavior. Whether it meets the service feature set F service :

[0115] ;

[0116] like , then it is classified as the service feature has been triggered; let the service feature occurrence probability threshold be ,like , it is classified as a potential trigger of service features; if , it is classified as a low-risk trigger of service characteristics.

[0117] S3: Execute different monitoring strategies based on the classification results.

[0118] For the node classification results for which the service feature has been triggered, continuous monitoring is performed within a complete monitoring cycle; for the classification results other than those for which the service feature has been triggered, periodic monitoring is performed by calculating the service feature score.

[0119] The service feature scoring includes: And the current user behavior feature vector B(t), weighted calculation of service feature score S:

[0120] ;

[0121] Among them, S represents the service feature score, n represents the number of features, and W i represents the weight of the i-th feature, B i (t) is the behavior feature value of the i-th user at the current time t;

[0122] For the i-th user behavior feature value B at the current time t i (t), we need to calculate the cosine similarity between the user behavior feature and all service features and take the minimum value; let the service feature vector set be , where j is the number of the service feature.

[0123] ;

[0124] Among them, cosine_similarity represents the function of calculating cosine similarity, min j Indicates taking the minimum value of the cosine similarity of all service features.

[0125] According to the service feature score, the monitoring period is defined for the nodes in the potential trigger of the service feature and the low-risk trigger of the service feature respectively:

[0126] nodes that potentially trigger the service feature, ;

[0127] For nodes with low risk triggering of the service feature, ;

[0128] Among them, S q represents the service feature score of the qth node, with a total of K nodes; T represents the completion cycle length of the monitoring cycle; represents the reduction factor of nodes classified as potential triggers of service features, represents the reduction factor of nodes classified as low-risk triggers of service characteristics, and .

[0129] It is important to know that for nodes that have triggered service features, continuous monitoring can ensure real-time tracking and management of high-risk nodes, timely detection and handling of potential problems, and prevent the spread of problems. By calculating the service feature score, periodically monitoring nodes that have not triggered service features, and adjusting the monitoring cycle according to their risk level, monitoring resources can be allocated more effectively and overall monitoring efficiency can be improved. Dividing nodes into "potential triggers" and "low-risk triggers" based on service feature scores and defining different monitoring cycles for each can more accurately match nodes of different risk levels and optimize monitoring frequency and strategy.

[0130] By weighted calculation of service feature scores, the probability of occurrence of service features and current behavior characteristics are comprehensively considered, making the scores more accurate and able to more accurately reflect the risk level of the node. The monitoring cycle is dynamically adjusted according to the risk level, and high-risk nodes are monitored more frequently, while the monitoring frequency of low-risk nodes is appropriately reduced, thereby optimizing the allocation of monitoring resources and improving resource utilization. The dynamically adjusted monitoring strategy can flexibly respond to changes in node characteristics and behaviors, has strong adaptability, and ensures that the monitoring system is always in the best state. Through in-depth analysis and scoring of node characteristics, false positives and false negatives can be effectively reduced, and the accuracy and reliability of the monitoring system can be improved.

[0131] Furthermore, through continuous monitoring by the monitoring strategy, when a service message is detected in a node, the monitoring data is packaged and put into the historical record to complete the update of the historical service message; using the updated monitoring data of the historical service message, the screening of the monitored nodes and the node feature classification process are updated, and finally the update of the monitoring strategy is completed, and the service message monitoring is performed according to the updated monitoring strategy.

[0132] Specifically, ensure that historical service SMS data is updated in real time. This helps to keep the data fresh and accurate, and avoid monitoring failures or errors caused by data lag. Using the latest historical service SMS data to update the screening and feature classification process for monitoring nodes can improve the adaptability and accuracy of the model. Over time, user behavior and service characteristics may change. Dynamically updating data helps the model reflect these changes in a timely manner and maintain efficient and accurate monitoring capabilities. Based on the updated monitoring data, optimize the monitoring strategy to ensure that the strategy always matches the current service characteristics and user behavior characteristics. This dynamic optimization helps to improve resource utilization, reduce false positives and false negatives, and improve the overall effectiveness of the monitoring system. A closed-loop feedback mechanism is implemented by feeding the newly monitored data back to the system. Every time a service SMS is monitored and the historical record is updated, the system becomes smarter and more accurate. The closed-loop feedback mechanism can continuously improve the system's self-learning and adaptive capabilities, so that the monitoring system can maintain efficient operation in continuous optimization.

[0133] Design a dynamic update mechanism to ensure that the monitoring system can operate effectively in a dynamic environment. The communication environment and user behavior may change at any time. Dynamic updates can ensure that the monitoring system can adapt to environmental changes in a timely manner and remain stable and efficient. The updated monitoring data provides more accurate and comprehensive information for node screening and classification, thereby enhancing decision support capabilities. The accuracy and timeliness of decision-making are important guarantees for the effective operation of the monitoring system.

[0134] On the other hand, this embodiment also provides a service SMS monitoring system based on big data, which includes:

[0135] Information management module, obtains monitoring data of historical service text messages.

[0136] The analysis module screens the monitored nodes and classifies the node features.

[0137] Execution module, which executes different monitoring strategies based on the classification results

[0138] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program code.

[0139] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0140] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0141] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0142] Example 2: The following is an embodiment of the present invention, which provides a service SMS monitoring method based on big data. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0143] Experiment 1: The main purpose of the experiment is to compare the performance differences between the present invention and the traditional method in service SMS monitoring, especially the recognition accuracy of service SMS triggering nodes, the utilization efficiency of monitoring resources and the real-time performance of the system.

[0144] Traditional methods mainly rely on simple rule matching and statistical analysis to monitor service SMS. The specific steps are as follows:

[0145] Data collection: Obtain service SMS data from service providers A, B, and C through a fixed interface. The data includes SMS content, sending and receiving node data (communication code), receiving location data (latitude and longitude), and user behavior data.

[0146] Keyword matching: Use a predefined keyword library to match the SMS content with keywords and identify possible service SMS messages.

[0147] Statistical analysis: Perform simple statistical analysis (such as frequency, duration, etc.) on the behavior data of the receiving node to determine the triggering probability of the service SMS.

[0148] Fixed strategy monitoring: Based on the statistical results, set a fixed monitoring strategy to monitor all nodes at the same frequency.

[0149] Experimental preparation:

[0150] Obtain one month of service SMS data from service providers A, B, and C, including SMS content, communication codes of sending and receiving nodes, receiving location data (latitude and longitude), and user behavior data.

[0151] The data are divided into two groups: one group is processed using the traditional method, and the other group is processed using the method of the present invention.

[0152] Data processing:

[0153] Traditional method: keyword matching of SMS content to identify possible service SMS. Simple statistics of the behavior data of the receiving node, calculation of the trigger probability, and setting of fixed monitoring strategy.

[0154] The method of the present invention: uses the BERT model, embedded coding, multi-dimensional scaling and standardization methods to extract features and vectorize data. Uses the LSTM model to predict future behavior features, calculate the probability of service feature occurrence, and dynamically adjust the monitoring strategy.

[0155] Monitoring strategy execution:

[0156] Traditional method: All nodes are monitored at a fixed frequency without distinguishing risk levels.

[0157] The method of the present invention: continuously monitor the nodes whose service features have been triggered, and dynamically adjust the monitoring frequency of other nodes according to the service feature scores. (After completing the first round of testing, the second round of testing is carried out again using the optimized solution trained by the first round of testing.)

[0158] Data Records:

[0159] The performance of the two groups of methods in terms of trigger node identification accuracy, resource utilization and response time are recorded, as shown in Table 1.

[0160] Table 1 Data comparison table

[0161]

[0162] By comparing and analyzing the data in Table 1, it can be seen that the performance indicators of the present invention in service SMS monitoring are superior to those of the traditional method.

[0163] The recognition accuracy of the traditional method is 85%. The accuracy of the present invention is increased to 90% in the initial stage, and further increased to 95% after optimization. This improvement is due to the fact that the present invention uses the advanced BERT model to perform high-dimensional vector processing on the SMS content, and combines embedded coding and multi-dimensional scaling technology to comprehensively and accurately extract and analyze node features. This enables the system to more accurately identify the service SMS trigger node, significantly reducing false positives and false negatives.

[0164] The resource utilization rate of the traditional method is 70%. The present invention dynamically adjusts the monitoring strategy and reasonably allocates monitoring resources according to the service feature scores of the nodes. The resource utilization rate is increased to 80% in the initial stage and reaches 90% after optimization. Efficient resource utilization means that the system can focus more monitoring resources on high-risk nodes and reduce the monitoring frequency of low-risk nodes, thereby improving the overall monitoring efficiency.

[0165] The average response time of the traditional method is 500 milliseconds. The present invention shortens the response time to 450 milliseconds in the initial stage, and further shortens it to 400 milliseconds after optimization. This is mainly due to the fact that the present invention uses non-negative matrix factorization (NMF) technology to reduce the dimensionality of high-dimensional feature vectors, reducing the computational complexity and improving the real-time performance of the system. The shorter response time can ensure that the system responds to the triggering event of the service SMS in a timely manner, improving the real-time performance of the monitoring system and user experience.

[0166] In summary, the present invention demonstrates obvious innovation and advantages in service SMS monitoring, and fully demonstrates its creativity and advancement in the application of big data technology and machine learning algorithms.

[0167] Experiment 2: The experiment is divided into five main steps: data collection, data processing and feature extraction, time series prediction, Bayesian inference, and monitoring strategy execution.

[0168] Test preparation:

[0169] One month of service SMS data was obtained from the service provider, including SMS content, communication codes of sending and receiving nodes, receiving location data (latitude and longitude), and user behavior data. The test subjects included multiple receiving nodes, and the user behavior characteristics, number of SMS received, and service characteristics occurrence probability of each node were recorded.

[0170] The implementation process includes data collection, data processing and feature extraction, time series prediction, Bayesian inference, and monitoring strategy execution. The data in the six nodes A, B, C, D, E, and F are recorded, as shown in Table 2.

[0171] Data collection: Use the interface to capture SMS data from service providers and import it into the system. The collected data includes the content of service SMS, communication codes of sending and receiving nodes, receiving location data (latitude and longitude), and user behavior data. The comprehensiveness and diversity of the data ensure the accuracy and representativeness of subsequent analysis.

[0172] Data processing and feature extraction: The BERT model is used to convert SMS text data into high-dimensional vectors to ensure that the semantic information of the SMS content can be fully extracted. Embedding coding is used to convert the communication code of the node into a vector representation, and the longitude and latitude of the geographic location are reduced in dimension using a multidimensional scaling method. In addition, a standardized method is used to process user behavior data to ensure that different feature data are compared on the same scale. Finally, all feature vectors are concatenated into a high-dimensional feature vector, and non-negative matrix factorization (NMF) is used to reduce the dimension to obtain the final monitoring data feature vector.

[0173] Time series prediction: Use the LSTM model to perform time series prediction on user behavior characteristics. Input user behavior data from the past period of time and predict the behavior characteristics in the future period of time.

[0174] Bayesian inference: Combine current and predicted behavior features to calculate the probability of occurrence of service features.

[0175] Monitoring strategy execution: According to the probability of occurrence of service features, monitoring nodes are classified into three categories: service feature triggered, service feature potentially triggered, and service feature low-risk triggered. Different monitoring strategies are implemented for different types of nodes to ensure efficient resource utilization and monitoring accuracy. Nodes with triggered service features are continuously monitored, nodes with potential triggering and low-risk triggering are periodically monitored, and monitoring strategies are adjusted dynamically.

[0176] Table 2 Node data table

[0177]

[0178] It can be seen from Table 2 that the present invention has a shorter response time of 400-450 milliseconds while ensuring monitoring accuracy, which is significantly better than the 500 milliseconds of the traditional method. Thanks to the NMF dimensionality reduction technology and optimized monitoring strategy, the system can still maintain high real-time performance when processing large-scale data.

[0179] Through advanced data processing and machine learning technology, efficient and accurate service SMS monitoring is achieved. It shows significant advantages in terms of the number of SMS received, behavioral feature prediction, service feature scoring and response time, which fully demonstrates the innovation and practicality of the invention. Through the comparison of experimental data, it can be clearly seen that compared with traditional methods, the invention has obvious advantages in improving monitoring accuracy, resource utilization efficiency and system real-time performance.

[0180] Example 3: In order to verify the application effect of the service SMS monitoring method and system based on big data in the monitoring of e-commerce promotion SMS, a specific example is designed. The experimental environment is a large e-commerce platform, and data of six users receiving promotion information service SMS are collected from the e-commerce platform, including SMS content, user communication code, receiving location data and user behavior data.

[0181] The user information is shown in Table 3.

[0182] Table 3 User data table

[0183]

[0184] SMS content vectorization:

[0185] Use the BERT model to convert SMS content data into high-dimensional vectors:

[0186] ;

[0187] Communication encoding vectorization:

[0188] Use embedding coding to convert the user's communication code into a vector representation:

[0189] ;

[0190] Use multidimensional scaling to reduce the latitude and longitude of the geographic location:

[0191] MDS(Geographic Location) ;

[0192] Use standardized methods to process user behavior data:

[0193] ;

[0194] Feature stitching:

[0195] User1=[0.25,0.35,0.15,...,0.12,0.22,0.32,...,0.05,0.15,0.25,...,−0.77,−0.69,−0.60];

[0196] User2=[0.45,0.25,0.35,...,0.42,0.12,0.22,...,0.35,0.25,0.15,...,1.24,1.20,1.20];

[0197] User3=[0.15,0.55,0.25,...,0.32,0.52,0.12,...,0.25,0.35,0.45,...,0.24,0.06,0.00];

[0198] User4=[0.30,0.45,0.20,...,0.25,0.35,0.45,...,0.20,0.30,0.40,...,−0.37,−0.32,0.00];

[0199] User5=[0.40,0.30,0.10,...,0.15,0.25,0.35,...,0.10,0.20,0.30,...,−1.17,−1.07,−1.20];

[0200] User6=[0.50,0.20,0.40,...,0.35,0.45,0.25,...,0.30,0.40,0.20,...,[1.25,0.82,0.60];

[0201] Use NMF to reduce the dimension and obtain the final monitoring data feature vector:

[0202] Monitoring data feature vector 1 = [0.30, 0.20, 0.40, ...];

[0203] Monitoring data feature vector 2 = [0.50, 0.30, 0.20, ...];

[0204] Monitoring data feature vector 3 = [0.20, 0.40, 0.30, ...];

[0205] Monitoring data feature vector 4 = [0.25, 0.30, 0.35, ...];

[0206] Monitoring data feature vector 5 = [0.35, 0.25, 0.20, ...];

[0207] Monitoring data feature vector 6 = [0.45, 0.35, 0.25, ...];

[0208] The calculated user feature vector and service feature score results are shown in Table 4.

[0209] Table 4 Results record

[0210]

[0211] Based on the above table, generate a monitoring strategy:

[0212] Nodes with triggered service features (service feature score ≥ 0.85):

[0213] User 2: Continuous monitoring period 1 minute.

[0214] User 6: The monitoring period lasts for 1 minute.

[0215] Nodes potentially triggered by service features (0.70≤service feature score<0.85):

[0216] User 1: Service feature score 0.82, monitoring period 2 minutes.

[0217] User 4: Service feature score 0.78, monitoring period 2.5 minutes.

[0218] Nodes with low-risk service feature triggers (service feature score < 0.70):

[0219] User 3: Service feature score 0.75, monitoring period 3 minutes.

[0220] User 5: Service feature score 0.60, monitoring period 4 minutes.

[0221] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A service SMS monitoring method based on big data, characterized in that: include: Get monitoring data of historical service SMS; Screen the monitored nodes and classify the node characteristics; Implement different monitoring strategies based on the classification results; The screening of monitored nodes and classification of node characteristics include extracting the service characteristics of the receiving node according to the sending records of the service SMS and the service characteristics content notified by the provider; Obtain all receiving nodes within the monitoring range, and determine whether they have the service characteristics by predicting the receiving nodes; if the receiving node has the probability of occurrence of the service characteristics, classify the receiving node into potential triggers of the service characteristics or low-risk triggers of the service characteristics; if the receiving node does not have the probability of occurrence of the service characteristics, screen and eliminate the receiving node; The extracting of the service feature of the receiving node refers to performing feature vectorization according to the sending record of the service SMS, which includes the following steps: Use the BERT model to convert SMS text data into high-dimensional vectors; Use embedding coding to convert the communication codes of nodes into vector representations; Use multidimensional scaling to reduce the latitude and longitude of geographic locations; Use standardized methods to process user behavior data; All feature vectors are concatenated into a high-dimensional feature vector, and NMF is used to reduce the dimension to obtain the final monitoring data feature vector: ; in, represents the monitoring data feature vector output by NMF; BERT (C) represents the BERT model vectorizing the SMS content C; Embed (N) represents Embedding vectorizing the node data N; MDS (L) represents multi-dimensional scaling to reduce the dimension of the node location data L; Norm (B) represents the standardized user behavior data B, which is used to perform the i To standardize: , represents the mean of user behavior data, represents the standard deviation of user behavior data; NMF represents non-negative matrix factorization function; Using the improved time series clustering algorithm, historical service SMS records are clustered and analyzed to extract service features. ; Where K represents the number of clusters, represents the historical record characteristic function, represents the weight of the kth cluster, C k represents the kth cluster center, is the covariance matrix of the kth cluster, T represents the transpose operation; Extract the service features informed by the provider: ; Service Characteristics Collection: ; in, represents the provider characteristic function, Indicates the service characteristics notified by the provider. Represents the semantic parsing results based on the BERT model. represents the features extracted from the domain knowledge graph, S represents the text data of the service feature content informed by the provider, Indicates the service feature content of the i-th feature notified by the provider.

2. The service SMS monitoring method based on big data according to claim 1, characterized in that: The monitoring data includes SMS content data, SMS node data, SMS receiving node location data, and user behavior data; The node data for sending and receiving text messages is the communication code between the sender and the receiver; The node location data for receiving SMS is the geographical location of the recipient when receiving the service SMS; The user behavior data refers to the recipient's usage behavior in a period of time before receiving the service SMS; Among them, service SMS is obtained by obtaining SMS data from different service providers and using interfaces or APIs to capture and import data.

3. The service SMS monitoring method based on big data as claimed in claim 2, characterized in that: The occurrence probability of the service feature includes: let B(t) represent the user behavior feature vector at time t, and use the LSTM model to predict the user behavior features in the future; ; in, The predicted future time is The user behavior feature vector at time n represents the time step of LSTM. Indicates the length of the forecast period; Suppose the service characteristics are , the historical data is H, then the occurrence probability of the service feature is expressed as: ; in, represents the probability of occurrence of a service feature given current and predicted user behavior characteristics and historical data, The value range of is [0,1]. The closer the value is to 1, the higher the probability of occurrence of the service feature is, and the closer the value is to 0, the lower the probability of occurrence is. is the joint probability of current and predicted user behavior characteristics under service characteristics and historical data; is the prior probability of the service feature under historical data; is the marginal probability of current and predicted user behavior features under historical data; ; Where N represents normal distribution; Indicated in service characteristics The mean vector under is the expected value of the user behavior characteristics; Indicated in service characteristics The covariance matrix below is the distribution of user behavior characteristics; Calculate the prior probability of service characteristics under historical data : ; Calculate the marginal probability of current and predicted user behavior characteristics under historical data : ; in, Indicates the number of times a service feature appears in historical data. Indicates the total number of historical data.

4. The service SMS monitoring method based on big data as claimed in claim 3, characterized in that: The node feature classification includes the feature vector of the user's current behavior. Whether it meets the service characteristics set ; like , then it is classified as the service feature has been triggered; let the service feature occurrence probability threshold be ,like , it is classified as a potential trigger of service features; if , it is classified as a low-risk trigger of service characteristics.

5. The service SMS monitoring method based on big data as claimed in claim 4, characterized in that: The monitoring strategy includes, for the node classification results triggered by the service feature, continuous monitoring within a complete monitoring cycle; For classification results other than those where the service feature has been triggered, periodic monitoring is performed by calculating the service feature score; The service feature scoring includes: And the current user behavior feature vector B(t), weighted calculation of service feature score S: ; Among them, S represents the service feature score, n represents the number of features, and W i represents the weight of the i-th feature, B i (t) is the behavior feature value of the i-th user at the current time t; For the i-th user behavior feature value B at the current time t i (t), we need to calculate the cosine similarity between the user behavior feature and all service features and take the minimum value; let the service feature vector set be , where j is the number of the service feature; ; Among them, cosine_similarity represents the function of calculating cosine similarity, min j Indicates taking the minimum value of the cosine similarity of all service features; According to the service feature score, the monitoring period is defined for the nodes in the potential trigger of the service feature and the low-risk trigger of the service feature respectively: nodes that potentially trigger the service feature, ; For nodes with low risk triggering of the service feature, ; Among them, S q represents the service feature score of the qth node; T represents the completion cycle length of the monitoring cycle; represents the reduction factor of nodes classified as potential triggers of service features, represents the reduction factor of nodes classified as low-risk triggers of service characteristics, and .

6. The service SMS monitoring method based on big data according to claim 5, characterized in that: Continuously monitor through the monitoring strategy, when a service message is detected in the node, the monitoring data is packaged and put into the historical record, and the historical service message is updated; The updated historical service SMS monitoring data is used to update the screening of monitored nodes and the node feature classification process, and finally the monitoring strategy is updated, and the service SMS monitoring is performed according to the updated monitoring strategy.

7. A service SMS monitoring system based on big data using the method according to any one of claims 1 to 6, characterized in that: Information management module, to obtain monitoring data of historical service SMS; The analysis module screens the monitored nodes and classifies the node characteristics; The execution module executes different monitoring strategies according to the classification results.