Method and system for dynamically identifying short message sending abnormality on basis of big data technology

Abstract

Disclosed in the present invention are a method and system for dynamically identifying a short message sending abnormality on the basis of big data technology. The method comprises: collecting historical short message data, and performing aggregation statistics collection to obtain an offline statistical result; collecting statistics about real-time short message data, and merging the real-time statistical result with the offline statistical result; generating an intelligent determination threshold on the basis of the offline statistical result; and determining, on the basis of the determination threshold, whether the real-time statistical result meets a requirement, and giving an alert for a real-time statistical result that does not meet the requirement. By collecting historical short message data, feature processing can be performed on date frequencies to obtain corresponding sending data, and fitting training is performed on the basis of the features, to construct a trend prediction model capable of predicting sending quantities at different date frequencies, so that the predicted short message sending quantities can be output; moreover, a determination threshold can be calculated on the basis of a set manual rule threshold, so that abnormal short message sending behaviors can be dynamically identified, and alerts can be provided accurately and in real time.

Description

A method and system for dynamically identifying SMS sending anomalies based on big data technology Technical Field

[0001] This invention relates to the field of SMS data anomaly identification technology, and in particular to a method and system for dynamically identifying SMS sending anomalies based on big data technology. Background Technology

[0002] With SMS becoming a widely used communication method for individuals and businesses, if telecom operators cannot promptly detect customer sending activity, it can lead to customer churn and customers receiving large amounts of spam messages in a short period. While some SMS anomaly detection methods exist, their limitations, such as insufficient rule bases and the constraints of traditional algorithms, prevent them from meeting real-time monitoring and adapting to constantly changing SMS sending behaviors. Summary of the Invention

[0003] In view of this, the present invention proposes a method and system for dynamically identifying abnormal SMS sending based on big data technology, which can dynamically identify abnormal SMS sending behavior and provide accurate and real-time warnings. The present invention provides the following technical solution:

[0004] A method for dynamically identifying SMS sending anomalies based on big data technology, the method comprising:

[0005] Collect historical SMS data, aggregate and statistically analyze the historical SMS data, and obtain offline statistical results;

[0006] Statistical analysis of real-time SMS data, and merging real-time and offline statistical results;

[0007] Intelligent judgment thresholds are generated based on offline statistical results;

[0008] The system determines whether the real-time statistical results meet the requirements based on the judgment threshold, and issues warnings for real-time statistical results that do not meet the requirements.

[0009] Furthermore, the method for collecting historical SMS data and aggregating and statistically analyzing the historical SMS data to obtain offline statistical results includes:

[0010] Historical SMS data is collected and imported into a big data warehouse;

[0011] A Spark job was designed using the offline processing framework Spark+Hive to aggregate and statistically analyze historical SMS data and obtain offline statistical results.

[0012] The offline statistical results are stored in the database.

[0013] Furthermore, the method for statistically analyzing real-time SMS data and merging the real-time statistical results with the offline statistical results includes:

[0014] The real-time processing framework Flink+Kafka is used to collect real-time SMS data.

[0015] Write real-time SMS data into Kafka and connect to Kafka using Flink's data source;

[0016] Real-time SMS data is processed using Flink;

[0017] Flink is used to group and aggregate the processed real-time SMS data to obtain real-time statistical results.

[0018] Real-time statistical results are stored in a database and merged with offline statistical results.

[0019] Furthermore, the method for generating intelligent judgment thresholds based on offline statistical results includes:

[0020] Calculate the statistical characteristics of different date frequencies based on offline statistical results;

[0021] Time features are extracted for different date frequencies to capture transmission patterns at different date frequencies;

[0022] Based on the time series model, the statistical and temporal characteristics of different date frequencies are input for fitting and training to obtain a prediction model of the sending trend at different date frequencies. The model outputs the predicted value of the number of SMS messages sent at the next date frequency.

[0023] Set manually defined thresholds and configure a threshold coefficient matrix based on these thresholds.

[0024] The judgment threshold is calculated using the threshold coefficient matrix and the predicted value.

[0025] Furthermore, the method for determining whether real-time statistical results meet the requirements based on a judgment threshold, and issuing an early warning for real-time statistical results that do not meet the requirements, includes:

[0026] Obtain real-time SMS data from Flink statistics and compare it with the judgment threshold;

[0027] If the threshold is reached, an alert message is generated and pushed to the Kafka alert message topic;

[0028] Create an alert project and consume Kafka alert message topics to push alert messages.

[0029] Furthermore, the method for calculating the threshold using the threshold coefficient matrix and the predicted value includes:

[0030] The judgment threshold is calculated by multiplying the predicted value and the threshold coefficient of a certain date frequency in the threshold coefficient matrix.

[0031] This invention further discloses a system for dynamically identifying SMS sending anomalies based on big data technology, comprising:

[0032] The offline data processing module is used to collect historical SMS data, aggregate and statistically analyze the historical SMS data, and obtain offline statistical results.

[0033] The real-time data processing module is used to statistically analyze real-time SMS data and merge the real-time statistical results with the offline statistical results.

[0034] The threshold calculation module is used to generate intelligent judgment thresholds based on offline statistical results;

[0035] The early warning judgment module is used to determine whether the real-time statistical results meet the requirements based on the judgment threshold, and to issue early warnings for real-time statistical results that do not meet the requirements.

[0036] The present invention further discloses a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0037] The present invention further discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described method.

[0038] The present invention further discloses a computer program product, including a computer program that implements the above-described method when executed by a processor.

[0039] According to the technical solution of the present invention, by collecting historical SMS data, statistical and temporal features can be processed for different date frequencies to obtain corresponding sending pattern data. Based on the statistical and temporal features, fitting training can be performed to construct a trend prediction model that can predict the number of SMS messages sent at different date frequencies. This model can output the predicted number of SMS messages sent. In addition, by combining it with a set manual rule threshold, a judgment threshold can be calculated, thereby dynamically identifying abnormal SMS sending behavior and providing accurate and real-time warnings. This makes an important contribution to protecting the rights and interests of users and telecommunications operators. Attached Figure Description

[0040] For illustrative and not limiting purposes, the present invention will now be described in conjunction with embodiments and accompanying drawings, wherein:

[0041] Figure 1 is a flowchart illustrating the method for dynamically identifying SMS sending anomalies based on big data technology in an embodiment of the present invention.

[0042] Figure 2 is a schematic diagram of the system structure for dynamically identifying SMS sending anomalies based on big data technology in an embodiment of the present invention.

[0043] Figure 3 is a schematic diagram of the electronic device structure in an embodiment of the present invention;

[0044] Figure 4 is a schematic diagram of the offline processing flow of historical SMS data in an embodiment of the present invention;

[0045] Figure 5 is a schematic diagram of the threshold determination process for real-time SMS data in an embodiment of the present invention. Detailed Implementation

[0046] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present application.

[0047] It should be noted that the terms "first," "second," etc., in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0048] In this application, the terms "upper," "lower," "left," "right," "front," "rear," "top," "bottom," "inner," "outer," "middle," "vertical," "horizontal," "lateral," and "longitudinal" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are primarily for the purpose of better describing this application and its embodiments, and are not intended to limit the indicated device, element, or component to having a specific orientation, or to be constructed and operated in a specific orientation.

[0049] Furthermore, some of the aforementioned terms, besides indicating location or positional relationships, may also have other meanings. For example, the term "above" may, in certain circumstances, indicate a dependency or connection. Those skilled in the art can understand the specific meaning of these terms in this application based on the specific circumstances. Additionally, the term "multiple" should mean two or more.

[0050] It should be noted that, where there is no conflict, the embodiments and features of the embodiments in this application can be combined with each other. The embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0051] For real-time data streams like SMS messages, real-time monitoring typically relies on rule-based processing. However, rule updates always lag behind the sending of abnormal SMS messages, making it impossible to meet the demands of real-time detection and adapt to constantly changing SMS sending behavior.

[0052] This implementation method discloses a method and system for dynamically identifying SMS sending anomalies based on big data technology.

[0053] Referring to Figure 1, the method for dynamically identifying SMS sending anomalies based on big data technology includes:

[0054] S100: Collects historical SMS data, aggregates and statistically analyzes the historical SMS data, and obtains offline statistical results.

[0055] Referring to Figures 1 and 4, this implementation method uses the offline processing framework Spark+Hive for collecting and statistically analyzing historical SMS data, identifying the number of SMS messages sent by each mobile phone number. Specifically:

[0056] S110: Collect historical SMS data and import it into a big data warehouse.

[0057] S120: A Spark job is designed using the offline processing framework Spark+Hive to aggregate and statistically analyze historical SMS data and obtain offline statistical results. Specifically, the aggregation and statistics are performed by the sending mobile phone number to count the sending frequency and the number of messages sent. Hive is used as a data warehouse for storage. Data tables are created in Hive, and historical SMS data is loaded into the data tables created in Hive. The designed Spark job batch processes the historical SMS data in the data tables and performs statistical analysis on the sending frequency and the number of messages sent by the sending mobile phone number.

[0058] S130: Store the offline statistical results in the database. Specifically: Create a storage table in the database to store the offline statistical results.

[0059] Referring to Figures 1, 4 and 5, S200: Statistically analyze real-time SMS data and merge the real-time statistical results with the offline statistical results.

[0060] For real-time generated SMS data, this implementation uses the real-time processing framework Flink+Kafka for statistics, identifying the amount of SMS data sent by each mobile phone number based on the same mobile phone number dimension as historical data. Specifically:

[0061] S210: Employs the real-time processing framework Flink+Kafka to collect real-time SMS data.

[0062] S220: Write real-time SMS data to Kafka and connect to Kafka using Flink's data source. To ensure accurate consumption of the real-time SMS data, relevant consumer information also needs to be configured.

[0063] S230: Processes real-time SMS data using Flink. This involves parsing, cleaning, and transforming the real-time SMS data for subsequent statistical analysis.

[0064] S240: The processed real-time SMS data is grouped and aggregated using Flink to obtain real-time statistical results. This utilizes Flink's window operations, grouping and aggregating SMS data by time windows. For example, a one-minute time window is used to cluster and statistically analyze the number of sent SMS messages. If the frequency of SMS messages sent is found to be 100 messages / min, this frequency can be used as data to determine whether there are any anomalies in the real-time SMS data.

[0065] S250: Stores real-time statistical results in a database and merges them with offline statistical data.

[0066] In step S130, a storage table is created to store both real-time and offline statistical results. After merging, an index is created to facilitate quick retrieval and use by the model.

[0067] In this embodiment, a method for real-time SMS data processing using Spark Streaming instead of the Flink framework is also disclosed, which can also achieve real-time processing of SMS data.

[0068] After obtaining the offline statistical results of historical SMS data and the real-time statistical data of real-time SMS data, it is necessary to determine whether there are any anomalies in the real-time statistical data. Therefore, the identification method of this embodiment further includes:

[0069] S300: Generate intelligent judgment thresholds based on offline statistical results. Offline statistical results can reflect the patterns and regularities of SMS sending over a period of time. However, for real-time rule changes or continuous adjustments on special dates, relying solely on offline statistical results to judge anomalies in real-time SMS data is inaccurate. Therefore, this implementation incorporates intelligent model prediction and manual rules into the generation of judgment thresholds. Specifically:

[0070] S310: Calculate the statistical characteristics of different date frequencies based on offline statistical results. Specifically:

[0071] The offline statistical results reflecting historical SMS data are preprocessed, abnormal noise data is replaced, and differential transformation is performed to obtain a stable offline statistical result sequence. Aggregate statistics are performed according to the date frequency of monthly, weekly, daily, and hourly sending, and statistical characteristics such as upper limit, lower limit, mean, and median are calculated to obtain statistical characteristics for different frequencies: Lower = Q1 - 1.5·IQR; Upper = Q3 + 1.5·IQR;

[0072] Where Q1 represents the lower quartile of historical SMS data when sorted in ascending order, meaning that data below this point accounts for 25% of the total data; Q3 represents the upper quartile of historical SMS data when sorted in ascending order, meaning that data below this point accounts for 75% of the total data; IQR represents the interquartile range, calculated as: IQR = Q3 - Q1; Lower represents the lower limit statistical value of SMS sending data, i.e., the lower limit standard used to identify SMS sending anomalies; Upper represents the upper limit statistical value of SMS sending data, i.e., the upper limit standard used to identify SMS sending anomalies; X represents the value to be replaced, which can be the median, the upper limit statistical value, or the lower limit statistical value; and x′ is the value after replacement.

[0073] S320: Extract time features for different date frequencies to capture transmission patterns for different date frequencies. For example, for monthly transmission frequencies, the time feature could be the week number of the month, or similarly, the day number of the week, the hour of the day, whether it is a weekday, or a holiday, etc. Transmission patterns for different date frequencies can be obtained through time feature extraction.

[0074] S330: Based on a time series model, statistical and temporal characteristics of different date frequencies are input for fitting and training to obtain a sending trend prediction model for different date frequencies. This model outputs a predicted value for the number of SMS messages sent on the next date frequency. The time series model can also be replaced with a similar time series model; this implementation method is not limited to this. Specifically:

[0075] The statistical and temporal characteristics of each date frequency are input into the Time Series model TimesNet using a sliding window. After multiple rounds of iterative training and validation, monthly, weekly, daily, and hourly transmission trend prediction models are obtained. A time sliding window N is defined, and the input is defined as M = [m1, m2, ..., m...]. n ], where m n Let K represent the feature vector of the nth input, and let K = [m] be the output. n+1 If the current window model input and output are K, then K = TimesNet(M);

[0076] For example, a sending trend prediction model built based on different date frequencies may include:

[0077] Monthly frequency model: The input data consists of the statistical and temporal characteristics of historical SMS data over the past N months, and the output is the predicted value of SMS sending data for the next month;

[0078] Weekly frequency model: The input data consists of the statistical and temporal characteristics of historical SMS data from the past N weeks, and the output is the predicted value of SMS sending data for the next week;

[0079] Daily frequency model: The input data consists of the statistical and temporal characteristics of historical SMS data from the past N days, and the output is the predicted value of SMS sending data for the next day;

[0080] Time-frequency model: The input data consists of the statistical and temporal characteristics of historical SMS data from the past N hours, and the output is the predicted value of SMS sending data for the next hour.

[0081] S340: Set the threshold for manual rules and configure the threshold coefficient matrix based on the threshold for manual rules.

[0082] Manual rules can be formulated based on specific business knowledge and experience. They are used to detect abnormal behavior by defining rules. For example, during special holidays, different types of companies may send a large number of SMS messages related to policies or business. In such cases, manual rules are needed to adjust thresholds. Furthermore, different threshold coefficients can be given for detection at different date frequencies. In step S330, the predicted value output by the model is obtained based on historical sending data. When triggering an alarm, a threshold needs to be manually set to determine the final result. The manual rule threshold, in addition to the threshold for the number of SMS messages sent, also includes rules such as whether to trigger an alarm when the number of SMS messages sent is 0, and the number of consecutive alarms. These manual rules are uniformly configured as a threshold coefficient matrix T: T = [t1, t2, ..., t n ];

[0083] Among them, t n This represents the threshold for the nth rule, representing the frequency of each date.

[0084] S350: The judgment threshold is calculated using the threshold coefficient matrix and the predicted value.

[0085] Based on steps S330 and S340, the threshold is calculated as follows:

[0086] As can be seen from the above, by constructing a sending trend prediction model and combining it with manual rules, we can make full use of the automation capabilities of the sending trend prediction model and the professional knowledge of manual rules, making the detection of abnormal SMS sending behavior more comprehensive and accurate, and also allowing for flexible adjustments and optimizations based on special dates and other needs.

[0087] After obtaining the judgment threshold, it is stored in Redis. When making subsequent warning judgments, the threshold can be directly compared to determine whether to issue a warning.

[0088] S400: Determines whether real-time statistical results meet requirements based on a judgment threshold, and issues an alert for real-time statistical results that do not meet the requirements. Specifically:

[0089] S410: Obtain real-time SMS data from Flink statistics and compare it with the judgment threshold.

[0090] Since SMS sending data is generated in real time, after acquiring and statistically processing real-time SMS data according to date frequency, the real-time statistical results can be compared with the judgment threshold stored in Redis, which can quickly and accurately identify the difference between the real-time SMS data and the judgment threshold.

[0091] S420: If the judgment threshold is reached, an alert message is generated and pushed to the Kafka alert message topic.

[0092] S430: Create an alert project and consume Kafka alert message topics to implement alert message push. For example, create an alert project using Spring Boot, and use this project to consume alert message topics from Kafka. Further, write functions to process alert messages and push them, such as sending SMS messages, emails, or app push notifications.

[0093] Since historical SMS data is stored after threshold judgments are performed at certain date frequencies, the large data warehouse used for storage will gradually fill up over time. Therefore, it is necessary to regularly back up and clean up historical data to avoid performance degradation due to excessive data accumulation. At the same time, it is also necessary to use the CDH management platform to monitor the server's CPU, memory, and other resources to prevent them from being consumed by other processes during model training and threshold judgment.

[0094] Compared with existing SMS alert methods, this invention has the following advantages and effects:

[0095] Massive data: Using big data technology, massive amounts of messages can be processed, and based on the scale of the big data service cluster, early warnings of TB and PB level data volumes can be achieved.

[0096] Accuracy: Based on massive amounts of historical SMS data, the data undergoes cleaning and other preprocessing to make the sending behavior statistics more accurate, providing more precise results for the model.

[0097] Dynamic threshold: The warning threshold can be dynamically adjusted based on real-time data, which can adapt to constantly changing abnormal SMS sending behavior and reduce the false alarm rate.

[0098] Real-time performance: It can collect and analyze data in real time, and issue early warning notifications within seconds based on judgment thresholds, thus protecting the rights and interests of users and telecommunications operators in a timely manner.

[0099] Flexible and scalable: The early warning project, developed based on Java, can be easily integrated with other systems and its functions can be expanded according to actual needs.

[0100] Referring to Figure 2, this embodiment further provides a system for dynamically identifying SMS sending anomalies based on big data technology, including:

[0101] The offline data processing module 21 is used to collect historical SMS data and perform aggregation and statistics on the historical SMS data to obtain offline statistical results. Specifically, it includes: collecting historical SMS data and importing it into a big data warehouse; designing a Spark job using the offline processing framework Spark+Hive to perform aggregation and statistics on the historical SMS data and obtain offline statistical results; and storing the offline statistical results in a database.

[0102] The real-time data processing module 22 is used to statistically analyze real-time SMS data and merge the real-time statistical results with the offline statistical results. Specifically, it uses the real-time processing framework Flink+Kafka to statistically analyze real-time SMS data; writes the real-time SMS data into Kafka and connects to Kafka using Flink's data source; processes the real-time SMS data using Flink; groups and aggregates the processed real-time SMS data using Flink to obtain real-time statistical results; stores the real-time statistical results in the database and merges them with the offline statistical results.

[0103] The judgment threshold calculation module 23 is used to generate intelligent judgment thresholds based on offline statistical results. Specifically, it includes: calculating statistical characteristics of different date frequencies based on offline statistical results; extracting time features for different date frequencies to capture the sending patterns of different date frequencies; using a time series model as a basis, inputting statistical and time features of different date frequencies for fitting training to obtain a sending trend prediction model for different date frequencies, which outputs a predicted value for the number of SMS messages sent on the next date frequency; setting a manual rule threshold and configuring a threshold coefficient matrix based on the manual rule threshold; and calculating the judgment threshold using the threshold coefficient matrix and the predicted value.

[0104] The early warning judgment module 24 is used to determine whether the real-time statistical results meet the requirements based on the judgment threshold, and to issue an early warning for real-time statistical results that do not meet the requirements. Specifically, it obtains real-time SMS data from Flink statistics and compares it with the judgment threshold. If the judgment threshold is reached, an early warning message is generated and pushed to the Kafka early warning message topic. An early warning project is created and consumes the Kafka early warning message topic to realize the push of early warning messages.

[0105] Figure 3 is a schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention. As shown in Figure 3, the electronic device 50 includes: a processor 501, a memory 502, and a bus 503.

[0106] The processor 501 and the memory 502 communicate with each other via the bus 503; the processor 501 is used to call the program instructions in the memory 502 to execute the method for dynamically identifying SMS sending anomalies based on big data technology provided in the above-described method implementations.

[0107] This embodiment provides a non-transitory computer-readable storage medium that stores computer instructions. The computer instructions cause the computer to execute the method for dynamically identifying SMS sending anomalies based on big data technology provided in the above-described method implementations.

[0108] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various storage media capable of storing program code, such as ROM, RAM, magnetic disk, or optical disk.

[0109] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0110] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of each embodiment or some parts of the embodiments.

[0111] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for dynamically identifying short message sending abnormalities based on big data technology, characterized in that, The method includes: Collect historical SMS data, aggregate and statistically analyze the historical SMS data, and obtain offline statistical results; Statistical analysis of real-time SMS data, and merging real-time and offline statistical results; Intelligent judgment thresholds are generated based on offline statistical results; The system determines whether the real-time statistical results meet the requirements based on the judgment threshold, and issues warnings for real-time statistical results that do not meet the requirements. 2.The method of claim 1, wherein, The method for collecting historical SMS data and aggregating and statistically analyzing the historical SMS data to obtain offline statistical results includes: Historical SMS data is collected and imported into a big data warehouse; A Spark job was designed using the offline processing framework Spark+Hive to aggregate and statistically analyze historical SMS data and obtain offline statistical results. The offline statistical results are stored in the database. 3.The method of claim 1, wherein, The method for statistically analyzing real-time SMS data and merging the real-time statistical results with offline statistical results includes: The real-time processing framework Flink+Kafka is used to collect real-time SMS data. Write real-time SMS data into Kafka and connect to Kafka using Flink's data source; Real-time SMS data is processed using Flink; Flink is used to group and aggregate the processed real-time SMS data to obtain real-time statistical results. Real-time statistical results are stored in a database and merged with offline statistical results. 4.The method of claim 1, wherein, The method for generating intelligent judgment thresholds based on offline statistical results includes: Calculate the statistical characteristics of different date frequencies based on offline statistical results; Time features are extracted for different date frequencies to capture transmission patterns at different date frequencies; Based on the time series model, the statistical and temporal characteristics of different date frequencies are input for fitting and training to obtain a prediction model of the sending trend at different date frequencies. The model outputs the predicted value of the number of SMS messages sent at the next date frequency. Set manually defined thresholds and configure a threshold coefficient matrix based on these thresholds. The judgment threshold is calculated using the threshold coefficient matrix and the predicted value. 5.The method of claim 3, wherein, The method for determining whether real-time statistical results meet the requirements based on a judgment threshold, and issuing early warnings for real-time statistical results that do not meet the requirements, includes: Obtain real-time SMS data from Flink statistics and compare it with the judgment threshold; If the threshold is reached, an alert message is generated and pushed to the Kafka alert message topic; Create an alert project and consume Kafka alert message topics to push alert messages. 6.The method of claim 4, wherein, The method for calculating the threshold using the threshold coefficient matrix and the predicted value includes: The judgment threshold is calculated by multiplying the predicted value and the threshold coefficient of a certain date frequency in the threshold coefficient matrix.

7. A system for dynamically identifying short message sending abnormalities based on big data technology, characterized in that, include: The offline data processing module is used to collect historical SMS data, aggregate and statistically analyze the historical SMS data, and obtain offline statistical results. The real-time data processing module is used to statistically analyze real-time SMS data and merge the real-time statistical results with the offline statistical results. The threshold calculation module is used to generate intelligent judgment thresholds based on offline statistical results; The early warning judgment module is used to determine whether the real-time statistical results meet the requirements based on the judgment threshold, and to issue early warnings for real-time statistical results that do not meet the requirements.

8. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method described in any one of claims 1-6.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method described in any one of claims 1-6.

10. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-6.

Images