A short message message squeezing processing method and system based on SMPP protocol

Abstract

This application discloses a method and system for handling SMS message compression based on the SMPP protocol, relating to the field of mobile communication technology. The method includes: receiving an SMS sending request and extracting target SMS data; generating a unique hash value based on the target SMS data and performing atomic deduplication in a distributed key-value storage system; performing frequency limiting control in the distributed key-value storage system based on a time window counter for the target mobile phone number; if the current SMPP link is in a sendable state, sending the message directly; if it is in an unsendable state, storing the message in a buffer pool; when the SMPP link changes from an unsendable state to a sendable state, a controlled flushing process is triggered, retrieving messages from the buffer pool in sequence and sending them to the user; the controlled flushing process controls the message retrieval order and flushing rate, solving the problems of message compression, duplicate sending, and frequency control during communication between the mobile SMS gateway and the operator.

Description

Technical Field

[0001] This application relates to the field of mobile communication technology, and in particular to a method and system for processing compressed SMS messages based on the SMPP protocol. Background Technology

[0002] With the rapid development of mobile internet, SMS, as a reliable communication method, is widely used in various scenarios such as identity verification, notification reminders, and marketing promotions. In these application scenarios, SMS gateways typically establish connections with telecom operators' Short Message Service Centers (SMSCs) and send messages via the SMPP (Short Message Peer to Peer) protocol. When mobile SMS gateways establish connections with operator centers and send messages via protocols such as SMPP, they often encounter problems such as message "push" on the service side (message push refers to the phenomenon in message processing systems where messages to be sent or processed accumulate in the upstream system due to reasons such as link disconnection, processing speed slower than receiving speed, or downstream system rate limiting. Message push leads to increased memory usage, increased message processing latency, and in severe cases, may lead to system crashes or message loss), repeated triggering, or excessive rate limiting. Summary of the Invention

[0003] The purpose of this application is to provide a method and system for processing SMS message compression based on the SMPP protocol, which solves the problems of message compression, repeated sending and frequency control in the communication process between the mobile SMS gateway and the operator.

[0004] To achieve the above objectives, this application provides the following solution:

[0005] Firstly, this application provides a method for processing compressed SMS messages based on the SMPP protocol, including:

[0006] The system receives SMS sending requests and extracts target SMS data, which includes a mobile phone number and message content. A unique hash value is generated based on the target SMS data, and atomic deduplication is performed in a distributed key-value storage system. Frequency limiting control is then implemented in the distributed key-value storage system based on a time window counter for the target mobile phone number.

[0007] Check the current working status of the SMPP link. If it is in a sendable state, send the message directly. If it is in a non-sendable state, store the message in the buffer pool.

[0008] When the working state of the SMPP link changes from an untransmittable state to a transmittable state, a controlled flushing process is triggered, in which messages are retrieved from the buffer pool in sequence and sent to the user; the controlled flushing process is used to control the message retrieval order and flushing rate.

[0009] Optionally, generating a unique hash value based on the target SMS data specifically includes:

[0010] A unique hash value is generated based on the target SMS data using the SHA256 algorithm, MD5 algorithm, SHA1 algorithm, or MurmurHash algorithm.

[0011] Optionally, the SMS message compression processing method based on the SMPP protocol further includes:

[0012] The time window counter includes at least two types: a daily counter and a minute counter; the daily counter is used to control the maximum daily transmission volume, and the minute counter is used to control the transmission frequency within a preset time period.

[0013] Optionally, the day counter and the minute counter are set with corresponding expiration times; the expiration time of the day counter is 24 hours, and the expiration time of the minute counter is 60 seconds.

[0014] Optionally, frequency limiting control is performed in the distributed key-value storage system based on a time window counter for the target mobile phone number, specifically including:

[0015] In the distributed key-value storage system, based on the daily and minute counters of the target mobile phone number, it is determined whether the actual daily count and the actual minute count exceed the corresponding preset thresholds. If so, the request is rejected.

[0016] Optionally, the controlled venting process specifically includes:

[0017] Set the drain status to running;

[0018] Retrieve messages according to the preset retrieval order;

[0019] Control the rate at which messages are retrieved from the buffer pool;

[0020] Before sending, check the frequency limiting status of the retrieved message again;

[0021] The message is processed based on the sending result. If the sending result is successful, the message is removed from the buffer pool. If the sending result is a retryable error, the message is put back into the buffer pool. If the sending result is a non-retryable error, the message is logged and then removed from the buffer pool.

[0022] Optionally, the preset retrieval order can be a first-in-first-out (FIFO) strategy, a priority-based sorting strategy, or a dynamic priority adjustment strategy based on message timeliness.

[0023] Optionally, the SMS message compression processing method based on the SMPP protocol further includes:

[0024] When the distributed key-value storage system is unavailable, switch to local memory for frequency limiting control.

[0025] Optionally, the working states of the SMPP link include: CLOSED, CONNECTING, CONNECTED, BINDING, BOUND_TX, BOUND_RX, BOUND_TRX, UNBINDING, and UNBOUND; wherein, the SMPP link is in a transmittable state in the BOUND_TX and BOUND_TRX states.

[0026] Secondly, this application provides a text message compression processing system based on the SMPP protocol, comprising:

[0027] The preprocessing, deduplication, and frequency limiting modules are used to receive SMS sending requests and extract target SMS data, which includes a mobile phone number and message content. A unique hash value is generated based on the target SMS data, and atomic deduplication is performed in a distributed key-value storage system. Frequency limiting control is performed in the distributed key-value storage system based on a time window counter of the target mobile phone number.

[0028] The SMPP connection management module is used to check the current working status of the SMPP link. If it is in a sendable state, it will send the message directly; if it is in a non-sendable state, the message will be stored in the buffer pool.

[0029] The message buffering and emptying module is used to trigger a controlled emptying process when the working state of the SMPP link changes from an untransmittable state to a transmittable state, and to retrieve messages from the buffer pool in sequence and send them to the user; the controlled emptying process is used to control the message retrieval order and emptying rate.

[0030] According to the specific embodiments provided in this application, the following technical effects are disclosed:

[0031] This application provides a method and system for processing SMS messages based on the SMPP protocol. The method includes receiving an SMS sending request and extracting target SMS data, which includes a mobile phone number and message content. A unique hash value is generated based on the target SMS data, and atomic deduplication is performed in a distributed key-value storage system. Frequency limiting control is applied in the distributed key-value storage system based on a time window counter for the target mobile phone number. The current working status of the SMPP link is checked; if it is in a sendable state, the message is sent directly; if it is in a non-sendable state, the message is stored in a buffer pool. When the working status of the SMPP link changes from non-sendable to sendable, a controlled flushing process is triggered, retrieving messages from the buffer pool sequentially and sending them to the user. The controlled flushing process controls the message retrieval order and flushing rate. This application implements cross-instance number-level frequency limiting and content deduplication through a distributed key-value storage system, and ensures that messages are not lost when the link is unavailable based on a connection-state-aware message buffering mechanism. The controlled emptying process strategy after the connection is restored ensures that messages are sent in an orderly manner without causing sudden pressure. Based on the unique hash processing of the target address and content, it prevents the repeated sending of the same content in a short period of time, thus solving the problems of message squeezing, repeated sending and frequency control in the communication process between the mobile SMS gateway and the operator. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 This is a flowchart illustrating a method for processing compressed SMS messages based on the SMPP protocol, provided in Embodiment 1 of this application.

[0034] Figure 2 This is a schematic diagram of the message processing flow provided in Embodiment 1 of this application.

[0035] Figure 3 This is a schematic diagram of a number-level frequency limiting and deduplication process based on distributed key-value storage provided in Embodiment 1 of this application.

[0036] Figure 4 This is a schematic diagram of the connection state-driven message buffer controlled emptying process provided in Embodiment 1 of this application.

[0037] Figure 5 This is a functional module diagram of a text message compression processing system based on the SMPP protocol provided in Embodiment 2 of this application. Detailed Implementation

[0038] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0039] When mobile SMS gateways establish connections with operator centers and send messages via protocols such as SMPP, the following factors often lead to message congestion, repeated triggering, or excessive rate limiting on the service side:

[0040] 1. Link not bound: After an SMPP connection is established, a binding operation needs to be completed before messages can be sent. Messages cannot be sent during the binding process.

[0041] 2. Network fluctuations: Network instability can cause connection drops, timeouts, or packet loss.

[0042] 3. Supplier-side rate limiting: The operator sets a limit on the message sending frequency for a single account or link;

[0043] 4. Delayed Message Status Receipts: Delayed message status receipts affect subsequent processing and resending strategies;

[0044] 5. Limited connection resources: The number of SMPP connections is limited, requiring efficient use of each connection;

[0045] 6. In a distributed deployment environment with multiple instances and multiple processes, how to maintain consistency in the sending frequency of the "same mobile phone number" while ensuring that messages are not lost when the link is unavailable and can be controlled to be cleared after the link is restored is a core challenge in engineering practice.

[0046] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0047] Example 1

[0048] In one exemplary embodiment, such as Figure 1 As shown, a method for processing SMS messages based on the SMPP protocol is provided, including the following steps 101 to 103.

[0049] Step 101: Receive SMS sending request and extract target SMS data; the target SMS data includes mobile phone number and message content; generate a unique hash value based on the target SMS data and perform atomic deduplication in the distributed key-value storage system; perform frequency limiting control based on the time window counter of the target mobile phone number in the distributed key-value storage system.

[0050] Step 102: Check the current working status of the SMPP link. If it is in a sendable state, send the message directly. If it is in a non-sendable state, store the message in the buffer pool.

[0051] Step 103: When the working state of the SMPP link changes from an untransmittable state to a transmittable state, a controlled flushing process is triggered to retrieve messages from the buffer pool in sequence and send them to the user; the controlled flushing process is used to control the message retrieval order and flushing rate.

[0052] The SMS message compression processing method proposed in this application based on the SMPP protocol is a collaborative scheme of "connection state-driven buffering and emptying mechanism + number-level frequency limiting based on distributed key-value storage". By implementing steps 101 to 103, cross-instance number-level frequency limiting and content deduplication are achieved through a distributed key-value storage system. A connection state-aware message buffering mechanism ensures that messages are not lost when the link is unavailable. A controlled emptying process strategy after connection recovery ensures orderly message delivery without causing sudden pressure. Unique hash processing based on the target address and content prevents the repeated transmission of the same content within a short period, thus solving the problems of message compression, duplicate transmission, and frequency control during communication between the mobile SMS gateway and the operator.

[0053] Distributed key-value storage is a computer data storage system that stores data in the form of key-value pairs and distributes this data across multiple servers. This storage method features high performance, high scalability, and high availability, making it suitable for handling large-scale concurrent access. In Redis, a key is a unique identifier used to identify and access stored data, typically a string. For example, when storing user information in Redis, "user:1001" can be used as the key, and the corresponding value can be the user's detailed information. This application can use distributed key-value storage systems such as Redis, Memcached, and Aerospike. The implementation method can be adjusted according to the characteristics of the selected storage system without affecting the core principles and effects of the overall solution.

[0054] Message processing flow as follows Figure 2As shown. After receiving an SMS sending request, the system performs SMS request preprocessing, generates a unique hash value based on the target SMS data, performs content-level deduplication (atomic deduplication judgment is performed in the distributed key-value storage system), then performs number-level frequency limiting control, checks the current working status of the SMPP link to determine if the connection is available, and if it is in a sendable state, directly sends the message and updates the sending status; if it is in a non-sendable state, the message is stored in the buffer pool; the system monitors the working status of the SMPP link, and when the working status of the SMPP link recovers from a non-sendable state to a sendable state, a controlled drain process is triggered.

[0055] Step 101 is divided into three steps: SMS request preprocessing, distributed deduplication processing, and distributed frequency limiting control.

[0056] The SMS request preprocessing includes: 1) receiving SMS sending requests and extracting information such as the target mobile phone number and message content; 2) performing basic verification on the request parameters to ensure that the format is correct; and 3) converting the request into a standard internal format for subsequent processing.

[0057] Distributed deduplication processing involves generating a unique hash value based on the target SMS data and performing atomic deduplication judgment in a distributed key-value storage system, specifically including the following steps 201 to 203.

[0058] Step 201: Generate a unique hash value for the target mobile phone number and message content. Use a hash algorithm to combine the mobile phone number and message content and calculate the hash value.

[0059] Generating a unique hash value based on the target SMS data specifically includes using hash algorithms such as SHA256 (Secure Hash Algorithm 256-bit), MD5, SHA1, or MurmurHash to generate a unique hash value based on the target SMS data. A suitable algorithm can be selected based on performance and security requirements.

[0060] In this application, the SHA256 algorithm can be used to convert input data of arbitrary length (i.e., target SMS data) into a fixed-length output of 256 bits (32 bytes). This algorithm has the following characteristics: the same input always produces the same output; small changes in the input will lead to significant differences in the output; it is almost impossible to deduce the input from the output.

[0061] Step 202: Construct a deduplication key in Redis and attempt to set it using an atomic operation of "set if key does not exist", while setting an expiration time (e.g., 5 minutes).

[0062] Step 203: Determine if it is a duplicate based on the setting result: If the setting is successful, it means it is the first time to send, and continue processing; if the setting fails, it means it is a duplicate message, and refuse to send.

[0063] Distributed frequency limiting control, specifically, involves using a time window counter based on the target mobile phone number within the distributed key-value storage system to perform frequency limiting control. This includes steps 301-303. The time window counter includes at least two types: a daily counter and a minute counter. The daily counter controls the maximum daily transmission volume, and the minute counter controls the transmission frequency within a preset time period. Both the daily and minute counters have corresponding expiration times; the daily counter has an expiration time of 24 hours, and the minute counter has an expiration time of 60 seconds.

[0064] Step 301: Construct a daily counter and a minute counter in Redis. The daily counter is used to control the maximum daily sending volume, and the minute counter is used to control the sending frequency within a short period of time.

[0065] Step 302: Increment the atomic counter and obtain the current value, then set the corresponding expiration time for the counter (24 hours for the daily counter, 60 seconds for the minute counter).

[0066] Step 303: Determine if the message count exceeds the limit: If the daily or minute count exceeds the corresponding preset threshold, the request will be rejected.

[0067] In the distributed key-value storage system, frequency limiting control is performed based on the time window counter of the target mobile phone number. Specifically, this includes: in the distributed key-value storage system, based on the daily counter and minute counter of the target mobile phone number, determining whether the actual daily count and actual minute count exceed the corresponding preset thresholds, and if so, rejecting the sending of the request.

[0068] The number-level frequency limiting and deduplication process based on distributed key-value storage is as follows: Figure 3 As shown. The process extracts the phone number and message content, generating a unique hash value based on them. In the Redis system, the SETNX command is used to set a deduplication key. The system checks if the key already exists; if it does, the message is rejected (indicating it's a duplicate). If it doesn't exist, an incrementing daily counter checks if the daily limit is exceeded. If it does, the message is rejected (daily frequency limiting). If it doesn't exceed the daily limit, an incrementing minute counter checks if the minute limit is exceeded. If it does, the message is rejected (minute frequency limiting). If it doesn't exceed the minute limit, the message is allowed to be sent.

[0069] Check the current working status of the SMPP link to determine whether the link is in a transmit-ready state. The connection status refers to the working status of the SMPP link at different times, mainly including: CLOSED (connection closed), CONNECTING (establishing TCP connection), CONNECTED (TCP connection established but not bound), BINDING (in progress), BOUND_TX (bound to transmit mode), BOUND_RX (bound to receive mode), BOUND_TRX (bound to transmit / receive mode), UNBINDING (unbinding), and UNBOUND (unbound but connection still maintained). Among them, the SMPP link is in a transmit-ready state in the BOUND_TX and BOUND_TRX states.

[0070] If the message is in a sendable state (such as BOUND_TX or BOUND_TRX), it is sent directly. If the message is in a non-sendable state (such as UNBOUND, BINDING, or CLOSED), it is stored in the buffer pool.

[0071] It should be noted that the memory buffer pool can be replaced by persistent storage based on the file system, or it can be implemented using embedded databases such as LevelDB and RocksDB. Different implementation methods differ in performance, reliability and resource consumption, but the functions are equivalent.

[0072] The following steps involve message buffering and emptying management:

[0073] 1) Message buffer: Assign a unique ID to each message, store messages in the buffer pool, and sort them by the time they were added to the pool;

[0074] 2) Connection status change monitoring and drain triggering: a) The drain process is triggered when the connection changes from unavailable to available; b) The drain process is paused when the connection changes from available to unavailable.

[0075] 3) The controlled evacuation process includes:

[0076] a) Set the drain status to running;

[0077] b) Retrieve messages according to a preset retrieval order; where the preset retrieval order is a First-In-First-Out (FIFO) strategy, a priority-based sorting strategy, or a dynamic priority adjustment strategy based on message timeliness. Different sorting strategies affect the message processing order but do not affect the core functions of buffering and emptying. In a specific example, messages can be retrieved according to a FIFO strategy: messages are retrieved in the order they entered the buffer pool.

[0078] c) Control the drain rate, that is, control the rate at which messages are retrieved from the buffer pool to avoid instantaneous high load; the drain rate can be fixed rate control or can be replaced by adaptive rate control. The drain rate can be dynamically adjusted based on the link response time. Different strategies have different efficiency and stability, but the basic functions are the same.

[0079] d) Before sending, check the frequency limiting status of the retrieved message again to ensure that it still meets the sending conditions;

[0080] e) Send the message and process it according to the sending result. If the sending result is successful, remove the message from the buffer pool. If the sending result is a retryable error, put the message back into the message queue of the buffer pool. If the sending result is a non-retryable error, log it and remove the message from the buffer pool.

[0081] Connection state-driven message buffer controlled drain process as follows Figure 4 As shown. Monitor the working status of the SMPP link, determine if the SMPP link connection status (SMPP link working status) has changed. If not, continue monitoring; if yes, determine the type of change in the SMPP link's working status. When the SMPP link's working status recovers from an untransmittable state to a transmittable state, initiate a controlled draining process; check if the buffer pool is empty. If empty, end the controlled draining process; if not empty, control the draining rate, retrieve messages from the buffer pool, and check the frequency limiting status again; determine if the frequency limiting check passes. If it passes, send the message and determine if the message transmission was successful. If successful, remove the message from the buffer pool and continue processing the next message; if transmission fails, determine if it's a retryable error. If a retryable error, put the message back into the buffer pool to wait for replenishment and continue processing the next message; if the result is a non-retryable error, record the error and remove the message from the buffer pool, and continue processing the next message; if the frequency limiting check fails, record the frequency limiting interception, remove the message from the buffer pool, and continue processing the next message.

[0082] The SMS message compression processing method based on the SMPP protocol also includes: exception handling and degradation mechanisms: 1) Local degradation when Redis is unavailable: When the distributed key-value storage system is unavailable, it temporarily switches to local memory for frequency limiting control. 2) Adaptive handling of connection parameter changes: When a change in connection parameters is detected, the existing connection is closed and a new connection is created using the new parameters.

[0083] When Redis is unavailable, local degradation can be replaced by other backup storage or strategies. Different fault tolerance and degradation schemes can be selected according to actual needs without affecting the system's normal functions.

[0084] It should be noted that deduplication checks can be performed before or after frequency limiting checks, or in parallel to improve efficiency. Adjusting the order does not affect the final judgment result. Connection status can be checked before or after frequency limiting checks. When a connection is unavailable and messages enter the buffer pool, frequency limiting checks can be delayed until actual transmission. Different orders result in different resource utilization efficiency, but the functionality remains the same, and no limitation is made here. Exceptions can be handled independently at each step, or all exceptions can be handled centrally at the end of the process. Different handling methods differ in code structure and error recovery capabilities. The above methods provide this application with high flexibility and adaptability, enabling customization and optimization according to different application needs and technical environments, while maintaining core functions and technical characteristics.

[0085] The following example illustrates the working process of this application. Scenario: A financial application needs to send a verification code SMS to users, containing the message "Your verification code is 123456, valid for 5 minutes." The system uses a distributed deployment, with multiple application instances running simultaneously.

[0086] Step 1: Receive and preprocess SMS requests:

[0087] 1) The application receives an API request to send an SMS message;

[0088] 2) Verify that the request parameters are in the correct format;

[0089] 3) Convert to an internal standard format, including information such as mobile phone number, SMS content, and priority.

[0090] Step 2: Distributed deduplication:

[0091] 1) Generate a unique hash: Calculate the SHA256 hash value by combining the phone number and message content;

[0092] 2) Try setting up deduplication keys in Redis and setting a 5-minute expiration time;

[0093] 3) A success message indicates that this is the first time the content has been sent within 5 minutes; continue processing.

[0094] Step 3: Distributed frequency limiting control:

[0095] 1) Construct a daily counter key, using the mobile phone number and the current date as identifiers to increment the daily counter, and return a value of 3 (indicating that 3 messages have been sent today, including this one);

[0096] 2) Construct a minute counter key, using the mobile phone number and the current minute as identifiers to increment the minute counter, and return a value of 1 (indicating the first transmission in this minute);

[0097] 3) If the daily count of 3 is less than the daily limit of 10 and the minute count of 1 is less than the minute limit of 3, the frequency limiting check is passed.

[0098] Step 4: SMPP connection status check:

[0099] 1) Check that the current SMPP connection status is BINDING (binding in progress);

[0100] 2) Since the connection is not in a sendable state, the message is placed in the buffer pool.

[0101] Step 5: Message buffering and emptying:

[0102] 1) Generate a unique ID for the message, in a format that includes a timestamp and a random number;

[0103] 2) Store the message in the buffer pool and add it to the queue;

[0104] 3) At this time, there are 3 messages waiting to be sent in the buffer pool.

[0105] Step 6: Connection status change triggers drain:

[0106] 1) After 2 seconds, the SMPP connection status changes from BINDING to BOUND_TRX (transmission-ready status).

[0107] 2) When a state change triggers the emptying process, the system begins retrieving messages from the buffer pool, starting with the earliest message that entered the pool;

[0108] 3) When it's our example message's turn, following the first-in-first-out (FIFO) order:

[0109] a) Recheck the frequency limiting status (still passes);

[0110] b) Send messages via SMPP connection;

[0111] c) If the message is successfully sent, remove it from the buffer pool.

[0112] Step 7: Handling duplicate requests:

[0113] 1) Within 1 minute, the system receives another request to send the same content to the same user;

[0114] 2) The generated unique hash is the same as the previous one; the deduplication key in Redis still exists (it has not expired).

[0115] 3) If the setup operation fails, it means that the duplicate message system refuses to send the message and returns a "duplicate message" status.

[0116] This example demonstrates how this application addresses key issues such as message deduplication, frequency limiting, connection state awareness, and message buffer emptying in a distributed environment.

[0117] The collaborative scheme of "connection state-driven buffer emptying mechanism + number-level frequency limiting based on distributed key-value storage" in this application brings the following benefits:

[0118] 1. Consistent Number-Level Frequency Limiting and Deduplication: Compared to the "local memory deduplication / frequency limiting scheme," this application uses an atomic conditional write and expiration mechanism of distributed key-value storage to ensure deduplication and frequency control for the same number within a time window in multi-instance scenarios, avoiding race conditions and duplicate transmission issues. Since the frequency limiting information is stored in a shared Redis instance, rather than in the local memory of a single instance, all instances can see a consistent frequency limiting status, avoiding "frequency limiting vulnerabilities" in a distributed environment.

[0119] 2. Highly Effective Protection Against Information Bombs: This application utilizes distributed storage to perform unique hashing of target addresses and content, achieving efficient content-level deduplication and effectively preventing information bombs (the repeated transmission of the same content within a short period). Compared to relational database counting control schemes, this solution avoids frequent database transaction operations, significantly reducing system load and latency while maintaining the same deduplication effect.

[0120] 3. Controllable and Recoverable Message Squeezing: Compared to the "blocking or dropping scheme when the link is unavailable," this application automatically buffers messages when the link is abnormal and discharges them in a controlled manner after the link is restored. This avoids both message sending blocking and message loss, significantly improving system availability and reliability. Especially in environments with frequent network fluctuations, this solution can smoothly handle the connection disconnection and recovery process, ensuring business continuity.

[0121] 4. Efficient System Resource Utilization: Compared to the "message middleware buffering scheme," this application directly uses a lightweight memory buffer pool and distributed key-value storage, avoiding the complexity and overhead of introducing heavyweight message middleware. The system architecture is simpler, and resource utilization is more efficient, making it particularly suitable for SMS sending scenarios requiring low-latency processing.

[0122] 5. Loose Coupling and Easy Integration: This application loosely couples connection management, frequency control, and service orchestration, outputting standardized classification results for easy and rapid integration into existing systems. Compared to existing solutions, it offers lower integration costs, greater adaptability, and the ability to flexibly address the needs of various business scenarios and deployment environments.

[0123] 6. Observability and Diagnostic Capabilities: This application combines delivery receipts and status resolution to form an end-to-end observable link, supporting alarms, tracing, and policy optimization, thus improving system maintainability. Operations personnel can clearly understand the system status, link health, and message processing status, enabling them to quickly locate and resolve problems.

[0124] This application also provides an application scenario in which the above-mentioned SMS message squeezing processing method based on the SMPP protocol is applied. Specifically, the SMS message squeezing processing method based on the SMPP protocol provided in this embodiment can be applied in an SMS message squeezing processing scenario. The SMS message squeezing processing scenario includes a content production stage, a content processing link, and a content distribution stage; the target SMS data enters the content processing link from the content production stage and then enters the downstream content distribution stage. The SMS message squeezing processing method based on the SMPP protocol provided in this embodiment belongs to the content processing link. Specifically, in the content processing link process for SMS sending requests, a unique hash value can be generated based on the target SMS data, and atomic deduplication can be performed in a distributed key-value storage system. Frequency limiting control is performed in the distributed key-value storage system based on the time window counter of the target mobile phone number. The working status of the current SMPP link is checked. If it is in a sendable state, the message is sent directly. If it is in an unsendable state, the message is stored in a buffer pool. When the working status of the SMPP link changes from an unsendable state to a sendable state, a controlled emptying process is triggered, and messages are retrieved from the buffer pool in sequence and sent to the user.

[0125] Currently, the industry mainly adopts the following technical solutions to address the issues of message compression and frequency control in SMS transmission under the SMPP protocol:

[0126] 1. Local Memory Deduplication / Frequency Limiting Scheme: This scheme maintains a sending record table and counter in the local memory of each application instance to implement deduplication and frequency limiting control. Workflow: 1) Upon receiving an SMS sending request, query the most recent sending record of the target mobile number in local memory; 2) Determine whether the frequency limit is exceeded or duplicate content exists based on the sending record; 3) Decide whether to allow sending based on the judgment and update the local sending record; 4) If sending is allowed, send the message through the SMPP link.

[0127] Main drawbacks: 1) It is only effective for a single instance and cannot achieve consistency control in a distributed environment; 2) There are concurrency race conditions between different instances, which may lead to the same content being sent repeatedly; 3) Memory data is lost after the instance restarts, and the frequency limiting and deduplication logic fails; 4) It cannot cope with the situation of link unavailability, either blocking the business or discarding messages.

[0128] 2. Relational Database Counting Control Scheme: This scheme uses a relational database (such as MySQL) to store sending records and counts, ensuring consistency through database transactions. Workflow: 1) Upon receiving a sending request, query the database for sending records and counts for that phone number; 2) Atomically update the count and add records using database transactions; 3) Determine whether to allow sending based on the update results; 4) Update the sending status in the database after the message is sent.

[0129] Main disadvantages: 1) Requires strong consistency writes and transaction support, leading to high latency and lock contention (lock contention refers to the competitive state that occurs when multiple execution units simultaneously attempt to acquire the same mutually exclusive resource (such as database record locks, file locks, or memory regions) in a multi-threaded or multi-process environment. Lock contention causes execution units to wait for lock release, thus affecting system performance and response time. In database systems, when multiple transactions attempt to modify the same row of data, the database uses a locking mechanism to ensure data consistency, but this also causes later transactions to wait for previous transactions to complete and release the lock, forming lock contention. High lock contention will cause a decrease in system throughput and an increase in user-perceived response latency). 2) The database becomes a performance bottleneck, with limited throughput, making it difficult to cope with high-concurrency scenarios. 3) Frequent database read and write operations increase system load and cost. 4) Database connection pool resources are limited and easily exhausted during peak periods.

[0130] 3. Message Queue Buffering Solution: This solution uses a message queue (such as Kafka or RabbitMQ) as an intermediate buffer layer. SMS sending requests are first written to the queue and then processed by the consumer. Workflow: 1) The business system sends SMS requests to the message queue; 2) A dedicated consumer service retrieves and processes messages from the queue; 3) The consumer maintains the connection with the SMPP server and is responsible for message sending; 4) If sending fails, the message can be re-enqueued or written to a dead-letter queue.

[0131] Main drawbacks: 1) The strategies for sequential emptying, duplicate suppression, and failure retry after link recovery are complex; 2) The implementation cost is high, requiring additional maintenance of message middleware; 3) The coupling with the business receipt closed loop is high, making it difficult to achieve end-to-end state tracking; 4) It increases the complexity of the system and the difficulty of operation and maintenance.

[0132] 4. Blocking or Dropping Scheme When Link is Unavailable: This scheme simply blocks requests or drops messages when the link is unavailable. Workflow: 1) Monitor SMPP link status; 2) Send messages normally when the link is available; 3) When the link is unavailable, either block sending requests until timeout or reject requests directly.

[0133] Main drawbacks: 1) Blocking mode affects the availability and response time of business systems; 2) Discarding mode causes direct loss of messages and is not recoverable; 3) Cannot cope with temporary link failures; 4) Does not provide deduplication and frequency limiting mechanisms, only solves the problem of link unavailability.

[0134] The above solutions each have their limitations and cannot simultaneously solve problems such as message squeezing, deduplication, frequency limiting, and recoverability, especially in terms of consistency control in a distributed environment.

[0135] This application aims to solve the following technical problems: 1) How to achieve consistent frequency limiting and deduplication control for the same mobile phone number in a distributed environment; 2) How to handle message congestion when the link is unavailable and ensure that messages are not lost; 3) How to achieve controllable and orderly message emptying after the link is restored; 4) How to prevent information bombs (the phenomenon of repeatedly sending the same content to the same recipient within a short period of time; information bombs in this application specifically refer to the phenomenon of SMS messages with the same content being repeatedly sent to the same recipient within a short period of time. This situation is usually caused by system failure, duplicate submission or malicious attack, which will cause harassment to the recipient, increase the consumption of operator resources, and may trigger anti-spam mechanisms).

[0136] Example 2

[0137] Based on the same inventive concept, this application also provides an SMPP-based SMS message compression processing system for implementing the above-mentioned SMPP-based SMS message compression processing method. The solution provided by this system is similar to the implementation described in the above method. Therefore, the specific limitations of one or more SMPP-based SMS message compression processing system embodiments provided below can be found in the limitations of the SMPP-based SMS message compression processing method described above, and will not be repeated here.

[0138] In one exemplary embodiment, a text message squeezing processing system based on the SMPP protocol is provided, comprising:

[0139] The preprocessing, deduplication, and frequency limiting modules are used to receive SMS sending requests and extract target SMS data, which includes a mobile phone number and message content. A unique hash value is generated based on the target SMS data, and atomic deduplication is performed in a distributed key-value storage system. Frequency limiting control is performed in the distributed key-value storage system based on a time window counter of the target mobile phone number.

[0140] The SMPP connection management module is used to check the current working status of the SMPP link. If it is in a sendable state, it will send the message directly; if it is in a non-sendable state, the message will be stored in the buffer pool.

[0141] The message buffering and emptying module is used to trigger a controlled emptying process when the working state of the SMPP link changes from an untransmittable state to a transmittable state, and to retrieve messages from the buffer pool in sequence and send them to the user; the controlled emptying process is used to control the message retrieval order and emptying rate.

[0142] SMS message compression processing system based on SMPP protocol, such as Figure 5As shown. The business application system sends an SMS sending request API, extracts the target SMS data through the request preprocessing module, generates a unique hash value based on the target SMS data, and performs distributed frequency-limited deduplication, including: performing atomic deduplication judgment in the distributed key-value storage system; checking the current working status of the SMPP link through the SMPP connection management module, if it is in a sendable state, directly sending the message through the message sending module; if it is in a non-sendable state, storing the message in a buffer pool. Through the message buffering and emptying module, when the working status of the SMPP link changes from a non-sendable state to a sendable state, a controlled emptying process is triggered, retrieving messages from the buffer pool in sequence and sending them to the user; the controlled emptying process is used to control the message retrieval order and emptying rate.

[0143] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0144] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for short message message squeezing based on SMPP protocol, characterized in that, The SMS message compression processing method based on the SMPP protocol includes: The system receives SMS sending requests and extracts target SMS data, including a mobile phone number and message content. A unique hash value is generated based on the target SMS data, and atomic deduplication is performed in a distributed key-value storage system. Frequency limiting control is implemented in the distributed key-value storage system based on a time window counter for the target mobile phone number. The time window counter includes at least two types: a daily counter and a minute counter. The daily counter controls the maximum daily sending volume, and the minute counter controls the sending frequency within a preset time period. Check the current working status of the SMPP link. If it is in a sendable state, send the message directly. If it is in a non-sendable state, store the message in the buffer pool. When the SMPP link changes from a non-transmittable state to a transmittable state, a controlled flushing process is triggered, retrieving messages from the buffer pool in sequence and sending them to the user. The controlled flushing process controls the message retrieval order and flushing rate. Specifically, the controlled flushing process includes: Set the drain status to running; Retrieve messages according to the preset retrieval order; Control the rate at which messages are retrieved from the buffer pool; Before sending, check the frequency limiting status of the retrieved message again; The message is processed based on the sending result. If the sending result is successful, the message is removed from the buffer pool. If the sending result is a retryable error, the message is put back into the buffer pool. If the sending result is a non-retryable error, the message is logged and then removed from the buffer pool.

2. The SMPP protocol based short message message squeezing processing method according to claim 1, characterized in that, Generating a unique hash value based on the target SMS data specifically includes: A unique hash value is generated based on the target SMS data using the SHA256 algorithm, MD5 algorithm, SHA1 algorithm, or MurmurHash algorithm.

3. The SMS message compression processing method based on the SMPP protocol according to claim 1, characterized in that, The day counter and the minute counter are set with corresponding expiration times; the expiration time for the day counter is 24 hours, and the expiration time for the minute counter is 60 seconds.

4. The SMS message compression processing method based on the SMPP protocol according to claim 1, characterized in that, In the distributed key-value storage system, frequency limiting control is performed based on a time window counter of the target mobile phone number, specifically including: In the distributed key-value storage system, based on the daily and minute counters of the target mobile phone number, it is determined whether the actual daily count and the actual minute count exceed the corresponding preset thresholds. If so, the request is rejected.

5. The SMS message compression processing method based on the SMPP protocol according to claim 1, characterized in that, The preset retrieval order can be a first-in-first-out (FIFO) strategy, a priority-based sorting strategy, or a dynamic priority adjustment strategy based on message timeliness.

6. The SMS message compression processing method based on the SMPP protocol according to claim 1, characterized in that, The SMS message compression processing method based on the SMPP protocol also includes: When the distributed key-value storage system is unavailable, switch to local memory for frequency limiting control.

7. The SMS message compression processing method based on the SMPP protocol according to claim 1, characterized in that, The working states of the SMPP link include: CLOSED, CONNECTING, CONNECTED, BINDING, BOUND_TX, BOUND_RX, BOUND_TRX, UNBINDING, and UNBOUND; among them, the SMPP link is in a transmittable state in the BOUND_TX and BOUND_TRX states.

8. A text message compression processing system based on the SMPP protocol, characterized in that, The SMS message compression processing system based on the SMPP protocol includes: The preprocessing, deduplication, and frequency limiting modules are used to receive SMS sending requests and extract target SMS data, which includes a mobile phone number and message content. A unique hash value is generated based on the target SMS data, and atomic deduplication is performed in a distributed key-value storage system. Frequency limiting control is performed in the distributed key-value storage system based on a time window counter for the target mobile phone number. The time window counter includes at least two types: a daily counter and a minute counter. The daily counter controls the maximum daily sending volume, and the minute counter controls the sending frequency within a preset time period. The SMPP connection management module is used to check the current working status of the SMPP link. If it is in a sendable state, it will send the message directly; if it is in a non-sendable state, the message will be stored in the buffer pool. The message buffering and emptying module is used to trigger a controlled emptying process when the working state of the SMPP link changes from an untransmittable state to a transmittable state. This process retrieves messages from the buffer pool sequentially and sends them to the user. The controlled emptying process controls the message retrieval order and emptying rate. Specifically, the controlled emptying process includes: Set the drain status to running; Retrieve messages according to the preset retrieval order; Control the rate at which messages are retrieved from the buffer pool; Before sending, check the frequency limiting status of the retrieved message again; The message is processed based on the sending result. If the sending result is successful, the message is removed from the buffer pool. If the sending result is a retryable error, the message is put back into the buffer pool. If the sending result is a non-retryable error, the message is logged and then removed from the buffer pool.

Images