Data processing method and device for short message template

Abstract

The application discloses a kind of data processing method and device for short message template, wherein method includes cloud receiving anonymous observation data sent by terminal, similarity matching is carried out between short message semantic identifier semantic_id and the semantic identifier corresponding to each short message template in short message template library;Based on the matching result, determine the aggregation cluster to which the anonymous observation data belongs;Based on the aggregation ID of aggregation cluster, the specified information of aggregation cluster, the abstract of analysis result, and / or non-sensitive statistical attribute information update specified structure's aggregate statistics table.The cloud relies on pre-constructed short message template library, receives terminal anonymous report and the irreversible short message semantic identifier, completes the same kind short message template aggregation classification by similarity matching, updates aggregate statistics table, and can continuously mine high-frequency unknown template to supplement library iteration, realizes global short message template observation and control optimization under the condition of privacy security without obtaining short message original text.

Description

Technical Field

[0001] This invention relates to the field of text processing technology, and specifically to a data processing method and apparatus for SMS templates. Background Technology

[0002] Currently, intelligent SMS capabilities in the industry are typically deployed locally on the mobile terminal. After receiving an SMS message, the terminal extracts key information and generates cards using an offline parsing engine, such as logistics reminders, bill reminders, travel reminders, verification codes, hotel notifications, etc. This type of solution has advantages such as low latency, offline operation, and strong privacy protection, making it suitable for large-scale deployment on partner mobile phones.

[0003] However, purely offline deployment also brings significant problems. Because the original SMS message is not transmitted back to the cloud, the system struggles to obtain the following information: 1. The distribution of SMS templates actually received by users, such as which SMS templates appear most frequently and which industry templates are growing the fastest; 2. The parsing performance of various SMS templates, such as which templates have been parsed stably, and which templates have low card hit rates and incomplete field extraction; 3. New SMS templates that have emerged, such as those not pre-set by partners, not covered by the rule base, or not yet learned by the model.

[0004] In existing technologies, directly uploading the original text of the SMS message, while convenient for statistical analysis, poses significant privacy risks and typically fails to meet terminal privacy protection requirements. Uploading only the traditional hash value of the SMS message, such as MD5 or SHA, while irreversible, makes it difficult to generate aggregated statistical results for a global SMS template.

[0005] Therefore, a new technical solution is needed to enable the terminal to determine the distribution of global SMS templates without revealing the original text of the SMS message. Summary of the Invention

[0006] The main objective of this invention is to provide a data processing method and apparatus for SMS templates to address the shortcomings in related technologies.

[0007] To achieve the above objectives, according to a first aspect of the present invention, a data processing method for SMS templates is provided, comprising receiving anonymous observation data sent by a terminal in a cloud, wherein the terminal receives SMS text, preprocesses the SMS text to obtain an SMS semantic identifier (semantic_id); uploads the anonymous observation data to the cloud, the anonymous observation data including the semantic_id, a parsing result summary, and / or non-sensitive statistical attributes; performs similarity matching between the SMS semantic identifier (semantic_id) and the semantic identifiers corresponding to each SMS template in an SMS template library; determines the cluster to which the anonymous observation data belongs based on the matching result; and updates an aggregate statistics table with a specified structure based on the cluster ID, specified information of the cluster, parsing result summary, and / or non-sensitive statistical attribute information.

[0008] Optionally, determining the cluster to which the anonymous observation data belongs based on the matching result includes: if the semantic_id corresponding to the target SMS template or the cluster ID of the target cluster is matched, then the cluster to which the semantic_id corresponding to the target SMS template belongs is taken as the cluster to which the anonymous observation data belongs, or the target cluster is directly taken as the cluster to which the anonymous observation data belongs.

[0009] Optionally, based on the matching results, determining the cluster to which the anonymous observation data belongs includes: if no matching SMS template's corresponding semantic_id or any cluster's aggregation ID can be found, then the semantic_id in the data is added to the unknown template pool, and an unknown cluster is created for the content in the template pool.

[0010] Optionally, determining the cluster to which the anonymous observation data belongs based on the matching results includes: if no matching is found for the semantic_id of any SMS template or the cluster ID of any cluster, then based on the semantic similarity calculated in the matching process, the semantic_id in the data is added to the unknown template pool, and an unknown cluster is created for the content in the template pool.

[0011] Optionally, the semantic_id of the unknown template pool, or the unknown aggregation cluster that has already been aggregated in the template pool, is marked as an object to be completed; SMS templates are then added for the objects to be completed.

[0012] Optionally, preprocessing the original SMS text to obtain an irreversible SMS semantic identifier (semantic_id) that preserves semantic proximity includes: standardizing the SMS text; extracting character n-gram features from the standardized text; mapping the features to a fixed-dimensional sparse vector using feature hashing; performing L2 normalization on the fixed-dimensional sparse vector and generating a fixed-length bit signature through symbolic random projection; and performing base64url encoding on the bit signature to obtain the semantic_id.

[0013] Optionally, the aggregate statistics table with the specified structure includes a representative SMS template, an aggregate ID of the aggregate cluster, a set of included SMS semantic IDs, frequency, and parsing performance metrics. The representative SMS template describes the content of the SMS template represented by the aggregate cluster. The set of included SMS semantic IDs represents the multiple semantic_ids merged into the aggregate cluster. After receiving the SMS text, the terminal also calls a local offline intelligent SMS engine to parse the SMS text, generate a parsing result summary, and determine the parsing performance metrics based on the parsing result summary.

[0014] According to a second aspect of the present invention, a data processing system for SMS templates is provided, comprising: a receiving unit, which receives anonymous observation data sent by a terminal via a cloud, wherein the terminal receives SMS text, preprocesses the SMS text to obtain an SMS semantic identifier (semantic_id); uploads the anonymous observation data to the cloud, the anonymous observation data including the semantic_id, a parsing result summary, and / or non-sensitive statistical attributes; a matching and processing unit, which performs similarity matching between the SMS semantic identifier (semantic_id) and the semantic identifiers corresponding to each SMS template in an SMS template library; and an aggregation unit, which determines the aggregation cluster to which the anonymous observation data belongs based on the matching result; and updates an aggregation statistics table with a specified structure based on the aggregation ID of the aggregation cluster, specified information of the aggregation cluster, a parsing result summary, and / or non-sensitive statistical attribute information.

[0015] According to a third aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing the computer to perform the method described in any one of the first aspects.

[0016] According to a fourth aspect of the present invention, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to cause the at least one processor to perform the method described in any implementation of the first aspect.

[0017] This embodiment describes a data processing method and apparatus for SMS templates. The method includes receiving anonymous observation data sent by a terminal to the cloud. The terminal receives SMS text, preprocesses the text to obtain a semantic identifier (semantic_id), uploads the anonymous observation data to the cloud (the anonymous observation data includes the semantic_id, a parsing result summary, and / or non-sensitive statistical attributes), performs similarity matching between the semantic identifier (semantic_id) and the semantic identifiers corresponding to various SMS templates in the SMS template library, determines the cluster to which the anonymous observation data belongs based on the matching result, and updates a specified structured aggregation statistics table based on the cluster ID, specified information of the cluster, parsing result summary, and / or non-sensitive statistical attribute information. The cloud relies on a pre-built SMS template library, receives anonymously reported and irreversible SMS semantic identifiers from the terminal, aggregates and categorizes similar SMS templates through similarity matching, updates the aggregation statistics table, and continuously mines high-frequency unknown templates for library replenishment and iteration. This achieves privacy-preserving global SMS template observation and management optimization without obtaining the original SMS text. Attached Figure Description

[0018] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is a flowchart of a data processing method for SMS templates according to an embodiment of the present invention; Figure 2 This is a schematic diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0020] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0021] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of the invention described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0022] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0023] According to embodiments of the present invention, a data processing method and apparatus for SMS templates are provided, such as... Figure 1 As shown, steps 101 to 103 are included below: Step 101: The cloud receives anonymous observation data sent by the terminal, wherein the terminal receives SMS text, preprocesses the SMS text to obtain the SMS semantic identifier semantic_id; and uploads the anonymous observation data to the cloud, wherein the anonymous observation data includes the semantic_id, a parsing result summary and / or non-sensitive statistical attributes.

[0024] In this step, an SMS template refers to a type of SMS content that shares a similar semantic structure but differs only in variable fields within the same business scenario. For example, "Your package has arrived at {location}, and the pickup code is {pickup code}" is a package pickup SMS template, where the location and pickup code are variable fields.

[0025] The overall architecture adopts a collaborative approach of "device-side semantic identifier generation + cloud-side template mapping and aggregation analysis." Upon receiving an SMS message, the terminal calls its local offline intelligent SMS parsing engine to parse the message and obtain a parsing result summary. Simultaneously, it performs normalization, feature extraction, feature compression, and semantic projection on the original SMS text to generate an irreversible SMS semantic identifier (semantic_id) that preserves local semantic proximity. The content uploaded by the terminal to the cloud does not include the original SMS text, but includes: the semantic_id, the parsing result summary, and optional non-sensitive statistical attributes (such as terminal version, time binning, region binning, and channel binning).

[0026] While generating the semantic_id, the terminal records the processing result of the local intelligent SMS engine for the SMS message, forming a parsing result summary. The summary may include, but is not limited to, whether a card was generated; the matched business category; the number of extracted key fields; the field completeness rate; and the parsing version number or rule version number.

[0027] As an optional implementation of this embodiment, preprocessing the original SMS text to obtain an irreversible SMS semantic identifier (semantic_id) that preserves semantic proximity includes: standardizing the SMS text; extracting character n-gram features from the standardized text; mapping the feature hash to a fixed-dimensional sparse vector; performing L2 normalization on the fixed-dimensional sparse vector and generating a fixed-length bit signature through symbolic random projection; and performing base64url encoding on the bit signature to obtain the semantic_id.

[0028] In this optional implementation, the terminal side generates SMS semantic identifiers using the following steps: standardize the SMS text; extract character n-gram features; map the features to a fixed-dimensional sparse vector through feature hashing; perform L2 normalization; generate a fixed-length bit signature through symbolic random projection; and perform base64url encoding on the bit signature to obtain the semantic_id.

[0029] For example, the terminal performs the following processing on the original SMS text: text normalization and desensitization processing, including removing leading and trailing whitespace, multi-space compression, truncating common unsubscribe information at the end, and removing variables such as names, amounts, and links; extracting character-level or word-level local features, preferably character 3-gram features or 4-gram features; performing feature hashing on the features and mapping them to a fixed-dimensional vector space; normalizing the vectors; generating a fixed-length binary signature based on random projection, locality-sensitive hashing, or other compression methods that maintain similarity; and encoding the binary signature to obtain the SMS semantic identifier `semantic_id`.

[0030] The semantic identifier generation method meets the following requirements: the semantic identifiers generated by the same or highly similar SMS templates are close in distance space; the original SMS text cannot be directly recovered from the semantic identifiers; the computational load of the generation process is controllable and suitable for local execution on the terminal.

[0031] Step 102: Perform similarity matching between the SMS semantic identifier (semantic_id) and the semantic identifiers corresponding to each SMS template in the SMS template library.

[0032] In this step, the terminal acquires SMS template samples from external sources to build a pre-collected template library. For each template sample, the original template text, template source, industry category, version information, and corresponding template semantic identifier are recorded. Multiple template samples that are semantically similar can be further merged into the same aggregation cluster, and a unique aggregation_id is assigned to this aggregation cluster.

[0033] During matching, the template's semantic_id can be matched first. If no match is found, the aggregate ID can be matched. For example, the reported semantic_id is first matched against the semantic_id of a known SMS template. If no match is found, a similarity match is performed with the cluster center of an existing aggregate cluster, representing the semantic_id or a set of semantic identifiers. If no match is found again, the template is entered into the unknown template pool.

[0034] Step 103: Based on the matching results, determine the cluster to which the anonymous observation data belongs; update the aggregate statistics table with a specified structure based on the cluster ID, specified information of the cluster, parsing result summary, and / or non-sensitive statistical attribute information; wherein, each SMS template corresponds to its own semantic_id in advance; the cloud merges SMS templates with similar semantics to divide them into several clusters, and assigns a unique cluster ID to each cluster; the semantic_id corresponding to each preset SMS template is pre-generated, and the cluster ID is used as a semantic identifier.

[0035] In this embodiment, the terminal generates irreversible semantic identifiers for the original text message; the semantic identifiers of similar text message templates can be automatically aggregated in the cloud, rather than being limited to exact matching of completely identical text; combined with a pre-collected text message template library in the cloud, an interpretable representative template is provided for the aggregation results.

[0036] A pre-maintained SMS template library is used in the cloud. The original SMS messages in the template library can be provided by merchants, partner channels, test sample libraries, or historical templates. The cloud pre-generates a corresponding semantic_id for each representative SMS message in the template library and further establishes a mapping relationship between templates and aggregation clusters, with each aggregation cluster corresponding to an aggregation_id. Through the above processing, the cloud generates a continuously updated aggregation statistics table.

[0037] As an optional implementation of this embodiment, determining the cluster to which the anonymous observation data belongs based on the matching result includes: if the semantic_id corresponding to the target SMS template or the cluster ID of the target cluster is matched, then the cluster to which the semantic_id corresponding to the target SMS template belongs is taken as the cluster to which the anonymous observation data belongs, or the target cluster is directly taken as the cluster to which the anonymous observation data belongs.

[0038] In this optional implementation, the cloud pre-generates a corresponding semantic_id for each representative SMS message in the template library, and further establishes a mapping relationship between templates and aggregation clusters. Each aggregation cluster corresponds to an aggregation_id.

[0039] As an optional implementation of this embodiment, determining the cluster to which the anonymous observation data belongs based on the matching results includes: if no matching is found for the semantic_id corresponding to any SMS template or the aggregation ID of any cluster, then the semantic_id in the data is added to the unknown template pool, and an unknown cluster is created for the content in the template pool.

[0040] After receiving the data reported by the client, the cloud compares the `semantic_id` with the semantic identifiers in the known template library. Matching methods can include exact matching or similarity matching. Similarity matching can be based on Hamming distance, cosine similarity, locality-sensitive hash bucket proximity, or other equivalent distance metrics. If the reported `semantic_id` meets a preset matching condition with a known template or a known aggregation cluster, it is merged into the corresponding `aggregation_id`. If it does not meet the condition, it is added to the unknown template pool, and a new unknown aggregation cluster can be created based on similarity.

[0041] As an optional implementation of this embodiment, the semantic_id of the unknown template pool, or the unknown aggregate cluster that has been aggregated in the template pool, is marked as an object to be completed; SMS templates are added for the objects to be completed.

[0042] In this optional implementation, completion refers to supplementing and associating one or more representative SMS templates for unknown aggregation clusters. The specific process is as follows: First, the cloud aggregates semantic_ids that cannot match known templates into unknown aggregation clusters. For high-frequency or important unknown aggregation clusters, corresponding SMS samples or templates are obtained through methods such as merchant template supplementation, manual sample review, and test sample re-feedback. Then, a semantic_id is generated for this template, and similarity matching is performed with the unknown aggregation cluster. If the matching condition is met, the template is used as the representative SMS template for that unknown aggregation cluster, and the template library and aggregation mapping relationship are updated.

[0043] For example, for high-frequency `semantic_id`s or unknown aggregation clusters that are not mapped to a known template library, the system marks them as objects to be completed. Subsequently, one or more representative SMS templates can be added to this unknown aggregation cluster through methods such as merchants supplementing templates, manual sample review, and test sample re-feeding. After completion, the template library and aggregation mapping relationship are updated, transforming the unknown aggregation cluster into a known aggregation cluster.

[0044] For example, if certain semantic_ids appear frequently over a long period but cannot be mapped to existing templates in the template library, an unknown aggregation cluster is created for this group of semantic_ids. Subsequently, when the cloud collects the corresponding SMS templates from merchants or other channels, the template semantic identifier can be regenerated and associated with this unknown aggregation cluster, thereby completing its representative SMS template field. Frequency statistics and parsing effect statistics before and after completion can be retained to avoid the loss of historical observation data.

[0045] Through a closed-loop mechanism of "known template mapping + similarity merging + unknown template discovery + subsequent database replenishment", the observation capabilities of the offline intelligent SMS system are continuously improved.

[0046] This forms a continuous evolutionary closed loop of "anonymous terminal reporting -> cloud-based aggregation and statistics -> discovery of unknown templates -> supplementation of the template library -> backfilling of mapping relationships".

[0047] As an optional implementation of this embodiment, preprocessing the original SMS text to obtain an irreversible SMS semantic identifier (semantic_id) that preserves semantic proximity includes: standardizing the SMS text; extracting character n-gram features from the standardized text; mapping the feature hash to a fixed-dimensional sparse vector; performing L2 normalization on the fixed-dimensional sparse vector and generating a fixed-length bit signature through symbolic random projection; and performing base64url encoding on the bit signature to obtain the semantic_id.

[0048] The above method, adopted in this optional implementation, aims to both protect the privacy of the original SMS message and identify SMS templates with similar semantics. Ordinary hashing can only determine if the text is completely identical, but it cannot aggregate SMS messages with different variables but the same semantics. In contrast, the semantic_id generated through standardization, n-gram, feature hashing, normalization, and random projection is both irreversible and preserves text similarity, making it suitable for similarity matching and aggregation in the cloud.

[0049] As an optional implementation in this embodiment, the aggregate statistics table with a specified structure includes a representative SMS template, an aggregate ID of the aggregate cluster, a set of included SMS semantic IDs, frequency, and parsing performance indicators. The representative SMS template describes the content of the SMS template represented by the aggregate cluster. The set of included SMS semantic IDs represents the multiple semantic_ids merged into the aggregate cluster. After receiving the SMS text, the terminal also calls a local offline intelligent SMS engine to parse the SMS text, generate a parsing result summary, and determine the parsing performance indicators based on the parsing result summary.

[0050] In this optional implementation, the data structure example is as follows: Representative SMS template | aggregation_id | semantic_id set | frequency | card hit rate | field completeness rate. For example, if the template library contains a representative template for "Your package has arrived at the station, please pick it up with the pickup code," it can correspond to an aggregation ID. Multiple SMS messages that differ only in the tracking number, pickup code, station name, and time fields, despite having different original text, can have their multiple semantic_ids reported by the client merged under this aggregation ID. The cloud can then determine the actual frequency of this type of package pickup SMS message and the current offline intelligent SMS engine's parsing performance for this type of SMS message.

[0051] For example, the cloud performs statistics around aggregation_id to generate an aggregation statistics table. For each aggregation ID, the statistics may include: representing the SMS template; the set of semantic_ids or a summary of the set; aggregation frequency; card generation rate; field extraction success rate; parsing failure rate; terminal version distribution; and time trend.

[0052] This embodiment is not limited to a specific combination of character n-gram, feature hashing, and random projection. Any equivalent technical means that can generate irreversible, comparable, and aggregable SMS semantic identifiers on the terminal side and implement template mapping, aggregation statistics, and unknown template completion in the cloud can fall within the protection scope of this invention.

[0053] Compared to existing technologies, this invention has at least the following advantages: It enables distributed observation of global SMS templates without uploading the original SMS text, balancing business analysis needs with user privacy protection requirements; it achieves automatic aggregation of similar SMS templates through SMS semantic identifiers with similarity preservation characteristics, outperforming traditional hashing schemes that can only identify completely identical text; it makes the aggregation results interpretable through cloud-based pre-collected template libraries and representative template mapping, clearly indicating "what" the high-frequency SMS templates "specifically are"; it allows for fine-grained evaluation of the card generation rate, field extraction quality, and coverage of the offline intelligent SMS parsing engine on the client side; it can automatically discover high-frequency unknown templates, providing a basis for SMS rule updates, model iterations, and merchant template supplementation; it only performs lightweight computation on the client side, suitable for large-scale terminal deployment; and it allows for continuous supplementation of the template library on the cloud side, forming a closed-loop evolving intelligent SMS observation system.

[0054] Compared to the solution of directly uploading the original text message, this embodiment is more in line with privacy protection requirements; compared to the solution of only uploading traditional hash values ​​such as MD5, it can aggregate similar text message templates; compared to the solution of simply counting the number of successful parsing, this invention can also bind the statistical results to specific template aggregation clusters and representative templates, thereby simultaneously obtaining frequency, parsing quality and unknown template discovery capabilities.

[0055] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0056] According to an embodiment of the present invention, a data processing system for SMS templates is also provided, characterized in that it includes: The receiving unit receives anonymous observation data sent by a terminal in the cloud. The terminal receives SMS text, preprocesses the SMS text to obtain a semantic identifier (semantic_id), and uploads the anonymous observation data to the cloud. The anonymous observation data includes the semantic_id, a parsing result summary, and / or non-sensitive statistical attributes. The matching and processing unit performs similarity matching between the SMS semantic identifier (semantic_id) and the semantic identifiers corresponding to various SMS templates in the SMS template library. The aggregation unit determines the aggregation cluster to which the anonymous observation data belongs based on the matching results and updates the specified structure of the aggregation statistics table based on the aggregation cluster ID, specified information of the aggregation cluster, parsing result summary, and / or non-sensitive statistical attribute information.

[0057] According to embodiments of the present invention, the present invention also provides an electronic device, the electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to implement the methods described in any of the above embodiments.

[0058] According to embodiments of the present invention, the present invention also provides a readable storage medium storing computer instructions that enable a computer to perform the methods described in any of the above embodiments when executed.

[0059] According to embodiments of the present invention, the present invention also provides a computer program product that, when executed by a processor, can implement the methods described in any of the above embodiments.

[0060] Figure 2 A schematic block diagram of an example electronic device 300 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices.

[0061] like Figure 2 As shown, the electronic device 300 includes a computing unit 301, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 302 or a computer program loaded from a storage unit 308 into a random access memory (RAM) 303. The RAM 303 may also store various programs and data required for the operation of the electronic device 300. The computing unit 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0062] Multiple components in electronic device 300 are connected to I / O interface 305, including: input unit 306, such as keyboard, mouse, etc.; output unit 307, such as various types of displays, speakers, etc.; storage unit 308, such as disk, optical disk, etc.; and communication unit 309, such as network card, modem, wireless transceiver, etc. Communication unit 309 allows electronic device 300 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0063] The computing unit 301 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 301 performs the various methods and processes described above, such as the object matching method. For example, in some embodiments, the object matching method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 308. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 300 via ROM 302 and / or communication unit 309. When the computer program is loaded into RAM 303 and executed by the computing unit 301, one or more steps of the methods described above may be performed.

[0064] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0065] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0066] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

Claims

1. A data processing method for SMS templates, characterized in that, include: The cloud receives anonymous observation data sent by the terminal, wherein the terminal receives SMS text, preprocesses the SMS text to obtain SMS semantic identifier semantic_id; and uploads the anonymous observation data to the cloud, wherein the anonymous observation data includes the semantic_id and a parsing result summary; The semantic identifier of the SMS message is matched with the semantic identifiers corresponding to each SMS template in the SMS template library based on similarity. Based on the matching results, determine the cluster to which the anonymous observation data belongs; update the aggregate statistics table with a specified structure based on the cluster ID, specified information of the cluster, and parsing result summary; Each SMS template is pre-assigned a semantic_id; the cloud merges semantically similar SMS templates to form several clusters, and assigns a unique cluster ID to each cluster; the semantic_id of each preset SMS template and the cluster ID are pre-generated as semantic identifiers.

2. The data processing method for SMS templates according to claim 1, characterized in that, Based on the matching results, the clusters to which the anonymous observation data belongs are determined to include: If a match is found between the semantic_id corresponding to the target SMS template or the aggregation ID of the target aggregation cluster, then the aggregation cluster to which the semantic_id corresponding to the target SMS template belongs is taken as the aggregation cluster to which the anonymous observation data belongs, or the target aggregation cluster is taken directly as the aggregation cluster to which the anonymous observation data belongs.

3. The data processing method for SMS templates according to claim 1, characterized in that, Based on the matching results, the clusters to which the anonymous observation data belongs are determined to include: If no matching SMS template's semantic_id or any aggregation cluster's aggregation ID can be found, the semantic_id in the data is added to the unknown template pool, and an unknown aggregation cluster is created for the content in the template pool.

4. The data processing method for SMS templates according to claim 1, characterized in that, Based on the matching results, the clusters to which the anonymous observation data belongs are determined to include: If no matching SMS template's corresponding semantic_id or any aggregation cluster's aggregation ID can be found, then based on the semantic similarity calculated during the matching process, the semantic_id in the data is added to the unknown template pool, and a new unknown aggregation cluster is created for the content in the template pool.

5. The data processing method for SMS templates according to claim 4, characterized in that, The semantic_id of the unknown template pool, or the unknown aggregate cluster that has already been aggregated in the template pool, is marked as an object to be completed; Supplement SMS templates for items that need to be completed.

6. The data processing method for SMS templates according to claim 1, characterized in that, Preprocessing the original SMS message yields an irreversible and semantically proximity-preserving semantic identifier, `semantic_id`, which includes: The SMS text is standardized; character n-gram features are extracted from the standardized text; the features are mapped to a fixed-dimensional sparse vector through feature hashing; the fixed-dimensional sparse vector is L2 normalized and a fixed-length bit signature is generated through symbolic random projection; the bit signature is base64url encoded to obtain the semantic_id.

7. The data processing method for SMS templates according to claim 1, characterized in that, The aggregate statistics table with a specified structure includes a representative SMS template, an aggregate ID of the aggregate cluster, a set of included SMS semantic IDs, frequency, and parsing performance metrics. The representative SMS template is used to describe the content of the SMS template represented by the aggregate cluster; the set of included SMS semantic IDs is used to represent the multiple semantic_ids merged into the aggregate cluster. After receiving the SMS text, the terminal also calls the local offline intelligent SMS engine to parse the SMS text, generate a parsing result summary, and determine the parsing effect index based on the parsing result summary.

8. A data processing system for SMS templates, characterized in that, include: The receiving unit receives anonymous observation data sent by the terminal in the cloud. The terminal receives SMS text, preprocesses the SMS text to obtain the SMS semantic identifier (semantic_id), and uploads the anonymous observation data to the cloud. The anonymous observation data includes the semantic_id, a parsing result summary, and / or non-sensitive statistical attributes. The matching and processing unit performs similarity matching between the SMS semantic identifier (semantic_id) and the semantic identifiers corresponding to each SMS template in the SMS template library. The aggregation unit determines the aggregation cluster to which the anonymous observation data belongs based on the matching results; and updates the aggregate statistics table of the specified structure based on the aggregation ID of the aggregation cluster, the specified information of the aggregation cluster, the parsing result summary, and / or non-sensitive statistical attribute information.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the method according to any one of claims 1-7.

10. An electronic device, characterized in that, include: At least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to cause the at least one processor to perform the method according to any one of claims 1-7.

Images