Short message aggregation grouping method and device, equipment and storage medium

Abstract

The invention discloses a short message aggregation grouping method and device, equipment and a storage medium, and the method comprises the steps: obtaining a preprocessed short message data set, calling a trained entity recognition model to carry out the tag recognition of short message data, and obtaining entity information; the method comprises the following steps of: intercepting entity information with a plurality of character lengths for comparison, classifying short message data with the same character into one group, and carrying out preliminary grouping processing on a short message data set to obtain a preliminary grouping result; and based on the preliminary grouping result, performing similarity judgment on each group by adopting a similarity algorithm so as to perform secondary grouping on the short message data set. According to the invention, the cost required by grouping can be reduced, and the short message aggregation grouping efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a method, device, equipment and storage medium for short message aggregation grouping. Background Art

[0002] With the rapid development of the Internet system, the number of users of instant messaging tools continues to rise, and the frequency of users receiving text messages is also increasing. Based on the above situation, text message cards come into being, which can help users understand the content of text messages more intuitively.

[0003] However, in the process of configuring SMS card regularization, there are a large number of repeated SMS messages that need to be configured separately, which not only consumes a lot of time and energy of configuration personnel, resulting in low configuration efficiency, but also increases labor costs. Configuration personnel need to screen different SMS contents one by one for regularization configuration, which is cumbersome and prone to omissions and errors. At the same time, manual screening is not only inefficient, but also easily affected by subjective factors, making it difficult to ensure the accuracy and consistency of the configuration. Summary of the invention

[0004] In view of this, the present application provides a method, device, equipment and storage medium for SMS aggregation grouping, which can reduce the cost required for grouping and improve the efficiency of SMS aggregation grouping.

[0005] The present application provides a method for aggregating and grouping short messages, the method comprising:

[0006] Obtain a preprocessed SMS data set, call the trained entity recognition model to perform label recognition on the SMS data, and obtain entity information;

[0007] intercepting the entity information of a certain length of characters for comparison, grouping the SMS data with the same characters into one group, performing preliminary grouping processing on the SMS data set, and obtaining a preliminary grouping result;

[0008] Based on the preliminary grouping result, a similarity algorithm is used to perform similarity judgment on each group, so as to perform secondary grouping on the SMS data set.

[0009] Optionally, the entity information includes names of people, places, names of organizations, numbers, dates and proper nouns.

[0010] Optionally, the step of obtaining the preprocessed SMS data and calling the trained entity recognition model to perform label recognition on the SMS data to obtain entity information further includes:

[0011] The numbers in the entity information are replaced with specific characters to achieve fuzzy processing of the entity information.

[0012] Optionally, the entity recognition model includes any one of the following models: a long short-term memory (LSTM)-CRF model; or a bidirectional long short-term memory (BiLSTM)-CRF model.

[0013] Optionally, the entity information of a certain length of characters is cut off for comparison, SMS data with the same characters are grouped together, and the SMS data set is preliminarily grouped to obtain a preliminary grouping result. The method includes:

[0014] The first 30 characters of the entity information are intercepted for comparison, the SMS data with the same first 30 characters are grouped together, and the SMS data in each group are sorted from high to low according to the quantity of the SMS data in each group, and each group is assigned a group number of 1 to n.

[0015] Optionally, based on the preliminary grouping result, a similarity algorithm is used to perform similarity judgment on each group to perform secondary grouping on the SMS data set, and the method further includes:

[0016] Based on the preliminary grouping results, similarity is judged, starting from group 1 and comparing with groups 2 to n one by one, and those with similarity greater than a preset threshold are classified into one category and removed from the data to be classified, and the remaining data to be classified are re-sorted;

[0017] Repeat the similarity judgment with the new groups 1 and 2 to n until all the SMS data are grouped.

[0018] Optionally, based on the preliminary grouping result, using a similarity algorithm to perform similarity judgment on each group to perform secondary grouping on the SMS data set includes:

[0019] A plurality of different similarity algorithms are combined to judge the similarity of each group. By setting different weights for the similarity algorithms, the comprehensive similarity between the groups is calculated so as to classify and group the SMS data more accurately. Among them, the similarity algorithms include the edit distance algorithm and the cosine similarity algorithm.

[0020] Correspondingly, the present application provides a device for aggregating and grouping short messages, including:

[0021] An entity recognition module is used to obtain a preprocessed SMS data set, call a trained entity recognition model to perform label recognition on the SMS data, and obtain entity information;

[0022] A preliminary grouping module, used to intercept the entity information of a certain length of characters for comparison, group the SMS data with the same characters into one group, perform preliminary grouping processing on the SMS data set, and obtain a preliminary grouping result;

[0023] The secondary grouping module is used to perform similarity judgment on each group using a similarity algorithm based on the preliminary grouping result, so as to perform secondary grouping on the SMS data set.

[0024] In addition, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the SMS aggregation grouping method as described in any one of the above items are implemented.

[0025] On this basis, the present application also provides a computer-readable storage medium on which a computer program is stored, wherein when the computer program is executed by a processor, the steps of the SMS aggregation grouping method as described in any of the above items are implemented.

[0026] The present application provides a method, device, equipment and storage medium for SMS aggregation and grouping. First, a pre-processed SMS data set is obtained, and an entity recognition model is called to identify the entity information. After the numbers are fuzzified, the first 30 characters of the entity information are intercepted for comparison and preliminary grouping. The group numbers are assigned according to the order of the number of SMS messages, and based on the preliminary grouping results, group 1 is compared with other groups one by one. Those with similarities greater than a preset threshold are classified into one category, and this process is repeated until all data grouping is completed. This method can effectively improve the efficiency of regular configuration of SMS cards, reduce labor costs, and eliminate the need for manual screening. Accurately grouping SMS messages helps to quickly understand the content of SMS messages, improve users' ability to manage and process SMS messages, and provide a more efficient data processing method for SMS-related services. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0028] Figure 1 A schematic diagram of the process of the SMS aggregation grouping method provided in the embodiment of the present application;

[0029] Figure 2 A schematic diagram of the structure of the SMS aggregation grouping device provided in the embodiment of the present application

[0030] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0032] It should be clear that the following embodiments of the present disclosure are described by specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other in the absence of conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present disclosure.

[0033] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present disclosure, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein may be used to implement this device and / or practice this method.

[0034] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. The drawings only show components related to the present disclosure rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0035] Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects described may be practiced without these specific details.

[0036] See also Figure 1 , Figure 1 A flowchart of a method for aggregating and grouping short messages provided according to an embodiment of the present specification is shown.

[0037] S1. Obtain a preprocessed SMS data set, call a trained entity recognition model to perform label recognition on the SMS data, and obtain entity information.

[0038] Optionally, obtain a preprocessed SMS data set, and the preprocessing may include operations such as noise removal and format standardization. For example, remove some special symbols, convert full-width characters to half-width characters, etc. Then, call the trained entity recognition model to perform label recognition on the SMS data. The entity recognition model can use a long short-term memory LSTM-CRF model; or a bidirectional long short-term memory BiLSTM-CRF model. The entity information includes names, place names, organization names, numbers, dates, and proper nouns. For example, when the SMS data content is "Your order number is 123456 and will be delivered by SF Express tomorrow", the model may recognize that "Order Number" is digital entity information, and "SF Express" is organization name entity information.

[0039] Optionally, obtaining preprocessed SMS data, calling a trained entity recognition model to perform label recognition on the SMS data, and obtaining entity information, the step further includes:

[0040] Replace the numbers in the entity information with specific characters to achieve blurring of the entity information.

[0041] Specifically, the numeric part of the entity information is determined and replaced with specific characters, which can be "NUM". This processing method can effectively protect sensitive information in SMS data, and can also reduce excessive segmentation caused by specific numeric differences. For example, for the SMS "Your order number is 12345, shipped.", after fuzzification processing, it becomes "Your order number is NUM, shipped." This allows the subsequent aggregation and grouping process to focus more on the overall semantic structure rather than the specific numeric content.

[0042] S2. intercepting the entity information of a certain length of characters for comparison, grouping the SMS data with the same characters into one group, performing preliminary grouping processing on the SMS data set, and obtaining a preliminary grouping result.

[0043] Optionally, entity information of a certain length of characters is intercepted for comparison, SMS data with the same characters are grouped together, and the SMS data set is preliminarily grouped to obtain a preliminary grouping result, the method comprising:

[0044] The first 30 characters of the entity information are intercepted for comparison, and the SMS data with the same first 30 characters are grouped together. The SMS data in each group are sorted from high to low according to the number of SMS data in each group, and each group is assigned a group number of 1 to n.

[0045] Specifically, the first several characters of each SMS entity information are intercepted from the SMS data after label recognition and fuzzification processing. For example, for the SMS "Your order has been shipped, the logistics order number is NUM." After intercepting the first several characters, it may be "Your order has been shipped, the logistics order number is". Then, the SMS data with the same first several characters are grouped together. This step can preliminarily group SMS with similar content at the beginning into the same group. Then, sort the SMS data in each group from high to low according to the number of SMS data, with the group with more number at the front, and assign group numbers from 1 to n to each group.

[0046] S3. Based on the preliminary grouping results, a similarity algorithm is used to judge the similarity of each group so as to perform secondary grouping on the SMS data set.

[0047] Optionally, based on the preliminary grouping result, a similarity algorithm is used to perform similarity judgment on each group to perform secondary grouping on the SMS data set. The method further includes:

[0048] Based on the preliminary grouping results, similarity judgment is performed, starting from group 1 and comparing it with groups 2 to n one by one. Those with similarity greater than the preset threshold are classified into one category and removed from the data to be classified, and the remaining data to be classified is re-sorted; repeat the similarity judgment with the new group 1 and groups 2 to n until all SMS data grouping is completed.

[0049] Specifically, first, based on the preliminary grouping results, group 1 is compared with groups 2 to n one by one to determine the similarity. If the similarity is greater than a preset threshold, they are grouped together and removed from the data to be classified, and the remaining data to be classified is re-sorted. Then the process of comparing the new group 1 with the subsequent groups is repeated until all SMS data are grouped. Among them, the preset threshold can be adjusted according to actual conditions, such as the type of SMS data, application scenario, etc. In this embodiment, the value of the preset threshold is set to 0.9.

[0050] Optionally, based on the preliminary grouping result, a similarity algorithm is used to judge the similarity of each group so as to perform secondary grouping on the SMS data set, including:

[0051] A plurality of different similarity algorithms are combined to judge the similarity of each group. By setting different weights for the similarity algorithms, the comprehensive similarity between the groups is calculated so as to classify and group the SMS data more accurately. Among them, the similarity algorithms include the edit distance algorithm and the cosine similarity algorithm.

[0052] Specifically, a variety of different similarity algorithms can also be combined, such as the edit distance algorithm and the cosine similarity algorithm. For the edit distance algorithm, the minimum number of editing operations required to convert the SMS content of one group into the SMS content of another group is calculated. For the cosine similarity algorithm, the SMS content is first vectorized, and then the cosine value between the vectors is calculated. By setting different weights for different similarity algorithms, the comprehensive similarity between the groups is comprehensively calculated.

[0053] like Figure 2 As shown, the present application also provides a short message aggregation grouping device, including:

[0054] The entity recognition module 101 is used to obtain a preprocessed SMS data set, call a trained entity recognition model to perform label recognition on the SMS data, and obtain entity information;

[0055] A preliminary grouping module 102 is used to intercept the entity information of a certain length of characters for comparison, group the SMS data with the same characters into one group, perform preliminary grouping processing on the SMS data set, and obtain a preliminary grouping result;

[0056] The secondary grouping module 103 is used to perform similarity judgment on each group using a similarity algorithm based on the preliminary grouping result, so as to perform secondary grouping on the SMS data set.

[0057] In addition, the present application also provides an electronic device, such as Figure 3 As shown, it shows a schematic diagram of the structure of the electronic device involved in the embodiment of the present application, specifically:

[0058] The electronic device may include components such as a processor 301 with one or more processing cores, a memory 302 with one or more computer-readable storage media, a power supply 303, and an input unit 304. Those skilled in the art will appreciate that Figure 3 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0059] The processor 301 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. By running or executing software programs and / or modules stored in the memory 302, and calling data stored in the memory 302, it executes various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole. Optionally, the processor 301 may include one or more processing cores; preferably, the processor 301 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 301.

[0060] The memory 302 can be used to store software programs and modules. The processor 301 executes various functional applications and image information communication methods by running the software programs and modules stored in the memory 302. The memory 302 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 302 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 302 may also include a memory controller to provide the processor 301 with access to the memory 302.

[0061] The electronic device also includes a power supply 303 for supplying power to each component. Preferably, the power supply 303 can be logically connected to the processor 301 through a power management system, so that the power management system can manage charging, discharging, power consumption and other functions. The power supply 303 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators and other arbitrary components.

[0062] The electronic device may further include an input unit 304, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0063] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 301 in the electronic device will load the executable files corresponding to the processes of one or more application programs into the memory 302 according to the following instructions, and the processor 301 will run the application programs stored in the memory 302, thereby realizing various functions, as follows:

[0064] The preprocessed SMS data set is obtained, and the trained entity recognition model is called to perform label recognition on the SMS data to obtain entity information; the entity information of a certain length of characters is intercepted for comparison, and SMS data with the same characters are grouped together, and the SMS data set is preliminarily grouped to obtain a preliminary grouping result; based on the preliminary grouping result, a similarity algorithm is used to judge the similarity of each group so as to perform secondary grouping on the SMS data set.

[0065] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0066] The present application provides an electronic device, which obtains a preprocessed SMS data set, calls a trained entity recognition model to perform label recognition on the SMS data, and obtains entity information; intercepts the entity information of a certain length of characters for comparison, groups SMS data with the same characters into one group, performs preliminary grouping processing on the SMS data set, and obtains preliminary grouping results; based on the preliminary grouping results, uses a similarity algorithm to perform similarity judgment on each group, so as to perform secondary grouping on the SMS data set. The present application can reduce the cost required for grouping and improve the efficiency of SMS aggregation grouping.

[0067] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0068] To this end, an embodiment of the present application provides a storage medium in which a plurality of instructions are stored, and the instructions can be loaded by a processor to execute the steps in any of the SMS aggregation grouping methods provided in the embodiments of the present application. For example, the instructions can execute the following steps:

[0069] The preprocessed SMS data set is obtained, and the trained entity recognition model is called to perform label recognition on the SMS data to obtain entity information; the entity information of a certain length of characters is intercepted for comparison, and SMS data with the same characters are grouped together, and the SMS data set is preliminarily grouped to obtain a preliminary grouping result; based on the preliminary grouping result, a similarity algorithm is used to judge the similarity of each group so as to perform secondary grouping on the SMS data set.

[0070] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0071] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0072] Since the instructions stored in the storage medium can execute the steps in any SMS aggregation and grouping method provided in the embodiments of the present application, the beneficial effects that can be achieved by any SMS aggregation and grouping method provided in the embodiments of the present application can be achieved. Please see the previous embodiments for details and will not be repeated here.

[0073] Although the present application has been shown and described with respect to one or more implementations, those skilled in the art will recognize equivalent variations and modifications based on reading and understanding this specification and the accompanying drawings. The present application includes all such modifications and variations and is limited only by the scope of the appended claims.

[0074] That is, the above description is only an embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the specification and drawings of the present application, such as the mutual combination of technical features between the embodiments, or direct or indirect application in other related technical fields, are also included in the patent protection scope of the present application.

[0075] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0076] In addition, for structural elements with the same or similar characteristics, the present application may use the same or different reference numerals for identification. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more features. In the description of the present application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0077] In the present application, in order to enable any person skilled in the art to implement and use the present application, the present application provides the above description. In the above description, various details are listed for the purpose of explanation. It should be understood that those of ordinary skill in the art can recognize that the present application can also be implemented without using these specific details. In other embodiments, well-known structures and processes will not be elaborated in detail to avoid unnecessary details that make the description of the present application obscure. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest range of principles and features disclosed in the present application.

Claims

1. A method for aggregating and grouping short messages, characterized in that: The method comprises: Obtain a preprocessed SMS data set, call the trained entity recognition model to perform label recognition on the SMS data, and obtain entity information; The entity information of a certain length of characters is intercepted for comparison, the SMS data with the same characters are grouped together, and the SMS data set is preliminarily grouped to obtain a preliminary grouping result.

2. Based on the preliminary grouping results, a similarity algorithm is used to perform similarity judgment on each group, so as to perform secondary grouping on the SMS data set. The SMS aggregation and grouping method according to claim 1, characterized in that: The entity information includes names of people, places, names of organizations, numbers, dates, and proper nouns.

3. The method for aggregating and grouping short messages according to claim 2, characterized in that: The step of obtaining the preprocessed SMS data and calling the trained entity recognition model to perform label recognition on the SMS data to obtain entity information further includes: The numbers in the entity information are replaced with specific characters to achieve fuzzy processing of the entity information.

4. The method for aggregating and grouping short messages according to claim 1, characterized in that: The entity recognition model includes any one of the following models: a long short-term memory LSTM-CRF model; or a bidirectional long short-term memory BiLSTM-CRF model.

5. The method for aggregating and grouping short messages according to claim 3, characterized in that: The entity information of the intercepted length of several characters is compared, the SMS data with the same characters are grouped together, and the SMS data set is preliminarily grouped to obtain a preliminary grouping result. The method includes: The first 30 characters of the entity information are intercepted for comparison, the SMS data with the same first 30 characters are grouped together, and the SMS data in each group are sorted from high to low according to the quantity of the SMS data in each group, and each group is assigned a group number of 1 to n.

6. The method for aggregating and grouping short messages according to claim 5, characterized in that: Based on the preliminary grouping result, a similarity algorithm is used to perform similarity judgment on each group to perform secondary grouping on the SMS data set. The method further includes: Based on the preliminary grouping results, similarity is judged, starting from group 1 and comparing with groups 2 to n one by one, and those with similarity greater than a preset threshold are classified into one category and removed from the data to be classified, and the remaining data to be classified are re-sorted; Repeat the similarity judgment with the new groups 1 and 2 to n until all the SMS data are grouped.

7. The method for aggregating and grouping short messages according to claim 6, characterized in that: Based on the preliminary grouping result, using a similarity algorithm to perform similarity judgment on each group to perform secondary grouping on the SMS data set includes: A plurality of different similarity algorithms are combined to judge the similarity of each group. By setting different weights for the similarity algorithms, the comprehensive similarity between the groups is calculated so as to classify and group the SMS data more accurately. Among them, the similarity algorithms include the edit distance algorithm and the cosine similarity algorithm.

8. A device for aggregating and grouping short messages, characterized in that: include: An entity recognition module is used to obtain a preprocessed SMS data set, call a trained entity recognition model to perform label recognition on the SMS data, and obtain entity information; A preliminary grouping module, used to intercept the entity information of a certain length of characters for comparison, group the SMS data with the same characters into one group, perform preliminary grouping processing on the SMS data set, and obtain a preliminary grouping result; The secondary grouping module is used to perform similarity judgment on each group using a similarity algorithm based on the preliminary grouping result, so as to perform secondary grouping on the SMS data set.

9. An electronic device, characterized in that: It includes a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the SMS aggregation grouping method as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, wherein when the computer program is executed by a processor, the steps of the SMS aggregation grouping method as described in any one of claims 1-7 are implemented.

Images