Text matching method and device, equipment and storage medium

Abstract

The present disclosure provides a text matching method, device, equipment, storage medium and program product, relates to the technical field of data processing, and particularly relates to the technical field of intelligent search and text processing. The specific implementation scheme is as follows: a target dictionary tree and a to-be-processed text are acquired, the target dictionary tree is obtained based on an initial dictionary tree adding text deformation information, the initial dictionary tree represents an initial text, the text deformation information represents the difference between the initial text and a deformation text, and the deformation text is obtained by deforming the initial text; text matching is performed on the to-be-processed text based on the target dictionary tree, and a matching result of the to-be-processed text is obtained, wherein the matching result of the to-be-processed text represents that the to-be-processed text comprises at least one of the initial text and the deformation text.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and more particularly to the field of intelligent search and text processing technology, specifically to a text matching method, apparatus, device, storage medium, and program product. Background Technology

[0002] With the rapid development of computer and network technologies, data processing can be applied to various tasks, such as maintaining network environments, and identifying sensitive words in text data through text matching. Summary of the Invention

[0003] This disclosure provides a text matching method, apparatus, device, storage medium, and program product.

[0004] According to one aspect of this disclosure, a text matching method is provided, comprising: obtaining a target trie and a text to be processed, wherein the target trie is obtained by adding text deformation information to an initial trie, the initial trie represents the initial text, the text deformation information represents the difference between the initial text and the deformed text, and the deformed text is obtained by deforming the initial text; performing text matching on the text to be processed based on the target trie to obtain a matching result of the text to be processed, wherein the matching result of the text to be processed represents that the text to be processed includes at least one of the initial text and the deformed text.

[0005] According to another aspect of this disclosure, a text matching apparatus is provided, comprising: an acquisition module for acquiring a target trie and a text to be processed, wherein the target trie is obtained by adding text deformation information to an initial trie, the initial trie representing the initial text, the text deformation information representing the difference between the initial text and the deformed text, and the deformed text being obtained by deforming the initial text; and a matching module for performing text matching on the text to be processed based on the target trie to obtain a matching result for the text to be processed, wherein the matching result for the text to be processed represents that the text to be processed includes at least one of the initial text and the deformed text.

[0006] According to another aspect of this disclosure, an electronic device is provided, comprising: at least one processor and a memory communicatively connected to the at least one processor. The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the methods of embodiments of this disclosure.

[0007] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing a computer to perform the methods of embodiments of this disclosure.

[0008] According to another aspect of this disclosure, a computer program product is provided, including a computer program stored on at least one of a readable storage medium and an electronic device, wherein the computer program, when executed by a processor, implements the methods of embodiments of this disclosure.

[0009] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0010] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0011] Figure 1 A schematic diagram illustrating the system architecture of a text matching method and apparatus according to embodiments of the present disclosure is shown.

[0012] Figure 2 A flowchart illustrating a text matching method according to an embodiment of the present disclosure is shown schematically;

[0013] Figure 3A A schematic diagram of an initial trie according to an embodiment of the present disclosure is shown;

[0014] Figure 3B Schematic illustration based on Figure 3A The diagram shows the target trie determined by the initial trie.

[0015] Figure 4 A schematic diagram illustrating a text matching method according to another embodiment of the present disclosure is shown.

[0016] Figure 5 A block diagram of a text matching apparatus according to an embodiment of the present disclosure is schematically shown; and

[0017] Figure 6 A block diagram of an electronic device that can implement the text matching method of embodiments of the present disclosure is shown schematically. Detailed Implementation

[0018] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0019] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0020] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0021] When using expressions such as "at least one of A, B, and C", they should generally be interpreted in accordance with the meaning that is commonly understood by a person skilled in the art (e.g., "a system having at least one of A, B, and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B, and C, etc.).

[0022] With technological advancements and improved living standards, the internet has become an integral part of people's lives, serving as a crucial channel for information access and communication. However, the internet is also rife with content unsuitable for browsing or dissemination, severely damaging user experience and disrupting the healthy internet ecosystem. While internet products typically employ sensitive word filters to ensure content security, malicious users often employ various variations of these filters, such as homophones and near-homophones, which ordinary filters cannot readily intercept. Therefore, identifying these modified sensitive words from massive datasets has become a pressing issue for internet products, a vital aspect of ensuring both user experience and product security.

[0023] Some implementations use sensitive word filtering word lists for sensitive word detection. However, these word filtering word lists are insufficient and cannot meet the requirements for detecting sensitive word variants.

[0024] Other implementations involve manually setting various variant types, enumerating all variant forms of a sensitive word, adding them to a filter word list, and then performing sensitive word detection through this filter word list. However, this method suffers from several drawbacks: sensitive word variants cannot be fully enumerated at once, repeated generation is resource-intensive, and maintenance costs are high; the number of candidate sensitive word variants is large, resulting in significant space consumption; and the enumeration results may inadvertently flag some normal expressions, leading to low accuracy.

[0025] Other implementations use neural network models for sensitive word detection, but neural network models are not robust enough and have high iterative update costs, making them unsuitable for frequently changing online language environments and malicious user attacks.

[0026] Figure 1 The schematic illustration shows the system architecture of a text matching method and apparatus according to an embodiment of the present disclosure. It should be noted that... Figure 1 The examples shown are merely examples of system architectures that can be applied to the embodiments of this disclosure, in order to help those skilled in the art understand the technical content of this disclosure, but do not mean that the embodiments of this disclosure cannot be used in other devices, systems, environments or scenarios.

[0027] like Figure 1 As shown, the system architecture 100 according to this embodiment may include clients 101, 102, and 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between clients 101, 102, and 103 and the server 105. The network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0028] Users can use clients 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on clients 101, 102, and 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0029] Clients 101, 102, and 103 can be various electronic devices with displays and web browsing capabilities, including but not limited to smartphones, tablets, laptops, and desktop computers. Clients 101, 102, and 103 in this embodiment of the disclosure can, for example, run applications.

[0030] Server 105 can be a server providing various services, such as a backend management server supporting websites browsed by users using clients 101, 102, and 103 (this is just an example). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the clients. Alternatively, server 105 can also be a cloud server, meaning server 105 has cloud computing capabilities.

[0031] It should be noted that the text matching method provided in this embodiment can be executed by server 105. Correspondingly, the text matching device provided in this embodiment can be located in server 105. The text matching method provided in this embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with clients 101, 102, 103 and / or server 105. Correspondingly, the text matching device provided in this embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with clients 101, 102, 103 and / or server 105.

[0032] In one example, server 105 can obtain the text to be processed from clients 101, 102, and 103 via network 104 and perform text matching. The target trie can be generated on server 105, or it can be generated on other terminals and obtained by server 105.

[0033] It should be understood that Figure 1 The number of clients, networks, and servers shown is merely illustrative. Depending on implementation needs, there can be any number of clients, networks, and servers.

[0034] It should be noted that the collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0035] In the technical solution disclosed herein, the user's authorization or consent is obtained before acquiring or collecting the user's personal information.

[0036] This disclosure provides a text matching method, which is described below in conjunction with... Figure 1 The system architecture, referencing Figures 2-6 This describes a text matching method according to exemplary embodiments of the present disclosure. The text matching method of the embodiments of the present disclosure may, for example, be derived from... Figure 1 The server 105 shown is used to execute this.

[0037] Figure 2 A flowchart illustrating a text matching method according to an embodiment of the present disclosure is shown.

[0038] like Figure 2 As shown, the text matching method 200 of this embodiment may include, for example, operations S210 to S220.

[0039] In operation S210, the target trie and the text to be processed are obtained.

[0040] The target trie is obtained by adding text variation information based on the initial trie, where the initial trie represents the initial text, the text variation information represents the difference between the initial text and the variant text, and the variant text is obtained by performing variation on the initial text.

[0041] It should be noted that the text matching method according to the embodiments of the present disclosure can be applied to scenarios such as matching and querying a pre-determined initial text and a variant text. The following description will be given by taking the initial text as a sensitive word as an example, and the text matching method according to the embodiments of the present disclosure is not limited to the application scenario of matching and querying sensitive words and sensitive word variants.

[0042] The initial text may include, for example, a sensitive word. The variant text may include, for example, a sensitive word that has undergone variation.

[0043] Exemplarily, taking the initial text as a Chinese word as an example, the variant text can be obtained by performing spelling variation, initial letter variation, radical variation and other manners on the initial text. For example, taking "sensitive word" as the initial text, the corresponding variant text may include "min gan word", "MG word", "敏赶 word", "敏g word", "敏感司", etc.

[0044] The trie, namely Trie Tree, is a tree structure, and each node of the trie except the root node represents one character. Each node of the trie except the root node represents only one character; the node sequence from the root node to a certain node in the trie can determine a corresponding character string; all characters represented by all child nodes of each node in the trie are different.

[0045] The text matching method according to the embodiments of the present disclosure involves two tries, which are respectively the initial trie and the target trie. The statement that "the initial trie represents the initial text" can be understood as that the initial trie represents an initial text set, and the initial text set includes at least one initial text. Specifically, it can be understood that the initial trie includes a plurality of nodes, any node in the initial trie represents one character of any initial text, a corresponding initial text can be determined through the corresponding node sequence of the initial trie, or the corresponding node sequence in the initial trie can be matched according to the initial text. The target trie obtained by adding text variation information based on the initial trie can represent the initial text set and a variant text set, and the variant text set includes at least one variant text.

[0046] In operation S220, text matching is performed on the to-be-processed text based on the target trie, to obtain a matching result of the to-be-processed text.

[0047] The matching result of the to-be-processed text indicates that the to-be-processed text includes at least one of the initial text and the variant text.

[0048] The text matching method according to the embodiments of the present disclosure, wherein the initial trie characterizes the initial text, the target trie is obtained by adding text deformation information based on the initial trie, the text deformation information characterizes the difference between the initial text and the deformed text, and the deformed text is obtained by deforming the initial text, so the target trie can characterize the initial text and the deformed text of the initial text. By performing text matching on the to-be-processed text based on the target trie, the matching of the initial text and the deformed text can be performed flexibly and efficiently, and an accurate matching result of the to-be-processed text can be obtained.

[0049] Specifically, compared with the manner of performing text matching by traversing the initial text vocabulary or traversing the deformed text vocabulary, performing text matching based on the target trie has a faster text matching speed. In addition, the target trie can be maintained, for example, by adding text deformation information, with low maintenance cost. Furthermore, compared with the manner of performing text matching by a neural network model, performing text matching based on the target trie has better robustness and interpretability, and the matching result of the to-be-processed text obtained by the text matching method according to the embodiments of the present disclosure is more accurate.

[0050] Exemplarily, for example, text preprocessing may also be performed on the to-be-processed text before operation S220. The text preprocessing may include, for example, removing null characters, removing line breaks, removing tabs, and the like.

[0051] In the text matching method according to another embodiment of the present disclosure, the target trie may be obtained, for example, by associating the character deformation nodes of the initial trie with character deformation information based on the initial trie, wherein the character deformation nodes are determined by comparing the initial text with the deformed text.

[0052] Figure 3A schematically shows a schematic diagram of an initial trie according to an embodiment of the present disclosure; Figure 3B schematically shows according to Figure 3A a schematic diagram of a target trie determined by the initial trie shown in the figure.

[0053] In Figure 3A the example of, a root node Node_0 is schematically shown, the child nodes of the root node Node_0 are Node_11 to Node_1x, the child nodes of Node_11 are Node_21 to Node_2y, and the child nodes of Node_2y are Node_31 to Node_3z. Taking the "sensitive word" as an example of the initial text, the character "Min" is characterized by Node_11, the character "Gan" is characterized by Node_2y, and the character "Ci" is characterized by Node_3z.

[0054] Taking the initial text as "sensitive word" where each character "敏", "感", "词" of the "sensitive word" has character deformation information as an example, the character deformation information var_1 of the character "敏" can be associated, for example, with the node Node_11 of "敏" in the initial trie, and the node Node_11 can serve as the character deformation node of the character "敏". Similarly, the character deformation information var_2 of the character "感" can be associated with the node of "感" in the initial trie, that is, the character deformation node Node_2y, and the character deformation information var_3 of the character "词" can be associated with the node of "词" in the initial trie, that is, the character deformation node Node_3z. The character deformation information var_1 of the character "敏" may include, for example: min, M, 每. The character deformation information var_2 of the character "感" may include, for example: gan, G, 咸. The character deformation information var_3 of the character "词" may include, for example: ci, C, 司.

[0055] In the text matching method according to the embodiments of the present disclosure, the target trie is obtained by associating the character deformation nodes of the initial trie with the character deformation information based on the initial trie, so that the character-unit fine-grained difference between the deformed text and the initial text can be associated with the target trie, enabling the target trie to represent the initial text as well as accurate and complete deformed text, which facilitates subsequent accurate text matching.

[0056] Exemplarily, for example, at least one of the character deformation nodes and the character deformation information of the target trie can be maintained. For example, the historical character deformation information var_2 of the character "感" may include: gan, G, 咸, and the maintenance of the character deformation information var_2 can be realized by adding "咸心" to the character deformation information var_2 at present. For example, relevant personnel determine that the deformation frequency of the character "词" in the initial text "sensitive word" is low through statistics and other methods, and for example, the node Node_3z corresponding to the character "词" may no longer be used as a character deformation node.

[0057] In the text matching method according to another embodiment of the present disclosure, for example, the following embodiment can be used to implement a specific example of performing text matching on the to-be-processed text based on the target trie to obtain a matching result of the to-be-processed text: sequentially matching a plurality of characters of the to-be-processed text with the target trie, and in response to determining that the first character among the plurality of characters matches the first node of the target trie, determining a candidate text fragment with the first character as the initial character; and sequentially matching each character of the candidate text fragment with the target trie, and in response to each character of the candidate text fragment matching the target trie, determining the candidate text fragment as the matching result of the to-be-processed text.

[0058] The candidate text fragment may be understood as a text fragment starting with the first character in the to-be-processed text, and the number of characters in the candidate text fragment is not fixed. For example, the text fragments with any number of characters in the to-be-processed text that start with the first character may be sequentially used as candidate text characters. For example, if the to-be-processed text has 10 characters and the first character is the 3rd character of the to-be-processed text, starting from the 3rd character, X characters starting from the 3rd character in the to-be-processed text may be used as the candidate text fragment, where X may take an integer value in the range of 1 to 8.

[0059] The text matching method according to the embodiments of the present disclosure may be used to match whether the to-be-processed text includes an original text or a deformed text. Description is made by taking an example where the to-be-processed text is "match sensitive word", "sensitive word" is an original text, and "sensitive w*rd" is used as a deformed text of the original text.

[0060] Taking the above example where neither "匹" nor "配" in "match sensitive word" appears in the target trie as an example, the first two characters of the to-be-processed text "match sensitive word" do not match the original text or the deformed text, and character-by-character text matching is continued sequentially. When the character "敏" is matched, "敏" may match the node Node_11 of the target trie, so Node_11 may be taken as the first character, and text matching of candidate text fragments is performed with this first character as the starting character. The candidate text fragments may include, for example, "敏gan" and "敏gan词". Taking the candidate text fragment "敏gan" as an example, both "敏" and "gan" match the target trie. For example, the matching process of "敏gan" may be recorded as a successful matching process, and the successful matching process is maintained using a queue. Subsequently, the candidate text fragment "敏gan词" is continued. Since "敏gan" in "敏gan词" has been successfully matched, "词" may be matched sequentially. In a case where "词" matches the target trie, it may be determined that "敏gan词" matches the target trie, and "敏gan词" matches a complete deformed text. The matching of the current candidate text fragment "敏gan词" ends, the matching process of "敏gan词" may be recorded as a complete matching process, and the candidate text fragment "敏gan词" is taken as the matching result of the to-be-processed text "match sensitive word". For example, a set st may be used to record the complete matching processes of the original text and the deformed text.

[0061] Exemplarily, for example, after the to-be-processed text completely matches a certain original text or a certain deformed text at any time, the to-be-processed text may next start re-matching the original text or the deformed text from the root node of the target trie.

[0062] It should be noted that the two characters "min" and "ci" can be respectively matched to the nodes Node_11 and Node_3z of the target trie, while the character "gan" can be matched to the node Node_2y of the target trie, specifically the character variation information of the matched node Node_2y.

[0063] According to the text matching method of the embodiments of the present disclosure, multiple characters of the to-be-processed text are sequentially matched with the target trie, so that the to-be-processed text can be matched character by character. In response to determining that the first character of the multiple characters matches the first node of the target trie, a candidate text segment starting with the first character is determined, so that the matched initial character can be used as the trigger character for initial text or variant text matching. The candidate text segment determined by the initial character can be subsequently accurately matched with the target trie sequentially in units of characters, and a complete initial text or variant text can be matched when every character of the candidate text segment is matched with the target trie.

[0064] According to the text matching method of another embodiment of the present disclosure, the candidate text segments include a first candidate text segment and a second candidate text segment. For example, the following embodiment can be used to implement the specific example of sequentially matching each character of the candidate text segment with the target trie, and determining the candidate text segment as the matching result of the to-be-processed text in response to each character of the candidate text segment being matched with the target trie: sequentially matching each character of the first candidate text segment with the target trie; in the case where the entire first candidate text segment is matched with the target trie, determining the matching result of the to-be-processed text according to the first candidate text segment and the second candidate text segment that satisfies the character number condition.

[0065] The character number condition represents that the number of characters of the second candidate text segment is less than or equal to the target number of characters, and the target number of characters is related to at least one of the following: the number of characters corresponding to the difference between the initial text and the variant text, and the number of characters inserted between any two characters.

[0066] Exemplarily, the target number of characters can be obtained, for example, by summing the number of characters corresponding to the difference between the initial text and the variant text and the number of characters inserted between any two characters.

[0067] The following situations exist: (1) For example, the character deformation information of the character "感" in the above embodiment includes "gan", the number of characters for storing "感" in some data structures may be 1, and the number of characters for storing "gan" in some data structures may be 3, which results in that the number of characters corresponding to the difference between the character "感" in the original text and the corresponding character "gan" in the deformed text is 2, and the number of characters corresponding to the difference between the original text "敏感词" and the deformed text "敏gan词" is also 2. (2) Irrelevant characters such as punctuation marks are inserted between any multiple characters of the sensitive word, which also makes the number of characters of the deformed text inconsistent with the number of characters of the original text. As a result, after the original text is deformed to obtain the deformed text, the number of characters of the deformed text is not consistent with the number of characters of the original text.

[0068] The text matching method according to the embodiments of the present disclosure can be adapted to the situation that a part of the text in the candidate text segment (the first candidate text segment) matches the target trie, and the other part of the candidate text segment does not match the target trie. By accurately matching the first candidate text segment in the candidate text segment and using the second candidate text segment that meets the character number condition, it can adapt to the situation that the deformed text has a different number of characters from the original text, and improve the matching accuracy of the deformed text.

[0069] Exemplarily, in the case that the first candidate text segment does not match the target trie and the second candidate text segment does not meet the character number condition, for example, the current text matching between the candidate text segment and the target trie may be abandoned. For example, the current candidate text segment does not match the target trie, and the second candidate text segment does not meet the character number condition. The number of characters of the candidate text segment obtained by adding characters on the basis of the current candidate text segment will increase, and the difference between the candidate text segment and the original text or the deformed text will be greater, so the text matching between the current candidate text segment and the target trie can be abandoned at this time.

[0070] Exemplarily, according to the text matching method of another embodiment of the present disclosure, for example, the following embodiment can be used to implement the specific example of sequentially matching each character of the first candidate text segment with the target trie: determining the candidate initial character of the first candidate text based on the target trie and the target character number; and matching each character of the first candidate text having the candidate initial character with the target trie.

[0071] Because the transformed text is derived from the initial text, there are cases where the number of characters in the transformed text does not match the number of characters in the initial text. This necessitates accurately locating which node in the target trie the initial character of the first candidate text segment matches in the target trie, before performing an accurate match on the first candidate text segment. For example, to match the first candidate text segment of the current candidate text segment, we can enumerate the nodes e′∈[e, e+K] in the target trie as candidate initial characters for the first candidate text segment, where e represents the node in the target trie corresponding to the last character of the previous candidate text segment, and K represents the number of target characters.

[0072] According to the text matching method of this disclosure, by determining the candidate initial character of the first candidate text based on the target trie and the target character number, it can adapt to the situation where the number of characters in the transformed text is not consistent with the number of characters in the initial text, accurately matching the initial character of the first candidate text segment to the corresponding node of the target trie, so that each character of the subsequent first candidate text segment can be accurately matched with the target trie.

[0073] Figure 4 A schematic diagram of a text matching method according to yet another embodiment of the present disclosure is shown.

[0074] like Figure 4 As shown, the text matching method according to another embodiment of this disclosure may further include, for example, operation S430.

[0075] In operation S430, the matching results of the text to be processed are filtered based on at least one of the text perplexity and text features to determine the target matching result 406.

[0076] For example, text features include at least one of the following: text length features, text language category features, text segmentation features, and text word frequency features.

[0077] The matching result of the text to be processed can be, for example, obtained by matching the text to be processed 401 based on the target trie.

[0078] For example, text length features can characterize the difference between the number of text characters in the original text and the number of text characters in the transformed text. Text language category features can characterize the difference between the number of Chinese characters in the original text and the number of Chinese characters in the transformed text. Text language category features can characterize the difference between the proportion of Chinese characters in the original text and the proportion of Chinese characters in the transformed text. Text word segmentation features can characterize the difference between the word segmentation length of the original text and the word segmentation length of the transformed text. Text word frequency features can characterize the difference between the word frequency features of the original text and the word frequency features of the transformed text. Text word frequency features can characterize the difference between the n-gram frequency features of the original text and the n-gram frequency features of the transformed text.

[0079] For example, text perplexity can be obtained by processing the text using natural speech processing models such as Bert and Ernie.

[0080] Matching the text to be processed against the target trie can yield matching results for either the initial text or variant text. However, the matching results may not be the expected initial or variant text, potentially leading to false positives. For example, some text to be processed may be in English and not contain Chinese characters. The matching results from the target trie might identify the correct English expression as variant text, when in fact it is not variant text.

[0081] The text matching method according to embodiments of this disclosure also takes into account that variant text generated by adversarial filtering vocabulary has anomalous text expression. By filtering the matching results of the text to be processed based on at least one of text perplexity and text features, the determined target matching results are more accurate.

[0082] like Figure 4 As shown, according to the text matching method of this disclosure, for example, an initial trie 403 can be maintained by maintaining initial text 405, and text deformation information 404 can also be maintained, thereby determining and maintaining the target trie 402.

[0083] Taking sensitive word matching as an example, the initial text can be maintained by maintaining the sensitive words set by relevant personnel. Text transformation information can be maintained through data collection, model training, and other means. Data collection can include collecting homophones, near-homophones, characters with different radicals, similar-looking characters, pinyin, pinyin initials, and emoji conversions from the initial text.

[0084] For example, when relevant personnel update the sensitive word list or text transformation information, they can do so by reading the updated sensitive word list or the updated text transformation information and constructing an updated target dictionary tree.

[0085] In summary, the text matching method according to the embodiments of this disclosure performs text matching on the text to be processed based on a target dictionary. Since the target dictionary is obtained by adding text transformation information to the initial dictionary, text matching on the text to be processed based on the target dictionary is faster and requires less storage space. Personnel only need to maintain the initial dictionary and the text transformation information, i.e., detect the transformed text after the initial text transformation. Furthermore, by filtering the matching results of the text to be processed, the target matching results can accurately filter out false positives. For example, when applied to sensitive word matching scenarios, the text matching method according to the embodiments of this disclosure can accurately match sensitive words or their transformed text from a massive amount of text to be processed. It has high matching accuracy, good robustness, flexible configuration, and low maintenance costs. It can assist in recalling sensitive content for review, optimize the content ecosystem of network products, and reduce network product security risks.

[0086] Figure 5 A block diagram of a text matching apparatus according to an embodiment of the present disclosure is shown schematically.

[0087] like Figure 5 As shown, the text matching device 500 of this embodiment includes, for example, an acquisition module 510 and a matching module 520.

[0088] The acquisition module 510 is used to acquire the target trie and the text to be processed. The target trie is obtained by adding text deformation information to the initial trie. The initial trie represents the initial text, and the text deformation information represents the difference between the initial text and the deformed text. The deformed text is obtained by deforming the initial text.

[0089] The matching module 520 is used to perform text matching on the text to be processed based on the target trie to obtain the matching result of the text to be processed, wherein the matching result of the text to be processed indicates that the text to be processed includes at least one of the initial text and the transformed text.

[0090] For example, the target trie is obtained by associating the character transformation nodes of the initial trie with character transformation information, based on the initial trie. The character transformation nodes are determined by comparing the initial text with the transformed text.

[0091] For example, the matching module includes: a candidate text fragment determination submodule, configured to sequentially match multiple characters of the text to be processed with a target trie, and in response to determining that the first character among the multiple characters matches the first node of the target trie, determine a candidate text fragment with the first character as the initial character; and a matching submodule, configured to sequentially match each character of the candidate text fragment with the target trie, and in response that each character of the candidate text fragment matches the target trie, determine the candidate text fragment as the matching result of the text to be processed.

[0092] For example, the matching submodule includes: a matching unit, configured to match each character of the first candidate text segment sequentially with the target dictionary; and a matching result determination unit, configured to determine the matching result of the text to be processed based on the first candidate text segments and the second candidate text segments that satisfy the character count condition, provided that all the first candidate text segments match the target dictionary. The character count condition indicates that the number of characters in the second candidate text segment is less than or equal to the target character count, and the target character count is related to at least one of the following: the number of characters corresponding to the difference between the initial text and the transformed text, and the number of characters inserted between any two characters.

[0093] For example, the matching unit includes: a candidate initial character determination subunit, used to determine the candidate initial character of the first candidate text based on the target trie and the number of target characters; and a matching subunit, used to match each character of the first candidate text with the candidate initial character with the target trie.

[0094] For example, the text matching apparatus according to an embodiment of the present disclosure further includes: a filtering module, configured to filter the matching results of the text to be processed based on at least one of text perplexity and text features, and determine a target matching result.

[0095] For example, text features include at least one of the following: text length features, text language category features, text segmentation features, and text word frequency features.

[0096] It should be understood that the embodiments of the apparatus portion of this disclosure correspond to the same or similar embodiments of the method portion of this disclosure, and the technical problems solved and the technical effects achieved are also the same or similar. This disclosure will not repeat them here.

[0097] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0098] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0099] like Figure 6 As shown, device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 602 or a computer program loaded from storage unit 608 into random access memory (RAM) 603. RAM 603 may also store various programs and data required for the operation of device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0100] Multiple components in device 600 are connected to I / O interface 605, including: input unit 606, such as keyboard, mouse, etc.; output unit 607, such as various types of monitors, speakers, etc.; storage unit 608, such as disk, optical disk, etc.; and communication unit 609, such as network card, modem, wireless transceiver, etc. Communication unit 609 allows device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0101] The computing unit 601 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as the text matching method. For example, in some embodiments, the text matching method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the text matching method described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the text matching method by any other suitable means (e.g., by means of firmware).

[0102] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0103] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0104] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0105] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0106] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0107] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

[0108] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0109] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A text matching method, comprising: Obtain a target trie and the text to be processed, wherein the target trie is obtained by adding text deformation information to an initial trie, the initial trie represents the initial text, the text deformation information represents the difference between the initial text and the deformed text, and the deformed text is obtained by deforming the initial text; and The process involves sequentially matching multiple characters of the text to be processed against the target trie; in response to determining that the first character among the multiple characters matches the first node of the target trie, a candidate text segment is determined using the first character as the initial character, the candidate text segment including a first candidate text segment and a second candidate text segment; and Each character of the first candidate text segment is sequentially matched with the target trie. If all the first candidate text segments match the target trie, the matching result of the text to be processed is determined based on the first candidate text segments and the second candidate text segments that satisfy the character count condition. The character count condition indicates that the number of characters in the second candidate text segment is less than or equal to the target character count. The target character count is related to at least one of the following: the number of characters corresponding to the difference between the initial text and the transformed text, and the number of characters inserted between any two characters. The matching result of the text to be processed indicates that the text to be processed includes at least one of the initial text and the transformed text.

2. The method according to claim 1, wherein, The target trie is obtained by associating the character deformation nodes of the initial trie with character deformation information, based on the initial trie. The character deformation nodes are determined by comparing the initial text with the deformed text.

3. The method according to claim 1, wherein, The step of matching each character of the first candidate text segment with the target trie in sequence includes: Based on the target trie and the target character count, determine the candidate initial characters of the first candidate text; and Each character of the first candidate text having the candidate initial character is matched against the target trie.

4. The method according to any one of claims 1-3, further comprising: The matching results of the text to be processed are filtered based on at least one of the text perplexity and text features to determine the target matching result.

5. The method according to claim 4, wherein, The text features include at least one of the following: text length features, text language category features, text word segmentation features, and text word frequency features.

6. A text matching device, comprising: The acquisition module is used to acquire a target trie and the text to be processed, wherein the target trie is obtained by adding text deformation information to an initial trie, the initial trie represents the initial text, the text deformation information represents the difference between the initial text and the deformed text, and the deformed text is obtained by deforming the initial text; and A matching module is configured to sequentially match multiple characters of the text to be processed against the target trie; in response to determining that a first character among the multiple characters matches a first node of the target trie, a candidate text segment is determined with the first character as the initial character, the candidate text segment including a first candidate text segment and a second candidate text segment; and each character of the first candidate text segment is sequentially matched against the target trie; if all the first candidate text segments match the target trie, a matching result of the text to be processed is determined based on the first candidate text segments and the second candidate text segments that satisfy the character count condition, wherein the character count condition indicates that the number of characters in the second candidate text segment is less than or equal to the target character count, the target character count being related to at least one of the following: the number of characters corresponding to the difference between the initial text and the transformed text, the number of characters inserted between any two characters, wherein the matching result of the text to be processed indicates that the text to be processed includes at least one of the initial text and the transformed text.

7. The apparatus according to claim 6, wherein, The target trie is obtained by associating the character deformation nodes of the initial trie with character deformation information, based on the initial trie. The character deformation nodes are determined by comparing the initial text with the deformed text.

8. The apparatus according to claim 7, wherein, The matching module includes: A candidate initial character determination subunit is configured to determine the candidate initial character of the first candidate text based on the target trie and the number of target characters; and A matching subunit is used to match each character of the first candidate text having the candidate initial character with the target trie.

9. The apparatus according to any one of claims 6-8, further comprising: The filtering module is used to filter the matching results of the text to be processed based on at least one of text perplexity and text features, and to determine the target matching result.

10. The apparatus according to claim 9, wherein, The text features include at least one of the following: text length features, text language category features, text word segmentation features, and text word frequency features.

11. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-5.

13. A computer program product comprising a computer program stored on at least one of a readable storage medium and an electronic device, the computer program implementing the method according to any one of claims 1-5 when executed by a processor.

Images