Method, device and equipment for target text to hit matched words, medium and program
Through the LRU cache and tree index model, the problem of inefficiency of target text matching words in real-time communication technology is solved, and the accurate hit and efficiency of combined relationship matching words is improved.
Patent Information
- Application Number
- CN202411121244.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-15
- Publication Date
- 2024-11-26
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
In existing instant communication technology, the method of target text matching words is inefficient, especially when dealing with a large number of matching words and inclusion combination relationships, it is difficult to accurately hit the matching words, resulting in high time cost and poor matching effect.
The LRU cache mechanism and tree index model are used to segment the matching word list into a string array by traversing the matching word list, and the mapping relationship between hash value and matching word is established in the tree index model to determine whether the target text hits the matching word. If a missed, the slice generates a character list to match. If a hit is hit, the LRU cache will be updated to improve efficiency.
It improves the efficiency of the target text matching words, can accurately hit matching words containing combination relationships, reduces duplicate processing, and significantly improves matching efficiency and accuracy.
Smart Images

Figure CN119025651A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of instant messaging technology, and in particular to a method, device, equipment, medium and program for matching words in a target text. Background Art
[0002] The field of instant messaging (IM) has the responsibility and obligation to ensure information security compliance. While satisfying the user experience, it is necessary to ensure information security compliance through technical means, which involves the target text hitting the matching words. In the related technology, regular expressions or strings.Contains (target text, matching words) and other methods are generally used to determine whether the target text hits the matching words. The above two methods have the following problems: When there are a large number of matching words, the above two methods result in the target text needing to be judged for each matching word, which is time-consuming; when the matching words contain some combination relationships, such as the target text "Hangzhou's weather is really good" needs to hit the matching words "Hangzhou + weather", the above two methods cannot achieve the above effects; when the matching words contain some combination relationships, but the actual semantic differences lead to poor matching effects.
[0003] In view of this, a method for matching words in a target text that can match words containing a combination relationship and improve the matching efficiency is needed. Summary of the invention
[0004] In view of this, the present invention provides a method, device, equipment, medium and program for matching words in a target text to solve the problems of low text matching efficiency and difficulty in processing complex matching rules.
[0005] In the first aspect, the present invention provides a method for a target text to hit a matching word, the method comprising: traversing a preset matching word list, dividing the matching words in the matching word list into a string array; traversing the string array, adding the sub-items of the string array to a tree index model, and establishing a mapping relationship between the hash value corresponding to the sub-item and the matching word in the tree index model; obtaining a target text and determining whether the target text hits an LRU cache; the LRU cache stores historical hit results; if the target text hits the LRU cache, returning the hit result corresponding to the target text from the LRU cache; if the target text does not hit the LRU cache, slicing the target text to generate a character list; traversing the character list and matching it with a tree index model; the tree index model is used to indicate the mapping relationship between a string and a matching word; if a tree leaf node in the tree index model is hit, determining that the target text hits a matching word; if the target text hits the matching word, obtaining the hit result corresponding to the target text and adding it to the LRU cache.
[0006] In a second aspect, the present invention provides a device for a target text to hit a matching word, the device comprising: a text acquisition module, used to acquire the target text; a hit judgment module, used to judge whether the target text hits the LRU cache; the LRU cache stores historical hit results; a result return module, used to return the hit result corresponding to the target text from the LRU cache if the target text hits the LRU cache; a matching word judgment module, used to judge whether the target text hits the matching word if the target text does not hit the LRU cache; a cache adding module, used to acquire the hit result corresponding to the target text and add it to the LRU cache if the target text hits the matching word.
[0007] In a third aspect, the present invention provides a computer device comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the target text matching word method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.
[0008] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the target text matching word method of the first aspect or any corresponding embodiment thereof.
[0009] In a fifth aspect, the present invention provides a computer program product, including computer instructions, which are used to enable a computer to execute the target text matching word method of the first aspect or any corresponding embodiment thereof.
[0010] The technical solution provided by one or more embodiments of the present invention is as follows: first, the system receives and obtains the target text to be processed, and checks whether the target text already exists in the LRU cache, which stores the previous matching results. If the target text has a record in the LRU cache, the system will directly retrieve and provide the previous historical hit results of the text from the cache. If the target text has no record in the LRU cache, that is, it does not hit the cache, the system will determine whether the target text matches any matching words. Once it is detected that the target text contains matching words, the system will obtain the corresponding hit results, update the LRU cache, and store the results of this match. The present invention supports the implementation method of the target text hitting the matching words containing the combination relationship, thereby improving the efficiency of hitting the matching words. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0012] Figure 1 is a schematic diagram of a flow chart of a method for finding matching words in a target text according to an embodiment of the present invention;
[0013] Figure 2 is a flow chart of a method for creating a tree index model corresponding to a matching word according to an embodiment of the present invention;
[0014] Figure 3 is a structural block diagram of a target text matching word device according to an embodiment of the present invention;
[0015] Figure 4 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0016] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.
[0017] An embodiment of the present invention provides a method for matching words in a target text. The method utilizes an LRU cache mechanism to achieve the effect of being able to hit matching words containing combination relationships and improving the hit efficiency.
[0018] According to an embodiment of the present invention, an embodiment of a method for hitting matching words in a target text is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0019] In this embodiment, a method for matching words in a target text is provided. Figure 1 is a flow chart of a method for finding a matching word for a target text according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:
[0020] Step S101, traverse the matching word list, and divide the matching words in the matching word list into a string array.
[0021] First, the predefined matching word list is traversed. For each matching word in the list, the system splits it according to a specific separator to form a string array. This step is to convert the matching word into a form that can be used in the tree index model to prepare for subsequent matching operations.
[0022] Step S102, traverse the string array, add the sub-items of the string array to the tree index model, and establish a mapping relationship between the hash value corresponding to the sub-item and the matching word in the tree index model.
[0023] Traverse the string array generated in step S101. In this process, the system adds each sub-item in the string array to the tree index model.
[0024] In addition, by creating a hash value for each sub-item in the tree index model and creating a mapping relationship between these hash values and the original matching words, it is possible to efficiently check whether matching words exist in the target text and their specific locations.
[0025] like Figure 2 As shown in the figure, it is a tree index model construction process. It first involves traversing the list of matching words and splitting each matching word into sub-matching words according to the separator. Subsequently, these sub-matching words are further sliced to generate a list of slices. In the construction of the tree structure, the root node represents the first target node. For each slice, the process checks whether it already exists as a child node in the tree. If the slice does not exist as a child node, a new node is created and added to the tree structure. This newly created node then becomes the new target node for the next slice processing. The whole process continues until all the slices of the sub-matching words have been processed, and finally a complete tree index model is formed.
[0026] Step S103, obtaining the target text and determining whether the target text hits the LRU cache; the LRU cache stores historical hit results.
[0027] In the field of instant messaging technology, in order to implement text content review and sensitive word filtering, it involves the target text hitting the matching words. First, the target text to be processed is received. The target text refers to the text to be matched, which can be a message entered by the user, an article, or any other form of text content, such as "The weather in Hangzhou is very good today."
[0028] After obtaining the target text, it is necessary to determine whether the obtained target text already exists in the LRU cache. The LRU cache is an efficient data storage structure that saves the most recently processed target text and its matching results for quick access. If the target text is found in the cache, it is called a "hit." This means that there is a history of the text in the cache and it can be used directly. When a hit occurs, according to the LRU strategy, the usage record of the target text will be updated to reflect its most recently accessed status. The LRU cache not only stores the data itself, but also the historical hit results, which are usually sorted by time. In this way, when the same target text is queried again, it can be quickly determined whether it is in the cache.
[0029] Step S104: if the target text hits the LRU cache, the hit result corresponding to the target text is returned from the LRU cache.
[0030] If the target text hits in the LRU cache, that is, the text has been processed before, the system will directly retrieve and return the historical hit result corresponding to the text from the cache. This step significantly improves the matching efficiency because it avoids repeated processing of the same text.
[0031] Step S105: if the target text does not hit the LRU cache, slice the target text to generate a character list.
[0032] When the target text does not hit in the LRU cache, the system will continue to perform a match check. In this step, the system will analyze the target text to determine whether it contains or hits the pre-defined matching words. Matching words are used to determine whether the target text is a hit, such as "Hangzhou", "Hangzhou + weather"; hit means that the target text is judged to contain matching words according to certain rules, such as "Hangzhou weather" hits "Hangzhou". The target text is split into individual characters and a character list is created. This process is called slicing. Slicing refers to generating a character array from a string according to the character dimension, such as "hello" becomes "you" and "good" after slicing. Slicing is a common operation in text processing, which allows the system to analyze and match text character by character.
[0033] Step S106, traverse the character list and match it with the tree index model; the tree index model is used to indicate the mapping relationship between the character string and the matching word.
[0034] Traverse the character list generated in step S105 and match each character or character sequence with a tree index model. The tree index model is an efficient data structure for storing matching words and quickly retrieving their existence. The model constructs a path from the root node to the leaf node by treating each character as a node in the tree, and each leaf node represents the end of a matching word.
[0035] Step S107: If a tree leaf node in the tree index model is hit, it is determined that the target text hits a matching word.
[0036] If during the traversal process, the system finds a path in the tree index model that matches the target text character sequence and eventually reaches a leaf node, this indicates that the target text hits a matching word. Hitting a leaf node means that the entire matching word has been completely recognized.
[0037] Step S108: If the target text hits the matching word, the hit result corresponding to the target text is obtained and added to the LRU cache.
[0038] If the target text hits the matching word, the system will obtain the corresponding hit result. The hit result may include the specific information and location of the matching word. Then, the system adds these new hit results to the LRU cache to prepare for possible future queries, and also updates the data in the cache.
[0039] The target text hit matching word method provided in this embodiment, by utilizing the LRU cache mechanism, can not only quickly respond to queries on processed texts, but also effectively match new texts and optimize the performance of future operations by caching new results.
[0040] In some optional implementations, in the above step S108, the target text hits the matching word, and obtaining the hit result corresponding to the target text includes the following steps:
[0041] Step S1081: If the target text matches the matching word, determine whether the matching word meets the matching rule.
[0042] The system first determines whether the matching words in the target text meet specific matching rules. These rules may include the combination relationship of matching words, semantic integrity, etc. For example, if the matching word is composed of multiple sub-words connected by separators (such as "matching word 1<separator>matching word 2"), it is necessary to verify whether these sub-words appear in the target text in the established order and structure. The separator is the separator symbol for multiple matching conditions in the matching word, generally "+", space, etc., such as "+" in "Hangzhou+weather".
[0043] Step S1082: If the matching words meet the hit rule, the intervals between the matching words are calculated, and the hit results corresponding to the target text are screened.
[0044] If the matching words meet the hit rules, the system will calculate the intervals between these matching words. The calculation of the intervals may involve the number of characters and their positions. Based on the calculated interval information, the system will filter and determine the final hit results. This step will remove matching words whose intervals exceed the preset threshold to ensure that only matching words that meet specific conditions are recognized as valid hits. The hit results obtained by filtering are added to the LRU cache for subsequent query and processing. This update operation not only improves future matching efficiency, but also enables the system to record and learn text matching patterns, thereby continuously optimizing the matching algorithm.
[0045] The method provided in this embodiment ensures that the target text matching process is efficient and accurate, and meets the strict requirements of the instant messaging field for text content review and sensitive word filtering.
[0046] In some optional implementations, determining whether the matching word meets the hit rule in the above step S1081 includes the following steps:
[0047] Step a11, obtaining the hash value corresponding to each matching word in the hit result, and obtaining a matching word hash value list.
[0048] Process the identified hit results and obtain the corresponding hash value for each matching word. A hash value is a fixed-size value generated by a hash function, which can represent the content of the matching word. Through this step, a matching word hash value list can be obtained to prepare for subsequent combination and verification.
[0049] Step a12, combining the hash values of the matching words in the hit results to generate a matching word hash value combination list.
[0050] Combine the hash values of each matching word in the hit result. This may involve combining multiple related hash values in a specific order or pattern to form a hash value combination list. This list is used to represent the possible combination relationship between matching words to meet specific hit rules. If the target text hits a and b, then generate the corresponding hash values of a, b, and a+b.
[0051] Step a13: for each combination in the matching word hash value combination list, check whether its hash value exists in the matching word hash list.
[0052] Traverse each combination in the matching word hash value combination list and check whether the hash values of these combinations exist in the pre-defined matching word hash list. This hash list contains the hash values of all valid matching words. If a hash value in the combination list exists in the hash list, it means that the current matching word combination meets the hit rule and the match result is confirmed to be valid. This step filters incomplete hits, such as "Hangzhou is just right" will not hit "Hangzhou + weather".
[0053] The method provided in this embodiment can accurately determine whether the matching words meet specific hit rules, thereby improving the accuracy and reliability of matching. It is suitable for processing matching words containing complex combination relationships, ensuring that only matching words that meet specific semantics or structures are identified and processed.
[0054] In some optional implementations, calculating the intervals between the hit matching words in the above step S1082 and screening to obtain the hit results corresponding to the target text includes the following steps:
[0055] Step b11, calculating the position of the matching word.
[0056] First, determine the specific starting and ending positions of each hit matching word in the target text, scan the text, and record the position information.
[0057] Step b12, sorting from large to small based on the index position of the matching words.
[0058] According to the matching word positions calculated in step b11, the matching words are sorted from large to small according to the index values. This step is for the convenience of subsequent processing and ensures that the intervals between matching words can be calculated in order.
[0059] Step b13, calculating the matching word interval according to the index position, if the matching word interval exceeds the preset interval, it is judged as a miss.
[0060] Calculate the interval between the sorted adjacent matching words. First, determine the distance between the end position and the start position of two adjacent matching words. If this interval exceeds the maximum interval value preset by the system, it is considered that the two matching words do not meet the hit condition, and the match is judged as a miss.
[0061] Step b14: if the interval between matching words does not exceed the preset interval, a matching result of the target text is generated according to the matching words.
[0062] If the matching word interval calculation in step b13 shows that the intervals between all adjacent matching words do not exceed the preset interval, the system will generate a final matching result of the target text based on these matching words.
[0063] The method provided in this embodiment can calculate and filter out matching words that meet specific spacing conditions, thereby ensuring the accuracy and relevance of the matching results and improving the quality and efficiency of complex text matching scenarios.
[0064] In some optional implementations, before determining whether the matching word meets the hit rule in the above step S1081, the following steps are further included:
[0065] Step c11: if the target text hits a matching word that is an English character, determine whether the target text is an English character.
[0066] First, it is detected whether the matching words in the target text are composed of English characters to determine whether corresponding processing rules need to be applied to the matching words, such as adding spaces or using regular expressions for matching.
[0067] Step c12: If the target text is in English characters, add spaces on both sides of the matching word to obtain a regular matching expression.
[0068] If the matching words in the target text are confirmed as English characters, the system will add spaces on both sides of these matching words. The purpose is to recognize these words as independent words in regular expression matching and avoid being recognized as part of other words.
[0069] Step c13, matching the target text based on the regular matching expression, and filtering out matching words that do not match the target text.
[0070] Based on the matching words with spaces generated in step c12, a regular matching expression is constructed. The target text is matched again using this expression. According to the results of the regular expression matching, those matching words that do not match the target text are screened out. Only those matching words that appear as independent words in the target text and meet all the established rules will be considered as valid hit results.
[0071] The method provided in this embodiment can accurately process matching words in English text, reduce mismatches, and improve the accuracy and reliability of matching results.
[0072] In the present embodiment, a target text hit matching word device is also provided, and the device is used to implement the above-mentioned embodiment and preferred implementation mode, and the description has been made no further. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the device described in the following embodiments is preferably implemented with software, the implementation of hardware, or a combination of software and hardware is also possible and conceived.
[0073] This embodiment provides a target text matching word device, such as Figure 3 As shown, including:
[0074] An array segmentation module 301 is used to traverse a preset matching word list and segment the matching words in the matching word list into a string array;
[0075] An array traversal module 302 is used to traverse the string array, add sub-items of the string array to a tree index model, and establish a mapping relationship between a hash value corresponding to the sub-item and a matching word in the tree index model;
[0076] A hit determination module 303 is used to obtain a target text and determine whether the target text hits an LRU cache; the LRU cache stores historical hit results;
[0077] A result returning module 304 is used to return the hit result corresponding to the target text from the LRU cache if the target text hits the LRU cache;
[0078] A text slicing module 305 is used for slicing the target text to generate a character list if the target text does not hit the LRU cache;
[0079] A model matching module 306 is used to traverse the character list and match it with a tree index model; the tree index model is used to indicate the mapping relationship between the character string and the matching word;
[0080] A matching judgment module 307 is used to determine that the target text hits a matching word if a tree leaf node in the tree index model is hit;
[0081] The cache adding module 308 is used to obtain the hit result corresponding to the target text if the target text hits the matching word, and add it to the LRU cache.
[0082] In some optional implementations, the cache adding module 308 includes:
[0083] A rule judgment unit, used to judge whether the matching word meets the matching rule if the target text hits the matching word;
[0084] The interval calculation unit is used to calculate the interval between the matching words if the matching words meet the matching rules, and screen to obtain the matching results corresponding to the target text.
[0085] In some optional implementations, the rule determination unit includes:
[0086] The first rule judgment subunit is used to obtain the hash value corresponding to each matching word in the hit result, and obtain a matching word hash value list;
[0087] The second rule judgment subunit is used to combine the hash values of the matching words in the hit result to generate a matching word hash value combination list;
[0088] The third rule judgment subunit is used to check, for each combination in the matching word hash value combination list, whether its hash value exists in the matching word hash list.
[0089] In some optional implementations, the interval calculation unit includes:
[0090] A first interval calculation subunit is used to calculate the position of the matching word;
[0091] A second interval calculation subunit is used for sorting the matching words from large to small based on their index positions;
[0092] A third interval calculation subunit, used to calculate the matching word interval according to the index position, and if the matching word interval exceeds a preset interval, it is judged as a miss;
[0093] The fourth interval calculation subunit is used to generate a hit result of the target text according to the hit matching word if the matching word interval does not exceed a preset interval.
[0094] In some optional embodiments, the method further comprises:
[0095] The first English screening unit is used to determine whether the target text is an English character if the matching word hit in the target text is an English character;
[0096] The second English screening unit is used to add spaces on both sides of the matching word to obtain a regular matching expression if the target text is English characters;
[0097] The third English screening unit is used to match the target text based on the regular matching expression and screen out the matching words that do not match the target text.
[0098] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.
[0099] The target text hit matching word device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.
[0100] The embodiment of the present invention also provides a computer device having the above Figure 3 The target text shown hits the matching word device.
[0101] See also Figure 4 , Figure 4 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 4As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 4 A processor 10 is taken as an example.
[0102] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.
[0103] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.
[0104] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0105] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.
[0106] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 4 The example of connecting through bus is taken in the following.
[0107] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.
[0108] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.
[0109] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.
[0110] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.
Claims
1. A method for matching words in a target text, characterized in that: The method comprises: Traverse the preset matching word list and split the matching words in the matching word list into string arrays; Traversing the string array, adding sub-items of the string array to a tree index model, and establishing a mapping relationship between hash values corresponding to the sub-items and matching words in the tree index model; Obtaining a target text and determining whether the target text hits an LRU cache; the LRU cache stores historical hit results; If the target text hits the LRU cache, returning the hit result corresponding to the target text from the LRU cache; If the target text does not hit the LRU cache, slicing the target text to generate a character list; Traversing the character list and matching it with a tree index model; the tree index model is used to indicate a mapping relationship between a character string and a matching word; If a tree leaf node in the tree index model is hit, it is determined that the target text hits a matching word; If the target text hits the matching word, the hit result corresponding to the target text is obtained and added to the LRU cache.
2. The method according to claim 1, characterized in that If the target text hits the matching word, obtaining the hit result corresponding to the target text includes: If the target text matches the matching word, determining whether the matching word meets the matching rule; If the matching words meet the hit rule, the intervals between the hit matching words are calculated, and the hit results corresponding to the target text are obtained by screening.
3. The method according to claim 2, characterized in that The determining whether the matching word meets the hit rule includes: Obtaining the hash value corresponding to each matching word in the hit result to obtain a matching word hash value list; Combine the hash values of the matching words in the hit results to generate a matching word hash value combination list; For each combination in the matching word hash value combination list, check whether its hash value exists in the matching word hash list.
4. The method according to claim 2, characterized in that: The calculating the intervals between the hit matching words and screening to obtain the hit results corresponding to the target text includes: Calculate the position of the matching word; Sorting from largest to smallest based on the index position of the matching words; Calculating the matching word interval according to the index position, and determining that it is a miss if the matching word interval exceeds a preset interval; If the matching word interval does not exceed the preset interval, a hit result of the target text is generated according to the hit matching word.
5. The method according to claim 2, characterized in that: Before determining whether the matching word meets the hit rule, the method further includes: If the target text hits the matching word which is an English character, determine whether the target text is an English character; If the target text is in English characters, add spaces on both sides of the matching word to obtain a regular matching expression; Based on the regular matching expression, the target text is matched, and the matching words that do not match the target text are screened out.
6. A target text matching word device, characterized in that: The device comprises: An array segmentation module is used to traverse a preset matching word list and segment the matching words in the matching word list into a string array; An array traversal module, used to traverse the string array, add sub-items of the string array to a tree index model, and establish a mapping relationship between a hash value corresponding to the sub-item and a matching word in the tree index model; A hit judgment module, used to obtain a target text and judge whether the target text hits an LRU cache; the LRU cache stores historical hit results; A result returning module, used for returning a hit result corresponding to the target text from the LRU cache if the target text hits the LRU cache; A text slicing module, used for slicing the target text to generate a character list if the target text does not hit the LRU cache; A model matching module, used to traverse the character list and match it with a tree index model; the tree index model is used to indicate a mapping relationship between a character string and a matching word; A matching judgment module, used for determining that the target text hits a matching word if a tree leaf node in the tree index model is hit; The cache adding module is used to obtain the hit result corresponding to the target text if the target text hits the matching word, and add it to the LRU cache.
7. The device according to claim 6, characterized in that The cache adding module is also used for: If the target text matches the matching word, determining whether the matching word meets the matching rule; If the matching words meet the hit rule, the intervals between the hit matching words are calculated, and the hit results corresponding to the target text are obtained by screening.
8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method according to any one of claims 1 to 5 by executing the computer instructions.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 5.
10. A computer program product, characterized in that The method comprises computer instructions for causing a computer to execute the method according to any one of claims 1 to 5.
Citation Information
Patent Citations
Monitoring method and device
CN112100616A
Sensitive text recognition method and device, electronic equipment and storage medium
CN115017898A
Method, device and equipment for identifying variant words in text and extracting original sensitive words
CN115081440A
Sensitive word matching method and device, electronic equipment and storage medium
CN116975379A
Combined word matching detection method and system, electronic equipment and storage medium
CN118245618A
