Word screening method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention relates to the artificial intelligence technology, in particular to a word screening method and device, electronic equipment and a storage medium. Obtaining a to-be-screened statement, and performing Chinese character recognition on the to-be-screened statement to obtain a to-be-screened Chinese character sequence; according to the Chinese character radical mapping table, splitting each Chinese character in the to-be-screened Chinese character sequence to obtain a to-be-screened radical sequence; according to the Chinese character radical mapping table, at least two adjacent radicals in the to-be-screened radical sequence are recombined, and to-be-matched Chinese characters are obtained; and matching the to-be-matched Chinese character with a preset screening word to obtain a word screening result. According to the embodiment of the invention, the screening effectiveness of the specific words is improved, and the recognition accuracy of the varied specific words is improved.

Description

Technical Field

[0001] This application relates to artificial intelligence technology, and more particularly to a word screening method, apparatus, electronic device, and storage medium. Background Technology

[0002] In the financial sector, effectively screening specific words in statements and user-submitted materials on corporate platforms is crucial for maintaining online order and ensuring the accuracy of data review results.

[0003] In existing technologies, the text to be screened is usually compared word by word with the pre-defined specific words to be screened.

[0004] However, with the emergence of various text variants, word-by-word comparison methods cannot effectively screen specific words. Summary of the Invention

[0005] This application provides a word screening method, apparatus, electronic device, and storage medium to improve the effectiveness of screening specific words and the accuracy of identifying specific words after textual variations.

[0006] In a first aspect, embodiments of this application provide a word screening method, which includes: Obtain the statement to be screened, and perform Chinese character recognition on the statement to be screened to obtain the sequence of Chinese characters to be screened; Based on the Chinese character radical mapping table, each Chinese character in the sequence to be screened is decomposed to obtain the radical sequence to be screened; Based on the Chinese character radical mapping table, at least two adjacent radicals in the radical sequence to be screened are recombined to obtain the Chinese characters to be matched; The Chinese characters to be matched are matched with the preset screening words to obtain the word screening results.

[0007] Secondly, embodiments of this application also provide a word screening device, which includes: The Chinese character recognition module is used to obtain the statement to be screened and to perform Chinese character recognition on the statement to obtain the sequence of Chinese characters to be screened. The Chinese character splitting module is used to split each Chinese character in the Chinese character sequence to be screened based on the Chinese character radical mapping table, so as to obtain the radical sequence to be screened. The radical recombination module is used to recombine at least two adjacent radicals in the radical sequence to be screened, based on the Chinese character radical mapping table, to obtain the Chinese character to be matched. The word screening result determination module is used to match the Chinese characters to be matched with preset screening words to obtain the word screening results.

[0008] Thirdly, embodiments of this application also provide an electronic device, which includes: One or more processors; Storage device for storing one or more programs; When one or more programs are executed by one or more processors, the one or more processors implement any of the word screening methods provided in the embodiments of this application.

[0009] Fourthly, embodiments of this application also provide a storage medium including computer-executable instructions, which, when executed by a computer processor, are used to perform any of the word screening methods provided in embodiments of this application.

[0010] Fifthly, embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements any of the word screening methods provided in embodiments of this application.

[0011] This application obtains a sentence to be screened and performs Chinese character recognition on the sentence to obtain a sequence of Chinese characters to be screened, accurately identifying the characters in the sentence. Based on a Chinese character radical mapping table, each character in the sequence of Chinese characters to be screened is split to obtain a sequence of radicals to be screened. Based on the Chinese character radical mapping table, at least two adjacent radicals in the sequence of radicals to be screened are recombined to obtain characters to be matched. These characters to be matched are then matched with preset screening words to obtain the word screening results. By splitting and recombining characters by radicals, it is possible to identify cases where specific word screening is circumvented through structural changes, improving the effectiveness of specific word screening and the accuracy of identifying specific words after variations. Therefore, the technical solution of this application solves the problem that the character-by-character comparison method cannot effectively screen specific words with the emergence of various character variations, achieving the effect of improving the effectiveness of specific word screening and the accuracy of identifying specific words after variations. Attached Figure Description

[0012] Figure 1 This is a flowchart of a word screening method according to Embodiment 1 of this application; Figure 2 This is a flowchart of a word screening method according to Embodiment 2 of this application; Figure 3 This is a flowchart of a word screening method according to Embodiment 3 of this application. Figure 4 This is a schematic diagram of the structure of a word screening device according to Embodiment 4 of this application; Figure 5 This is a schematic diagram of the structure of an electronic device according to Embodiment 5 of this application. Detailed Implementation

[0013] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0014] It should be noted that the terms "first" and "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0015] Example 1 Figure 1 This is a flowchart of a word screening method provided in Embodiment 1 of this application. This embodiment can be applied to the screening of specific words. The method can be executed by a word screening device, which can be implemented in software and / or hardware and specifically configured in a network platform, such as a server of the network platform.

[0016] See Figure 1 The word screening method shown includes the following steps: S110. Obtain the statement to be screened and perform Chinese character recognition on the statement to be screened to obtain the sequence of Chinese characters to be screened.

[0017] The statement to be screened can be a text sequence obtained in a scenario requiring specific word screening, used to obtain the Chinese character sequence to be screened. For example, the statement to be screened can be statements from user posts, comments, or submitted materials awaiting review on an enterprise platform. Specific words can be sensitive words or words set by the enterprise according to business needs. Chinese character recognition can identify the Chinese characters in the statement to be screened, used to obtain the Chinese character sequence to be screened. For example, regular expressions can be used to extract the Chinese character sequence from the statement to be screened, filtering out non-Chinese content such as numbers and punctuation. The Chinese character sequence to be screened can be the sequence of characters in the statement to be screened identified after Chinese character recognition.

[0018] Optionally, if simplified Chinese characters are used in the current region, after obtaining the sequence of Chinese characters to be screened, the sequence of Chinese characters to be screened can be detected for traditional Chinese characters, and after detecting traditional Chinese characters, according to the simplified-traditional Chinese mapping table, the traditional Chinese characters can be mapped to simplified Chinese characters.

[0019] After obtaining the sequence of Chinese characters to be screened, determine whether it is necessary to screen for traditional Chinese characters according to the type of text commonly used in the region where the corresponding sentence to be screened belongs. If the type of text commonly used in the region is simplified Chinese characters, the sequence of Chinese characters to be screened is detected for traditional Chinese characters, and after detecting traditional Chinese characters, according to the simplified-traditional Chinese mapping table, the traditional Chinese characters are mapped to simplified Chinese characters; if the type of text commonly used in the region is traditional Chinese characters, no screening for traditional Chinese characters is performed. The simplified-traditional Chinese mapping table can be a one-to-one mapping relationship between simplified Chinese characters and traditional Chinese characters established based on the "General Standard Chinese Character Table". By detecting traditional Chinese characters and mapping traditional Chinese characters to simplified Chinese characters according to the simplified-traditional Chinese mapping table, it is possible to avoid evading the screening of specific words through traditional Chinese characters and reduce the probability of missed detection.

[0020] Optionally, after obtaining the sequence of Chinese characters to be screened, the characters in the obtained sequence of Chinese characters to be screened can be converted into the UTF-8 (a professional term, a coding method) encoding format to provide a standardized input for subsequent Chinese character decomposition.

[0021] S120. According to the Chinese character radical mapping table, each Chinese character in the sequence of Chinese characters to be screened is split to obtain the sequence of radicals to be screened.

[0022] According to the Chinese character radical mapping table, each Chinese character in the sequence of Chinese characters to be screened is split to obtain the sequence of radicals to be screened.

[0023] In an optional embodiment, the Chinese character radical mapping table includes: a two-way mapping relationship between Chinese characters and corresponding radicals, variants corresponding to radicals, and the position weights of radicals in Chinese characters.

[0024] The Chinese character radical mapping table can be a radical decomposition rule library containing single characters and compound characters constructed in advance based on 201 main radicals and 100 affiliated radicals in the "Chinese Character Radical Table" specification. The mapping table supports two-way mapping in the forward direction (converting from Chinese characters to a radical sequence) and the reverse direction (combining radicals to candidate Chinese characters), and marks the combination priority of compound radicals. Since the text in the sentence to be screened is arranged horizontally, when mapping a single character to radicals, the processing of radicals with an up-down structure can be omitted. For example, the Chinese character "语" is only converted to the radicals "讠" and "吾".

[0025] The Chinese character radical mapping table can use a three-layer index structure to store radical features: the base layer stores the radical metadata defined by the Unicode standard; the association layer records the variants corresponding to the radical, which can include traditional characters; and the application layer maintains the positional weight of the radical in the Chinese character. For rare character scenarios, high-frequency, uncommon radical mapping rules are set to ensure that variant characters can be correctly identified when processing rare characters.

[0026] S130. Based on the Chinese character radical mapping table, recombined at least two adjacent radicals in the radical sequence to be screened to obtain the Chinese character to be matched.

[0027] A preset number of adjacent radicals can be selected from the sequence of radicals to be screened and recombine using a sliding window. Based on the Chinese character radical mapping table, the probability of the radicals in the sliding window recombinating into a Chinese character is determined. If the recombination probability is greater than a preset threshold, the Chinese character formed by the radicals in the sliding window is used as the character to be matched. When the radicals in the sliding window cannot be combined into a Chinese character, the size of the sliding window is increased, and recombination continues until the sliding window reaches its maximum size. Then, the sliding window is slid to the right and restored to its original size, and recombination of at least two adjacent radicals continues.

[0028] S140. Match the Chinese characters to be matched with the preset screening words to obtain the word screening results.

[0029] Preset screening terms can be words that need to be screened in advance based on relevant documents, regulations, or business needs, and are used to obtain the word screening results. Preset screening terms can be obtained from a predefined specific word database. For example, a specific word database can include a sensitive word database. The sensitive word database can be a database storing sensitive words. For example, preset screening terms can include: specific nouns, names of high-risk items, names of dangerous organizations, and high-risk business terms, etc. Word screening results can include matches and non-matches. When the word screening result is a match, the matched characters are output; when the word screening result is a non-match, an empty string or no matching characters are output.

[0030] The matching Chinese characters are matched one by one with each preset screening word. If the matching Chinese characters include words that are the same as preset screening words, the match is successful. At this time, the word screening result is determined to be a match, and the matched Chinese characters are output. If the matching Chinese characters do not match any preset screening words, the word screening result is determined to be a mismatch, and empty or no matching text is output.

[0031] It should be noted that all information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) involved in this disclosure are information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data comply with the relevant laws, regulations and standards of the relevant regions.

[0032] In existing technologies, the screening of specific words is mainly achieved by scanning the information text character by character or using regular expressions. However, this text matching method often fails when there are variations in the information text, and it is also difficult to work effectively when the text is deliberately processed, especially in a highly structured language like Chinese.

[0033] The technical solution of this embodiment obtains the statement to be screened and performs Chinese character recognition on the statement to obtain the sequence of Chinese characters to be screened, accurately identifying the characters in the statement; according to the Chinese character radical mapping table, each Chinese character in the sequence of Chinese characters to be screened is split to obtain the sequence of radicals to be screened; according to the Chinese character radical mapping table, at least two adjacent radicals in the sequence of radicals to be screened are recombined to obtain the Chinese characters to be matched; the Chinese characters to be matched are matched with preset screening words to obtain the word screening result. By splitting and recombining the characters into radicals, it is possible to identify cases where specific word screening is circumvented by structural changes, thereby improving the effectiveness of specific word screening and the accuracy of identifying specific words after variants. Therefore, the technical solution of this application solves the problem that the character-by-character comparison method cannot effectively screen specific words with the emergence of various character variants, achieving the effect of improving the effectiveness of specific word screening and the accuracy of identifying specific words after variants.

[0034] Example 2 Figure 2 This is a flowchart of a word screening method provided in Embodiment 2 of this application. The technical solution of this embodiment is further refined based on the above technical solution.

[0035] Furthermore, the phrase "based on the Chinese character radical mapping table, recombining at least two adjacent radicals in the radical sequence to be screened to obtain the Chinese character to be matched" is refined to: "selecting at least two adjacent radicals in the radical sequence to be screened through the matching window as the radicals to be matched; determining whether there are corresponding candidate Chinese characters for the radicals to be matched according to the Chinese character radical mapping table; if so, using the candidate Chinese characters as the Chinese characters to be matched" to recombine the Chinese characters to be matched.

[0036] See Figure 2 One word screening method shown includes: S210. Obtain the statement to be screened and perform Chinese character recognition on the statement to be screened to obtain the sequence of Chinese characters to be screened.

[0037] S220. According to the Chinese character radical mapping table, each Chinese character in the to-be-screened Chinese character sequence is split to obtain a to-be-screened radical sequence.

[0038] In an optional embodiment, according to the Chinese character radical mapping table, each Chinese character in the to-be-screened Chinese character sequence is split to obtain a to-be-screened radical sequence, including: determining the radical similarity between each to-be-split Chinese character and each candidate mapping Chinese character according to the radical edit distance, radical semantic coefficient, and radical position coefficient between each to-be-split Chinese character in the to-be-screened Chinese character sequence and each candidate mapping Chinese character in the Chinese character radical mapping table; determining the target mapping Chinese character from each candidate mapping Chinese character with a radical similarity greater than a preset threshold, so as to obtain the to-be-screened radical sequence according to the target mapping Chinese character.

[0039] The radical edit distance can be used to calculate the difference degree between two Chinese characters at the radical level to achieve similarity evaluation. First, a standardized feature library containing 214 Kangxi radicals is established, and a shape similarity weight is assigned to each radical. Three basic operations are defined: radical replacement (cost = 1 - shape similarity weight), radical insertion / deletion (fixed cost 1), and the dynamic programming algorithm is used to determine the radical edit distance. For the two input Chinese characters A and B, an (m + 1)×(n + 1) edit distance matrix is constructed, where m and n respectively represent the lengths of the radical sequences of Chinese characters A and B, and the sequence length is the number of radicals included. The minimum cumulative operation cost is calculated through the state transition equation, and finally normalized to a similarity score in the range of 0 - 1 to obtain the radical edit distance.

[0040] The weight calculation based on radical features introduces a semantic association factor and a position association factor of the radical. The semantic association factor uses the radical interpretation data in the Modern Chinese Dictionary to assign a semantic similarity of 0 - 9 to radicals with similar semantics and stores it in the semantic factor database. For example, radicals with similar semantics can be "言" and "讠". The structural position factor sets a difference coefficient according to the position of the radical in the Chinese character (left radical or right radical). The position weight formula can be determined by analyzing the Chinese character structure data through a convolutional neural network: W = 1 - 0.2×|pos1 - pos2|, where pos1 is the proportional coefficient of the radical on the left side and pos2 is the proportional coefficient of the radical on the right side. The structural position factor can be stored in the Chinese character radical mapping table.

[0041] The Chinese characters to be split can be any characters in the sequence of characters to be screened. Querying the Chinese character radical mapping table yields the radical edit distance between the character to be split and each candidate mapped character. Convolutional neural networks can be used to analyze the character structure data to obtain candidate radicals. A semantic factor database is then used to determine the semantic factors of each candidate radical and each radical in each candidate mapped character, which are then defined as radical semantic coefficients. Based on the position of each candidate radical and the structural position factor of each candidate mapped character, the radical position coefficient is determined. The final radical similarity is determined by the base edit distance score, the radical semantic coefficient, and the radical position coefficient, using the formula Sim=α×(1-tDistance)+(1-α)×(sWeight×pWeight). Here, tDistance is the radical edit distance, sWeight is the radical semantic coefficient, pWeight is the radical position coefficient, and α is the preset weight coefficient.

[0042] For example, candidate mapped Chinese characters whose radical similarity is greater than a preset threshold can be identified as target mapped Chinese characters. The preset threshold can be a pre-set minimum value of radical similarity for identifying target mapped Chinese characters.

[0043] Preferably, when multiple target mapped Chinese characters are obtained through a preset threshold, only the candidate mapped Chinese characters with the highest radical similarity can be identified as target mapped Chinese characters. The radical corresponding to the target mapped Chinese character in the Chinese character radical mapping table is identified as the target radical of the Chinese character to be split. The target radicals are arranged in order from left to right to obtain the radical sequence to be screened.

[0044] S230. Select at least two adjacent radicals in the radical sequence to be screened through the matching window as radicals to be matched.

[0045] The matching window can be a sliding window used to select radicals for recombination. The default size of the active window is 2, meaning two adjacent radicals can be selected. When the radicals in the sliding window cannot be recombinated into characters, the size of the sliding window can be increased, for example, by 1 each time. When a window threshold is reached, the sliding window can slide to the right one radical distance, continuing the selection process. The radicals in the matching window are used as radicals to be matched and recombinated to determine if there are corresponding candidate Chinese characters.

[0046] S240. Based on the Chinese character radical mapping table, determine whether there is a corresponding candidate Chinese character for the radical to be matched.

[0047] The candidate Chinese characters can be the Chinese characters that the to-be-matched radicals may be recombined into. According to the to-be-matched radicals, look up the position weights, semantic similarities, and radical edit distances corresponding to the to-be-matched radicals in each Chinese character in the Chinese character radical mapping table, and determine the similarity between the to-be-matched radicals and each Chinese character. Determine whether it is the corresponding candidate Chinese character through the similarity.

[0048] S250. If so, use the candidate Chinese character as the to-be-matched Chinese character.

[0049] If so, that is, if there are corresponding candidate Chinese characters for the to-be-matched radicals, use the candidate Chinese character as the to-be-matched Chinese character.优先选择在《现代汉语词典》中出现频率前30%的汉字组合作为待匹配汉字。每个组合阶段都会记录生成汉字的Unicode编码和部首位置信息,确保重组过程可追溯。

[0050] After determining the candidate Chinese characters, perform legality and probability verification on the candidate Chinese characters.

[0051] First, use a language model to perform legality verification on the candidate Chinese characters to exclude invalid combinations that do not conform to the character formation rules, such as "氵木木"; then use N-gram statistical probability to sort the remaining candidate Chinese characters, and preferentially try high-frequency reasonable combinations such as "沐". For polysemous radicals (such as "月" can be used as a radical or an independent character), the system dynamically adjusts the recombination weight through the radical position characteristics and the relationship with adjacent radicals. In the combination verification stage, introduce the Beam Search (a professional term, a heuristic search algorithm) algorithm to maintain the Top-K optimal candidates. When the optimal combination fails to match, automatically trigger the verification process of the sub-optimal combination until all possible combinations are exhausted or the preset attempt threshold is reached. This dual screening mechanism makes the recombination process follow both the Chinese character construction rules and the actual language usage habits. Each recombination attempt records the radical combination path and the matching result, and optimizes the subsequent recombination strategy through dynamic weight adjustment.

[0052] In an optional embodiment, after determining whether there are corresponding candidate Chinese characters for the to-be-matched radicals according to the Chinese character radical mapping table, it further includes: if not, determine whether the size of the matching window is the window threshold; if so, slide the matching window to the right by one sliding unit, and restore the size of the matching window to the initial value, and return to the step of selecting the to-be-matched radicals; otherwise, increase the size of the matching window, and return to the step of selecting the to-be-matched radicals.

[0053] If not, meaning there are no corresponding candidate characters for the radical to be matched, then it is determined whether the size of the matching window is equal to the window threshold. The window threshold can be the maximum number of radicals included in the matching window as preset. If yes, meaning the size of the matching window is equal to the window threshold, then the matching window is slid one unit to the right, and the size of the matching window is restored to the initial value, returning to S230, and the radical in the slid-up matching window is taken as the radical to be matched. For example, the initial window value can be 2. Otherwise, meaning the size of the matching window is less than the window threshold, then the size of the matching window is increased, and S230 is returned, and the radical in the increased-size matching window is taken as the radical to be matched.

[0054] To ensure matching efficiency, a constraint mechanism is implemented to determine whether a candidate Chinese character corresponds to a given radical in the matching process. First, a circuit breaker threshold is set for the maximum number of attempts. When the number of matching attempts reaches the preset limit, the process automatically terminates to avoid infinite looping and resource consumption. Second, a dynamic similarity threshold is introduced. The search is terminated early when the radical-level edit distance between a candidate Chinese character and its corresponding character in the sequence to be screened is lower than a set threshold. This circuit breaker mechanism ensures system stability through hard constraints.

[0055] S260. Match the Chinese characters to be matched with the preset screening words to obtain the word screening results.

[0056] The technical solution of this embodiment selects at least two adjacent radicals in the sequence of radicals to be screened through a matching window as radicals to be matched; according to the Chinese character radical mapping table, it is determined whether there are corresponding candidate Chinese characters for the radicals to be matched; if so, the candidate Chinese characters are used as the Chinese characters to be matched, and the split Chinese characters are recombined. The comprehensiveness of the recombination is ensured by the sliding window, which can improve the recognition accuracy of specific words after the structure has been changed.

[0057] Example 3 Figure 3 This is a flowchart of a word screening method provided in Embodiment 3 of this application. The technical solution of this embodiment is further refined based on the above technical solution.

[0058] Furthermore, the step of "matching the Chinese character to be matched with the preset screening words to obtain the word screening result" is further refined into: "matching the Chinese character to be matched with the shape and semantics of each target screening Chinese character in the preset screening words; if the match is successful, the matched Chinese character is taken as the word screening result; otherwise, the Chinese character to be matched is split and the process of reorganizing the radical sequence to be screened is returned to obtain the word screening result."

[0059] See Figure 3 One word screening method shown includes: S310. Obtain the statement to be screened and perform Chinese character recognition on the statement to be screened to obtain the sequence of Chinese characters to be screened.

[0060] S320. Based on the Chinese character radical mapping table, each Chinese character in the Chinese character sequence to be screened is split to obtain the radical sequence to be screened.

[0061] S330. Based on the Chinese character radical mapping table, recombine at least two adjacent radicals in the radical sequence to be screened to obtain the Chinese character to be matched.

[0062] S340. Match the Chinese characters to be matched with the character shapes and semantics of each preset screening word.

[0063] Character shape matching determines whether Chinese characters have the same shape, thus determining the word screening result. Semantic matching determines whether the semantics of words composed of adjacent characters to be matched match match the semantics of preset screening words, thus determining the word screening result. For example, context analysis can be used to determine whether the semantics of words composed of adjacent characters to be matched match match the semantics of preset screening words.

[0064] S350. If a match is successful, the matched Chinese character will be used as the word screening result.

[0065] If a match is successful, meaning that the word composed of adjacent Chinese characters to be matched matches the target word in the preset screening words in terms of both character shape and semantics, then the successfully matched Chinese characters to be matched will be output as the word screening result.

[0066] S360. Otherwise, split the Chinese characters to be matched and return to the step of obtaining the sequence of radicals to be screened.

[0067] Otherwise, if the character to be matched does not match the target character in the preset screening words in terms of shape or meaning, the character to be matched is split again, and the process returns to the step of recombining the radical sequence to be screened, i.e., S330. After the character to be matched does not match the target character in the preset screening words in terms of shape or meaning, the character to be matched is split according to the splitting record, the corresponding radical sequence to be screened is restored, and it is determined whether the size of the matching window when the character to be matched is recombined is the window threshold. If not, the matching window is increased and recombined, and the subsequent matching steps continue until the matching is successful or the size of the matching window is the window threshold, then the matching stops, and the word screening result is determined to be empty.

[0068] For cases that fail to match, the system will automatically record the following key analysis data: the original input text, the radical conversion sequence, all generated candidate combinations, the similarity calculation results, and the final matching status. The analysis module uses a statistical attribution method to focus on tracking two typical failure modes: one is the indivisible Chinese characters caused by the lack of a radical mapping table, and the other is the invalid Chinese characters caused by ambiguity in radical combinations.

[0069] The technical solution of this embodiment improves the accuracy of matching and the accuracy of specific word recognition by matching the Chinese character to be matched with the shape and semantics of each preset screening word through dual matching of shape and semantics. If the match is successful, the matched Chinese character is taken as the word screening result; otherwise, the Chinese character to be matched is split and the process returns to the step of recombining the radical sequence to be screened. If no match is found, the process returns to the recombining step to ensure the comprehensiveness of the combination, guarantee the accuracy of word screening, and reduce the probability of missed detection.

[0070] Example 4 Figure 4 The diagram shown is a structural schematic of a word screening device provided in Embodiment 4 of this application. This embodiment is applicable to situations where specific words need to be screened. The specific structure of the word screening device is as follows: The Chinese character recognition module 410 is used to acquire the statement to be screened and to perform Chinese character recognition on the statement to be screened to obtain the sequence of Chinese characters to be screened. The Chinese character splitting module 420 is used to split each Chinese character in the Chinese character sequence to be screened according to the Chinese character radical mapping table, so as to obtain the radical sequence to be screened. The radical recombination module 430 is used to recombine at least two adjacent radicals in the radical sequence to be screened based on the Chinese character radical mapping table to obtain the Chinese character to be matched. The word screening result determination module 440 is used to match the Chinese characters to be matched with preset screening words to obtain the word screening results.

[0071] The technical solution of this embodiment obtains the statement to be screened and performs Chinese character recognition on the statement to obtain the sequence of Chinese characters to be screened, accurately identifying the characters in the statement; according to the Chinese character radical mapping table, each Chinese character in the sequence of Chinese characters to be screened is split to obtain the sequence of radicals to be screened; according to the Chinese character radical mapping table, at least two adjacent radicals in the sequence of radicals to be screened are recombined to obtain the Chinese characters to be matched; the Chinese characters to be matched are matched with preset screening words to obtain the word screening result. By splitting and recombining the characters into radicals, it is possible to identify cases where specific word screening is circumvented by structural changes, thereby improving the effectiveness of specific word screening and the accuracy of identifying specific words after variants. Therefore, the technical solution of this application solves the problem that the character-by-character comparison method cannot effectively screen specific words with the emergence of various character variants, achieving the effect of improving the effectiveness of specific word screening and the accuracy of identifying specific words after variants.

[0072] Optional, the radical reorganization module 430 includes: The component to be matched determination unit is used to select at least two adjacent components from the component sequence to be screened through a matching window as components to be matched; The candidate Chinese character judgment unit is used to determine whether there is a corresponding candidate Chinese character for the radical to be matched, based on the Chinese character radical mapping table. The unit for determining the Chinese character to be matched is used to select the candidate Chinese character as the Chinese character to be matched if the condition is met.

[0073] Optionally, the radical reorganization module 430 also includes: The window threshold comparison unit is used to determine whether the size of the matching window is equal to the window threshold if the condition is not met. The matching window sliding unit is used to slide the matching window one sliding unit to the right if the condition is met, restore the size of the matching window to its initial value, and return to the step of selecting the radical to be matched. The matching window enlargement unit is used otherwise to increase the size of the matching window and return to the step of selecting the radical to be matched.

[0074] Optional, the Chinese character splitting module 420 includes: The radical similarity determination unit is used to determine the radical similarity between each Chinese character to be split and each candidate mapped Chinese character based on the radical edit distance, radical semantic coefficient, and radical position coefficient of each Chinese character to be split in the Chinese character sequence to be screened and each candidate mapped Chinese character in the Chinese character radical mapping table. The unit for determining the radical sequence to be screened is used to determine the target mapped Chinese character from each candidate mapped Chinese character whose radical similarity is greater than a preset threshold, so as to obtain the radical sequence to be screened based on the target mapped Chinese character.

[0075] Optionally, the word screening result determination module 440 includes: The character matching unit is used to match the Chinese character to be matched with the character shape and semantics of each preset screening word; The word screening result determination unit is used to determine the word screening result if a match is successful. Return to the reorganization step unit, otherwise, split the Chinese characters to be matched and return to the step of reorganizing the radical sequence to be screened.

[0076] Optional, Chinese character radical mapping table, including: The two-way mapping relationship between Chinese characters and their corresponding radicals, the variants of the radicals, and the positional weight of the radicals in Chinese characters.

[0077] The word screening device provided in this application embodiment can execute the word screening method provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects for executing the word screening method.

[0078] According to embodiments of the present invention, the present invention also provides an electronic device, a readable storage medium, and a computer program product.

[0079] Example 5 Figure 5 This is a schematic diagram of the structure of an electronic device provided in Embodiment 5 of this application, as shown below. Figure 5 As shown, the electronic device includes a processor 510, a memory 520, an input device 530, and an output device 540; the number of processors 510 in the electronic device can be one or more. Figure 5 Taking a processor 510 as an example; the processor 510, memory 520, input device 530, and output device 540 in the electronic device can be connected via a bus or other means. Figure 5 Taking the example of a connection between China and Israel via a bus.

[0080] The memory 520, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the word screening method in this application embodiment (e.g., Chinese character recognition module 410, Chinese character splitting module 420, radical recombination module 430, and word screening result determination module 440). The processor 510 executes various functional applications and data processing of the electronic device by running the software programs, instructions, and modules stored in the memory 520, thereby implementing the aforementioned word screening method.

[0081] The memory 520 may primarily include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a given function; the data storage area may store data created based on terminal usage. Furthermore, the memory 520 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 520 may further include memory remotely located relative to the processor 510, which can be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0082] Input device 530 can be used to receive input character information and generate key signal inputs related to user settings and function control of the electronic device. Output device 540 may include display devices such as a display screen.

[0083] Example 6 Embodiment Six of this application also provides a storage medium containing computer-executable instructions. When executed by a computer processor, the computer-executable instructions are used to perform a word screening method. The method includes: acquiring a sentence to be screened and performing Chinese character recognition on the sentence to be screened to obtain a sequence of Chinese characters to be screened; splitting each Chinese character in the sequence of Chinese characters to be screened according to a Chinese character radical mapping table to obtain a sequence of radicals to be screened; recombining at least two adjacent radicals in the sequence of radicals to be screened according to the Chinese character radical mapping table to obtain a Chinese character to be matched; and matching the Chinese character to be matched with a preset screening word to obtain a word screening result.

[0084] Of course, the computer-executable instructions provided in the embodiments of this application are not limited to the method operations described above, but can also perform related operations in the word screening method provided in any embodiment of this application.

[0085] Based on the above description of the implementation methods, those skilled in the art can clearly understand that this application can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0086] It is worth noting that in the above-mentioned word screening device embodiments, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of this application.

[0087] Note that the above are merely preferred embodiments and the technical principles employed in this application. Those skilled in the art will understand that this application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of this application. Therefore, although this application has been described in detail through the above embodiments, this application is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of this application, the scope of which is determined by the scope of the appended claims.

Claims

1. A word screening method, characterized in that, include: Obtain the statement to be screened, and perform Chinese character recognition on the statement to be screened to obtain the sequence of Chinese characters to be screened; Based on the Chinese character radical mapping table, each Chinese character in the sequence of Chinese characters to be screened is decomposed to obtain the sequence of radicals to be screened; Based on the Chinese character radical mapping table, at least two adjacent radicals in the radical sequence to be screened are recombined to obtain the Chinese characters to be matched; The Chinese characters to be matched are matched with preset screening words to obtain the word screening results.

2. The method according to claim 1, characterized in that, The process of recombining at least two adjacent radicals in the radical sequence to be screened to obtain the Chinese character to be matched includes: Select at least two adjacent radicals from the radical sequence to be screened through the matching window as radicals to be matched; Based on the Chinese character radical mapping table, determine whether there is a corresponding candidate Chinese character for the radical to be matched; If so, the candidate Chinese character will be used as the Chinese character to be matched.

3. The method according to claim 2, characterized in that, After determining whether a candidate Chinese character exists for the radical to be matched based on the Chinese character radical mapping table, the method further includes: If not, then determine whether the size of the matching window is the window threshold; If so, slide the matching window one sliding unit to the right, restore the size of the matching window to its initial value, and return to the step of selecting the radical to be matched; Otherwise, increase the size of the matching window and return to the step of selecting the radical to be matched.

4. The method according to claim 1, characterized in that, The step involves splitting each Chinese character in the sequence to be screened according to the Chinese character radical mapping table to obtain the radical sequence to be screened, including: Based on the radical edit distance, radical semantic coefficient, and radical position coefficient of each Chinese character to be split in the Chinese character sequence to be screened and each candidate mapped Chinese character in the Chinese character radical mapping table, the radical similarity between each Chinese character to be split and each candidate mapped Chinese character is determined. The target mapped Chinese character is determined from each of the candidate mapped Chinese characters whose radical similarity is greater than a preset threshold, so as to obtain the radical sequence to be screened based on the target mapped Chinese character.

5. The method according to claim 1, characterized in that, The step of matching the Chinese characters to be matched with preset screening words to obtain the word screening results includes: The Chinese characters to be matched are matched with the character shapes and semantics of each preset screening word; If a match is successful, the matched Chinese character will be used as the word screening result. Otherwise, the Chinese characters to be matched are split and the process returns to the step of recombining the radical sequence to be screened.

6. The method according to claim 1, characterized in that, The Chinese character radical mapping table includes: The two-way mapping relationship between Chinese characters and their corresponding radicals, the variants of the radicals, and the positional weight of the radicals in Chinese characters.

7. A word screening device, characterized in that, include: The Chinese character recognition module is used to acquire the statement to be screened and to perform Chinese character recognition on the statement to be screened to obtain the sequence of Chinese characters to be screened. The Chinese character splitting module is used to split each Chinese character in the Chinese character sequence to be screened according to the Chinese character radical mapping table to obtain the radical sequence to be screened; The radical recombination module is used to recombine at least two adjacent radicals in the radical sequence to be screened based on the Chinese character radical mapping table to obtain the Chinese character to be matched. The word screening result determination module is used to match the Chinese characters to be matched with preset screening words to obtain word screening results.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the word screening method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the word screening method as described in any one of claims 1-6.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the word screening method as described in any one of claims 1-6.

Images