Method and device for expanding corpus, medium and program product

Abstract

The invention provides a method and device for expanding a corpus, electronic equipment, a computer readable medium and a computer program product. The method comprises the steps of obtaining multiple target corpora based on a to-be-expanded corpus; for each target corpus, generating one or more fuzzy matching vocabularies corresponding to the target corpus based on fuzzy matching processing, and taking the one or more fuzzy matching vocabularies as candidate expanded corpora; and selecting one or more candidate expanded corpora as the expanded corpora of the target corpus based on the correlation scores corresponding to the candidate expanded corpora so as to expand the corpus. According to the method, the corpus in the to-be-expanded corpus is subjected to fuzzy matching processing to generate the related fuzzy matching vocabularies, so that the corpus is effectively expanded; four-corner coding, character splitting or image processing are carried out through the pre-trained model to generate fuzzy matching vocabularies, and the efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device, electronic device, computer-readable medium and computer program product for expanding a corpus. Background Art

[0002] Optical character recognition (OCR) technology plays an important role in the field of text recognition. Although it has significant effects in automatically extracting and filtering information, it also has some challenges. First, when predicting the risk of data content, the risk vocabulary that OCR technology relies on needs to be manually updated and maintained regularly, which not only involves high costs, but also makes it difficult to ensure the instant update of information, resulting in some newly emerging specific words or their derivative forms may not be recognized in time. Secondly, relying solely on the preset vocabulary database, its coverage is limited. In the face of some obscure or indirect information expressions, traditional recognition methods may find it difficult to achieve effective detection and response. Summary of the invention

[0003] Various aspects of the present application provide a method, an apparatus, an electronic device, a computer-readable medium, and a computer program product for expanding a corpus.

[0004] In one aspect of the present application, a method for expanding a corpus is provided, wherein the method comprises:

[0005] Based on the corpus to be expanded, multiple target corpora are obtained;

[0006] For each target corpus, one or more fuzzy matching words corresponding to the target corpus are generated based on fuzzy matching processing as candidate expanded corpus;

[0007] Based on the relevance scores corresponding to the candidate expanded corpora, one or more candidate expanded corpora are selected as expanded corpora of the target corpus to expand the corpus.

[0008] In one aspect of the present application, a device for expanding a corpus is provided, wherein the device comprises:

[0009] A device for acquiring a plurality of target corpora based on the corpus to be expanded;

[0010] A device for generating, for each target corpus, one or more fuzzy matching words corresponding to the target corpus based on fuzzy matching processing as candidate expanded corpus;

[0011] A device for selecting one or more candidate expanded corpora as expanded corpora of the target corpus based on the relevance scores corresponding to the candidate expanded corpora, so as to expand the corpus.

[0012] Another aspect of the present application is an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method of an embodiment of the present application.

[0013] In another aspect of the present application, a computer-readable storage medium is provided, on which computer program instructions are stored. The computer program instructions can be executed by a processor to implement the method of the embodiment of the present application.

[0014] Another aspect of the present application provides a computer program product, including a computer program, which implements the method of the embodiment of the present application when executed by a processor.

[0015] In the solution provided in the embodiment of the present application, the corpus is effectively expanded by performing fuzzy matching processing on the corpus in the corpus to be expanded to generate relevant fuzzy matching vocabulary; the efficiency is improved by performing four-corner coding, character decomposition or image processing through a pre-trained model; the relevance of the fuzzy matching vocabulary is improved by scoring the relevance of the fuzzy matching vocabulary and selecting to add it based on the score; in the risk prediction scenario, the prediction model is trained based on the expanded risk vocabulary obtained in the embodiment of the present application, which can not only recognize traditional risk words, but also recognize more vague or indirect inappropriate expressions, thereby improving the coverage and accuracy of the audit, reducing the workload of manual maintenance, and improving the timeliness of the audit. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0017] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0018] Figure 1 A schematic diagram of a process for expanding a corpus provided in an embodiment of the present application is shown;

[0019] Figure 2 A schematic diagram of the structure of a device for expanding a corpus provided in an embodiment of the present application is shown;

[0020] Figure 3A schematic diagram of the structure of a device suitable for implementing the solution in the embodiment of the present application is shown.

[0021] The same or similar reference numerals in the drawings represent the same or similar components. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0023] In a typical configuration of the present application, the terminal and the equipment of the service network each include one or more processors (CPU), input / output interface, network interface and memory.

[0024] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0025] Computer readable media include permanent and non-permanent, removable and non-removable media, and can be implemented by any method or technology to store information. Information can be computer program instructions, data structures, modules of programs or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0026] Figure 1 A schematic flow chart of a method for expanding a corpus provided in an embodiment of the present application is shown. The method at least includes step S101, step S102 and step S103.

[0027] In actual scenarios, the execution subject of the method can be a network device, or an application running on a network device, wherein the network device includes but is not limited to a network host, a single network server, a plurality of network server sets, or a collection of computers based on cloud computing, and can be used to implement some processing functions when setting an alarm. Here, the cloud is composed of a large number of hosts or network servers based on cloud computing, wherein cloud computing is a type of distributed computing, a virtual computer composed of a group of loosely coupled computer sets.

[0028] In order to overcome the limitations of traditional solutions that rely on risk vocabulary for risk prediction, the present application proposes a method for expanding risk words based on a model to generate fuzzy matching vocabulary. The method of the embodiment of the present application uses known risk words as seed words to generate fuzzy matching vocabulary related to them, thereby effectively expanding the risk vocabulary. It can not only identify traditional risk words, but also identify more fuzzy or indirect inappropriate expressions corresponding to more risk words, thereby improving the coverage and accuracy of the audit.

[0029] After the risk vocabulary is expanded according to the method of the embodiment of the present application, a model for outputting risk prediction results can be trained based on the expanded risk vocabulary.

[0030] Reference Figure 1 In step S101, a plurality of target corpora are obtained based on the corpus to be expanded.

[0031] The corpus includes various electronic text libraries that can be used for language research and natural language processing. The method can select all or part of the corpus as the target corpus.

[0032] In the risk prediction scenario, the corpus to be expanded is a risk word library, and the target corpus is a plurality of keywords contained in the risk word library, which can be used as seed words for expansion. The risk word library can be regarded as a special form of corpus, which is specifically used for risk prediction of content. When constructing a risk word library, manual construction or machine learning-based methods can be used to ensure the update and accuracy of the risk word library.

[0033] In step S102, for each target corpus, one or more fuzzy matching words corresponding to the target corpus are generated based on fuzzy matching processing as candidate expanded corpus.

[0034] Specifically, the method uses each target corpus as a seed word and adopts a predetermined fuzzy matching algorithm to generate one or more fuzzy matching words corresponding to the target corpus.

[0035] According to one embodiment, the method for generating one or more fuzzy matching words corresponding to the target corpus based on fuzzy matching processing includes but is not limited to at least any one of the following:

[0036] 1) Generating one or more fuzzy matching words corresponding to the target corpus by means of four-corner coding;

[0037] Among them, four-corner coding is a Chinese character retrieval method that sorts Chinese characters by encoding the strokes of the four corners of a Chinese character (i.e., the upper left corner, the upper right corner, the lower left corner, and the lower right corner) into numbers. This method can help identify characters that are similar in shape to sensitive words and may thus be used to generate fuzzy matching words.

[0038] Specifically, each character in the target corpus is converted into a four-corner code. According to the rules of four-corner coding, codes that are similar to the original code in one or some positions are found. These similar codes are then converted back into Chinese characters to form new words, which serve as the fuzzy matching words for the target corpus.

[0039] 2) Generating one or more fuzzy matching words corresponding to the target corpus by performing character decomposition processing;

[0040] Character decomposition processing decomposes each character in the target corpus into its basic components, such as radicals or other structural units. By this means, words that are structurally similar to the target corpus are found and used as the fuzzy matching words for the target corpus. For example, decomposing the character "Zhang Ming" results in "bow long sun moon".

[0041] 3) Generating one or more fuzzy matching words corresponding to the target corpus by performing picture conversion processing; by means of picture conversion processing, characters that are similar to a specific Chinese character can be found in terms of visual similarity.

[0042] Among them, the steps of generating one or more fuzzy matching words corresponding to the target corpus by performing picture conversion processing further include steps S1021 to S1023:

[0043] In step S1021, each Chinese character in the target corpus is converted into Chinese character pictures in multiple predetermined fonts.

[0044] Among them, the Chinese character pictures may include black-and-white pictures with a fixed pixel count.

[0045] In step S1022, a predetermined image processing algorithm is used to generate similar Chinese character pictures corresponding to each Chinese character picture.

[0046] Among them, the similar Chinese character pictures include pictures that are similar to the original Chinese character pictures but slightly different.

[0047] The image processing algorithm includes but is not limited to dilation, erosion, filtering, etc.

[0048] Optionally, the method selects one or more similar Chinese character pictures with high similarity from the generated similar Chinese character pictures.

[0049] In step S1023, each Chinese character of the target corpus and its corresponding similar Chinese character image are combined to obtain the fuzzy matching vocabulary of the target corpus.

[0050] Optionally, for each Chinese character, the Chinese character corresponding to the similar Chinese character picture of the Chinese character is combined with other Chinese characters in the target corpus to form a new vocabulary as the fuzzy matching vocabulary of the target corpus.

[0051] Optionally, for each Chinese character, similar Chinese character images of the Chinese character and similar Chinese character images of other Chinese characters in the target corpus are spliced ​​together using image stitching technology to obtain a composite image; then, the text contained in the composite image is identified to obtain a new vocabulary as the fuzzy matching vocabulary of the target corpus.

[0052] Optionally, the method can also be combined with other methods to select candidate expansion corpora. For example, the edit distance algorithm is used to generate vocabulary within a certain edit distance from the target corpus as candidate expansion corpora. Edit distance is a method of measuring the difference between two strings, and calculates the minimum number of editing operations required to convert one string into another. The editing operations include but are not limited to inserting, deleting, and replacing characters.

[0053] Continue to Figure 1 To illustrate, in step S103, based on the relevance scores corresponding to the candidate expanded corpora, one or more candidate expanded corpora are selected as expanded corpora of the target corpus to expand the corpus.

[0054] According to one embodiment, step S103 includes step S1031 and step S1032.

[0055] In step S1031 , for each target corpus, a relevance score is performed on each candidate expansion corpus of the target corpus and the target corpus based on a predetermined algorithm.

[0056] Optionally, the method scores each candidate expanded corpus based on the relevance between the candidate expanded corpus and the target corpus, and the method evaluates the relevance between the candidate expanded corpus and the target corpus based on at least any one of the following:

[0057] 1) Semantic relevance: evaluate whether the candidate expansion words are semantically close or related to the target corpus words;

[0058] 2) Co-occurrence frequency: Count the frequency of co-occurrence of candidate expansion words and target corpus in a large amount of text;

[0059] 3) Contextual consistency: Analyze the context in which the candidate expansion word appears to see if it matches the common context of the target corpus.

[0060] According to one embodiment, if the corpus to be expanded is a risk word library, the score can reflect the similarity between the word and the original risk word. Optionally, the relevance scoring algorithm can be set based on the edit distance between the word and the risk word, the frequency of occurrence of the word, context information, etc.

[0061] In step S1032, based on the scores corresponding to the candidate expanded corpora, one or more candidate expanded corpora are selected as expanded corpora of the target corpus.

[0062] Optionally, the candidate expanded corpora are sorted from high to low according to the similarity based on the scores, and a predetermined number of candidate expanded corpora with high rankings are selected as expanded corpora of the target corpus.

[0063] Optionally, a preset scoring threshold is obtained, and a candidate expanded corpus with a score higher than the preset threshold is selected as the expanded corpus of the target corpus.

[0064] After obtaining the expanded corpus corresponding to each target corpus, the expanded corpus is added to the corpus, thereby completing the expansion of the corpus.

[0065] According to one embodiment, the method periodically obtains a corpus to be expanded, and performs operations from step S101 to step S103 to update the corpus.

[0066] According to one embodiment, the method inputs the target corpus into a pre-trained corpus expansion model, thereby performing the above-mentioned fuzzy matching process and expanding the corpus through the corpus expansion model.

[0067] The method of this embodiment trains the corpus expansion model through the following steps S104 and S105.

[0068] In step S104, training data corresponding to the fuzzy matching process is obtained.

[0069] The training data includes data corresponding to one or more predetermined fuzzy matching algorithms.

[0070] Optionally, the training data includes but is not limited to at least any of the following:

[0071] 1) Four-corner code data: collect Chinese characters and their corresponding four-corner codes to build a four-corner code database;

[0072] 2) Character decomposition data: You can use the Chinese character decomposition library to obtain the radical and structure information of Chinese characters;

[0073] 3) Chinese character image data: By preparing the image data of Chinese characters, ensure that each Chinese character has corresponding images in multiple fonts;

[0074] In step S105, a model is trained based on the training data, so that the model learns how to perform fuzzy matching processing on input Chinese characters.

[0075] Optionally, a feature extraction tool is used to extract features of the training data as input features of the pre-trained corpus expansion model. For example, for four-corner coded data, information such as the pronunciation (including initials, finals, and tones), character shapes (radicals, radicals), and four-corner symbols of Chinese characters are extracted, and these feature information is used as input features of the model.

[0076] Optionally, the pre-trained corpus expansion model is a multi-task learning model that uses a shared underlying feature extraction layer and outputs four-corner codes, word splitting results, and image features at the same time. For four-corner codes and word splitting processing, a classification loss function can be used for model training; for image processing, a regression loss function can be used for model training.

[0077] For example, a model can be trained based on the above steps S104 and S105 to simultaneously process four-corner coding, character splitting and image conversion, thereby expanding the risk word library. Specifically, all or part of the keywords in the risk word library are input into the trained model, and in the model, four-corner coding, character splitting and image processing are performed on each Chinese character included in the input keyword to generate fuzzy matching vocabulary. Then, the generated vocabulary is merged with the existing risk word library to update the risk word library.

[0078] According to the method of the embodiment of the present application, the corpus is effectively expanded by performing fuzzy matching processing on the corpus in the corpus to be expanded to generate relevant fuzzy matching vocabulary; the efficiency is improved by performing four-corner coding, character decomposition or image processing through a pre-trained model to generate fuzzy matching vocabulary; the relevance of the fuzzy matching vocabulary is improved by scoring the relevance and selecting to add to based on the score; in the risk prediction scenario, the prediction model is trained based on the expanded risk vocabulary obtained in the embodiment of the present application, which can not only identify traditional risk words, but also more vague or indirect inappropriate expressions, thereby improving the coverage and accuracy of the audit, reducing the workload of manual maintenance, and improving the timeliness of the audit.

[0079] Figure 2A schematic diagram of the structure of a device for expanding a corpus provided in an embodiment of the present application is shown. The device includes: a device for acquiring multiple target corpora based on the corpus to be expanded (hereinafter referred to as "corpus acquisition device 101"), a device for generating one or more fuzzy matching words corresponding to the target corpus as candidate expanded corpora based on fuzzy matching processing for each target corpus (hereinafter referred to as "fuzzy matching device 102"), and a device for expanding the corpus based on the candidate expanded corpora corresponding to each target corpus (hereinafter referred to as "corpus expansion device 103").

[0080] Reference Figure 2 The corpus acquisition device 101 acquires a plurality of target corpora based on the corpus to be expanded.

[0081] The corpus includes various electronic text libraries that can be used for language research and natural language processing. The method can select all or part of the corpus as the target corpus.

[0082] In the risk prediction scenario, the corpus to be expanded is a risk word library, and the target corpus is a plurality of keywords contained in the risk word library, which can be used as seed words for expansion. The risk word library can be regarded as a special form of corpus, which is specifically used for risk prediction of content. When constructing a risk word library, manual construction or machine learning-based methods can be used to ensure the update and accuracy of the risk word library.

[0083] For each target corpus, the fuzzy matching device 102 generates one or more fuzzy matching words corresponding to the target corpus as candidate expanded corpus based on fuzzy matching processing.

[0084] Specifically, the fuzzy matching device 102 uses each target corpus as a seed word and adopts a predetermined fuzzy matching algorithm to generate one or more fuzzy matching words corresponding to the target corpus.

[0085] According to one embodiment, the manner in which the fuzzy matching device 102 generates one or more fuzzy matching words corresponding to the target corpus based on the fuzzy matching process includes but is not limited to at least any one of the following:

[0086] 1) Generate one or more fuzzy matching words corresponding to the target corpus by four-corner coding;

[0087] Among them, four-corner coding is a Chinese character search method that sorts Chinese characters by encoding the strokes of the four corners of the Chinese character (i.e., upper left corner, upper right corner, lower left corner, and lower right corner) into numbers. This method can help identify characters that are similar in shape to sensitive words, and thus may be used to generate fuzzy matching words.

[0088] Specifically, each character of the target corpus is converted into a four-corner code. According to the rules of the four-corner code, codes that are similar to the original code at one or more positions are found. These similar codes are then converted back into Chinese characters to form new words, which serve as the fuzzy matching words for the target corpus.

[0089] 2) Generate one or more fuzzy matching words corresponding to the target corpus by performing character decomposition processing;

[0090] Character decomposition processing decomposes each character of the target corpus into its basic components, such as radicals or other structural units. In this way, words that are structurally similar to the target corpus are found and used as the fuzzy matching words for the target corpus. For example, decomposing the character "Zhang Ming" results in "bow long sun moon".

[0091] 3) Generate one or more fuzzy matching words corresponding to the target corpus by performing image conversion processing; By means of image conversion processing, characters that are visually similar to a specific Chinese character can be found.

[0092] Specifically, each Chinese character of the target corpus is converted into Chinese character images in multiple predetermined fonts, where the Chinese character images may include black and white images with a fixed number of pixels.

[0093] Next, use a predetermined image processing algorithm to generate similar Chinese character images for each Chinese character image, and select one or more similar Chinese character images with a high degree of similarity.

[0094] Next, combine each Chinese character of the target corpus with its corresponding similar Chinese character image to obtain the fuzzy matching words for the target corpus.

[0095] Optionally, for each Chinese character, the Chinese character corresponding to the similar Chinese character image of the Chinese character is combined with other Chinese characters in the target corpus to form new words, which serve as the fuzzy matching words for the target corpus.

[0096] Optionally, for each Chinese character, the similar Chinese character image of the Chinese character and the similar Chinese character images of other Chinese characters in the target corpus are spliced together through image splicing technology to obtain a composite image; Next, the text contained in the composite image is recognized to obtain new words, which serve as the fuzzy matching words for the target corpus.

[0097] Optionally, the fuzzy matching device 102 can also combine other methods to select candidate extended corpora. For example, generate words within a certain edit distance from the target corpus through the edit distance algorithm as candidate extended corpora. The edit distance is a method for measuring the difference between two strings, and it calculates the minimum number of edit operations required to convert one string into another string. Among them, the edit operations include but are not limited to inserting, deleting, replacing characters, etc.

[0098] Continue with Figure 2 To illustrate, the corpus expansion device 103 selects one or more candidate expansion corpora as expansion corpora of the target corpus based on the relevance scores corresponding to the candidate expansion corpora, so as to expand the corpus.

[0099] According to one embodiment, the corpus expansion device 103 further includes a corpus scoring device and a corpus selection device.

[0100] The corpus scoring device scores each target corpus on each candidate expanded corpus of the target corpus and the target corpus based on a predetermined algorithm.

[0101] Optionally, the corpus scoring device scores each candidate expanded corpus based on the relevance between the candidate expanded corpus and the target corpus. The corpus scoring device evaluates the relevance between the candidate expanded corpus and the target corpus based on at least one of the following:

[0102] 1) Semantic relevance: evaluate whether the candidate expansion words are semantically close or related to the target corpus words;

[0103] 2) Co-occurrence frequency: Count the frequency of co-occurrence of candidate expansion words and target corpus in a large amount of text;

[0104] 3) Contextual consistency: Analyze the context in which the candidate expansion word appears to see if it matches the common context of the target corpus.

[0105] According to one embodiment, if the corpus to be expanded is a risk word library, the score can reflect the similarity between the word and the original risk word. Optionally, the relevance scoring algorithm can be set based on the edit distance between the word and the risk word, the frequency of occurrence of the word, context information, etc.

[0106] The corpus selection device selects one or more candidate expanded corpora as expanded corpora of the target corpus based on the scores corresponding to the candidate expanded corpora.

[0107] Optionally, the corpus scoring device sorts the candidate expanded corpora from high to low according to similarity based on the scores, and selects a predetermined number of candidate expanded corpora with high rankings as expanded corpora of the target corpus.

[0108] Optionally, the corpus scoring device obtains a preset scoring threshold, and selects a candidate expanded corpus with a score higher than the preset threshold as the expanded corpus of the target corpus.

[0109] After obtaining the expanded corpus corresponding to each target corpus, the device adds the expanded corpus to the corpus, thereby completing the expansion of the corpus.

[0110] According to one embodiment, the device periodically acquires a corpus to be expanded, and performs operations of the corpus acquisition device 101, the fuzzy matching device 102 and the corpus expansion device 103 to update the corpus.

[0111] According to one embodiment, the device inputs the target corpus into a pre-trained corpus expansion model, thereby performing the above-mentioned fuzzy matching processing and expanding the corpus through the corpus expansion model.

[0112] The device of this embodiment further includes a training data acquisition device and a model matching processing device.

[0113] The training data acquisition device acquires the training data corresponding to the fuzzy matching process.

[0114] The training data includes data corresponding to one or more predetermined fuzzy matching algorithms.

[0115] Optionally, the training data includes but is not limited to at least any of the following:

[0116] 1) Four-corner code data: collect Chinese characters and their corresponding four-corner codes to build a four-corner code database;

[0117] 2) Character decomposition data: You can use the Chinese character decomposition library to obtain the radical and structure information of Chinese characters;

[0118] 3) Chinese character image data: By preparing the image data of Chinese characters, ensure that each Chinese character has corresponding images in multiple fonts;

[0119] The model matching processing device trains the model based on the training data, so that the model learns how to perform fuzzy matching processing on the input Chinese characters.

[0120] Optionally, a feature extraction tool is used to extract features of the training data as input features of the pre-trained corpus expansion model. For example, for four-corner coded data, information such as the pronunciation (including initials, finals, and tones), character shapes (radicals, radicals), and four-corner symbols of Chinese characters are extracted, and these feature information is used as input features of the model.

[0121] Optionally, the pre-trained corpus expansion model is a multi-task learning model that uses a shared underlying feature extraction layer and outputs four-corner codes, word splitting results, and image features at the same time. For four-corner codes and word splitting processing, a classification loss function can be used for model training; for image processing, a regression loss function can be used for model training.

[0122] For example, a model can be trained based on the above steps S104 and S105 to simultaneously process four-corner coding, character splitting and image conversion, thereby expanding the risk word library. Specifically, all or part of the keywords in the risk word library are input into the trained model, and in the model, four-corner coding, character splitting and image processing are performed on each Chinese character included in the input keyword to generate fuzzy matching vocabulary. Then, the generated vocabulary is merged with the existing risk word library to update the risk word library.

[0123] According to the device of the embodiment of the present application, the corpus is effectively expanded by performing fuzzy matching processing on the corpus in the corpus to be expanded to generate related fuzzy matching vocabulary; the efficiency is improved by performing four-corner coding, character decomposition or image processing through a pre-trained model to generate fuzzy matching vocabulary; the relevance of the fuzzy matching vocabulary is improved by scoring the relevance and selecting to add to based on the score; in the risk prediction scenario, the prediction model is trained based on the expanded risk vocabulary obtained in the embodiment of the present application, which can not only identify traditional risk words, but also more vague or indirect inappropriate expressions, thereby improving the coverage and accuracy of the audit, reducing the workload of manual maintenance, and improving the timeliness of the audit.

[0124] Based on the same inventive concept, an electronic device is also provided in an embodiment of the present application, and the method corresponding to the electronic device may be the method for expanding the corpus in the aforementioned embodiment, and its principle of solving the problem is similar to that of the method. The electronic device provided in an embodiment of the present application includes: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the methods and / or technical solutions of the aforementioned multiple embodiments of the present application.

[0125] The electronic device may be a user device, or a device formed by integrating a user device and a network device through a network, or may be an application running on the above device. The user device includes but is not limited to various terminal devices such as computers, mobile phones, tablet computers, smart watches, and bracelets. The network device includes but is not limited to network hosts, single network servers, multiple network server sets, or cloud computing-based computer sets, which can be used to implement some processing functions when setting an alarm. Here, the cloud is composed of a large number of hosts or network servers based on cloud computing, where cloud computing is a type of distributed computing, a virtual computer composed of a group of loosely coupled computer sets.

[0126] Figure 3The structure of a device suitable for implementing the method and / or technical solution in the embodiment of the present application is shown, and the device 1200 includes a central processing unit (CPU, Central Processing Unit) 1201, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM, Read Only Memory) 1202 or the program loaded from the storage part 1208 to the random access memory (RAM, Random Access Memory) 1203. In RAM1203, various programs and data required for system operation are also stored. CPU 1201, ROM 1202 and RAM 1203 are connected to each other through bus 1204. Input / output (I / O, Input / Output) interface 1205 is also connected to bus 1204.

[0127] The following components are connected to the I / O interface 1205: an input section 1206 including a keyboard, a mouse, a touch screen, a microphone, an infrared sensor, etc.; an output section 1207 including a cathode ray tube (CRT), a liquid crystal display (LCD), an LED display, an OLED display, etc., and a speaker, etc.; a storage section 1208 including one or more computer-readable media such as a hard disk, an optical disk, a magnetic disk, a semiconductor memory, etc.; and a communication section 1209 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 1209 performs communication processing via a network such as the Internet.

[0128] In particular, the methods and / or embodiments in the embodiments of the present application may be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. When the computer program is executed by the central processing unit (CPU) 1201, the above functions defined in the method of the present application are executed.

[0129] Another embodiment of the present application further provides a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions can be executed by a processor to implement the methods and / or technical solutions of any one or more embodiments of the present application described above.

[0130] Specifically, the present embodiment may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device, or device.

[0131] Computer readable signal media may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer readable program code. Such propagated data signals may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than a computer readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0132] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0133] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0134] The flow chart or block diagram in the accompanying drawings shows the possible architecture, function and operation of the equipment, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated system for hardware that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0135] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0136] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or page components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0137] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0138] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0139] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform some steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.

[0140] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

[0141] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in a device claim can also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any particular order.

Claims

1. A method for expanding a corpus, wherein: The method comprises: Based on the corpus to be expanded, multiple target corpora are obtained; For each target corpus, one or more fuzzy matching words corresponding to the target corpus are generated based on fuzzy matching processing as candidate expanded corpus; Based on the relevance scores corresponding to the candidate expanded corpora, one or more candidate expanded corpora are selected as expanded corpora of the target corpus to expand the corpus.

2. The method according to claim 1, wherein: For each target corpus, generating one or more fuzzy matching words corresponding to the target corpus based on fuzzy matching processing includes: One or more fuzzy matching words corresponding to the target corpus are generated by four-corner coding.

3. The method according to claim 1, wherein: For each target corpus, generating one or more fuzzy matching words corresponding to the target corpus based on fuzzy matching processing includes: One or more fuzzy matching words corresponding to the target corpus are generated by performing character decomposition processing.

4. The method according to claim 1, wherein: For each target corpus, generating one or more fuzzy matching words corresponding to the target corpus based on fuzzy matching processing includes: One or more fuzzy matching words corresponding to the target corpus are generated by performing image conversion processing.

5. The method according to claim 4, wherein: The generating one or more fuzzy matching words corresponding to the target corpus by performing image conversion processing comprises: Convert each Chinese character in the target corpus into Chinese character images in multiple predetermined fonts; Generate a similar Chinese character picture corresponding to each Chinese character picture using a predetermined image processing algorithm; Each Chinese character in the target corpus and its corresponding similar Chinese character image are combined to obtain the fuzzy matching vocabulary of the target corpus.

6. The method according to claim 5, wherein: The fuzzy matching vocabulary of the target corpus obtained by combining each Chinese character of the target corpus and its corresponding similar Chinese character image includes: For each Chinese character, the Chinese character corresponding to the similar Chinese character image of the Chinese character is combined with other Chinese characters in the target corpus to form a new vocabulary as the fuzzy matching vocabulary of the target corpus.

7. The method according to claim 5, wherein: The fuzzy matching vocabulary of the target corpus obtained by combining each Chinese character of the target corpus and its corresponding similar Chinese character image includes: For each Chinese character, similar Chinese character images of the Chinese character and similar Chinese character images of other Chinese characters in the target corpus are spliced ​​together using image splicing technology to obtain a composite image; Identify the text contained in the synthetic image and obtain new words as fuzzy matching words of the target corpus.

8. The method according to claim 1, wherein: The method further comprises: The target corpus is input into a pre-trained corpus expansion model, so that the fuzzy matching process is performed through the corpus expansion model and the corpus is expanded.

9. A device for expanding a corpus, wherein: The device comprises: A device for acquiring a plurality of target corpora based on the corpus to be expanded; A device for generating, for each target corpus, one or more fuzzy matching words corresponding to the target corpus based on fuzzy matching processing as candidate expanded corpus; A device for expanding the corpus based on candidate expansion corpora corresponding to each target corpus.

10. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

11. A computer readable medium having computer program instructions stored thereon, wherein the computer program instructions can be executed by a processor to implement the method according to any one of claims 1 to 8.

12. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images