Method and equipment for correcting regional noun error

Abstract

The invention aims to provide a method and equipment for correcting regional noun errors. Compared with the prior art, the method and the device have the advantages that the preset number of correct corpora containing the regional nouns are constructed, the regional nouns in each corpus are subjected to erroneous character construction based on the preset erroneous character construction rule, the constructed erroneous character corpus is determined, model training is performed based on the correct corpora and the erroneous character corpus, and the trained error correction model is determined; wherein the error correction model comprises region noun recognition and region noun error correction, and then region noun error correction is carried out on sentences containing region noun based on the trained error correction model. In this way, the effect of correcting regional nouns can be improved, and the error correction accuracy can be improved by assisting the text error correction function through the entity recognition function.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a technique for correcting regional terms. Background Technology

[0002] Existing technologies for region-specific word correction rely on lexicons and rule-based syntax, requiring the manual construction of a massive error set. However, this set lacks generalization; words not in the error set cannot be corrected, and it may also generate strange false alarms, such as mismatches where the text match does not produce a complete word. Therefore, maintaining the lexicon manually is costly. This rule-based approach itself has significant limitations. Summary of the Invention

[0003] One objective of this application is to provide a method and apparatus for correcting regional terminology errors.

[0004] According to one aspect of this application, a method for correcting regional terminology is provided, wherein the method includes: Construct a pre-defined corpus containing a set number of correct words and phrases related to geographic regions; Based on the preset misspelling construction rules, misspellings are constructed for regional terms in each corpus to determine the constructed misspelling corpus; The model is trained based on the correct corpus and the misspelled corpus to determine the trained error correction model, wherein the error correction model includes regional term recognition and regional term error correction; The trained error correction model is used to correct the location-related terms in sentences.

[0005] Optionally, the preset misspelling construction rules include at least one of the following: Replace any one character in the region name; Replace words with the same pinyin in regional terms; Replace words with the first pinyin in regional terms; There are multiple errors in the construction of regional terms.

[0006] Optionally, replacing a single character in the regional terminology includes at least one of the following: Replace with a character that begins with the same pinyin; Replace with a character that has the same pinyin; Replace with near-homophones; Replace with similar-looking characters; Replace with any character.

[0007] Optionally, the multi-word construction error in the regional terminology includes at least one of the following: Repeat a word; Repeat a word; Add a word; Add a word with the same prefix; Add a word with the same last suffix.

[0008] Optionally, the step of performing location-based word correction on sentences containing location-based terms based on the trained error correction model includes: Input the sentence containing the region name into the trained error correction model; The trained error correction model outputs the region name and word recognition results and the region name and word error correction results.

[0009] According to another aspect of this application, a device for correcting regional terminology is also provided, the device comprising: Corpus construction unit, used to construct a preset number of correct corpora containing geographic terms; The misspelling construction unit is used to construct misspellings for regional terms in each corpus based on preset misspelling construction rules, and to determine the constructed misspelling corpus. An error correction training unit is used to train a model based on the correct corpus and the misspelled corpus, and to determine the trained error correction model, wherein the error correction model includes regional term recognition and regional term error correction; The error correction execution unit is used to perform location-specific word correction on sentences containing location-specific words based on the trained error correction model.

[0010] According to another aspect of this application, a computer-readable medium is also provided, on which computer-readable instructions are stored, which can be executed by a processor to perform the operations as described above.

[0011] According to another aspect of this application, a device for correcting regional terminology is also provided, wherein the device includes: One or more processors; and A memory storing computer-readable instructions, which, when executed, cause the processor to perform operations as described above.

[0012] Compared with existing technologies, this application constructs a preset number of correct corpora containing regional terms, constructs misspellings for regional terms in each corpus based on preset misspelling rules, determines the constructed misspelled corpus, and trains a model based on the correct and misspelled corpora to determine the trained error correction model. The error correction model includes regional term recognition and regional term correction. Then, based on the trained error correction model, sentences containing regional terms are corrected. This approach improves the effectiveness of regional term correction, and the accuracy of error correction is enhanced by using entity recognition to assist the text correction function. Attached Figure Description

[0013] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This diagram illustrates a method for correcting regional terminology according to one aspect of this application. Figure 2 This diagram illustrates the output of an error correction based on a trained error correction model according to this application. Figure 3 A schematic diagram of a device for correcting regional terminology is shown according to another aspect of this application.

[0014] The same or similar reference numerals in the accompanying drawings represent the same or similar parts. Detailed Implementation

[0015] The present application will now be described in further detail with reference to the accompanying drawings.

[0016] In a typical configuration of this application, the terminal, the device of the service network, and the trusted party all include one or more processors (e.g., a central processing unit (CPU)), input / output interfaces, network interfaces, and memory.

[0017] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0018] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change RAM (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device. As defined in this article, computer-readable media do not include non-transitory computer-readable media, such as modulated data signals and carrier waves.

[0019] Figure 1 This application provides a flowchart of a method for correcting regional terminology errors, the method comprising the following steps: S11 constructs a preset number of correct corpora containing regional terms; S12 constructs misspelled words based on preset misspelling rules, and determines the constructed misspelled word corpus by constructing misspelled words for regional terms in each corpus. S13. Based on the correct corpus and the misspelled corpus, the model is trained to determine the trained error correction model, wherein the error correction model includes regional term recognition and regional term error correction; S14 performs location-specific word correction on sentences containing location-specific words based on the trained error correction model.

[0020] In this embodiment, in step S11, a preset number of correct corpora containing place names are constructed. Specifically, a complete list of Chinese provinces, cities, and counties is obtained based on publicly available geographical data. During the construction of the corpus, sentences containing any place name are obtained through a preset program.

[0021] Continuing in this embodiment, in the step S12, based on a preset misspelling construction rule, misspellings are constructed for geographical nouns in each corpus to determine the misspelled corpus after construction. Here, the misspelled corpus after construction is a sentence containing misspellings for geographical nouns. Preferably, the misspelling construction can be carried out according to the sentence length. For example, if the sentence length is less than 6 words, only one error is constructed for the sentence; otherwise, 2 errors are constructed for the sentence.

[0022] Preferably, the preset misspelling construction rule includes but is not limited to at least any one of the following: randomly replacing one character in the geographical noun; replacing with a homophonic word; replacing with a word with the same initial pinyin; constructing multi-character errors in the geographical noun.

[0023] Preferably, randomly replacing one character in the geographical noun includes but is not limited to at least any one of the following: replacing with a character starting with the same pinyin; replacing with a homophonic character; replacing with a near-homophone; replacing with a character with similar form; replacing with any character. Specifically, the construction methods can be exemplified as follows: (1)Characters starting with the same pinyin. For example, the character '上' in '上海' starts with the pinyin's', and the character '尚' also starts with the pinyin's'. The character '上' can be modified to '尚'.

[0024] (2)Homophonic characters. Here, in addition to exactly the same pinyin, homophonic characters also include easily confused pinyin. For example, '来' and '乃' are classified here because the 'l' and 'n' in pinyin are easily confused pronunciations; '上' and '三' are also classified here because the retroflex / dental consonants and the nasalized / non-nasalized vowels are also easily confused pronunciations. For example, the homophonic character of '上' is '三', and the character '上' can be modified to '三'.

[0025] (3)Near-homophones. Here, near-homophones include various similar possibilities: such as the same initial consonant, the same final consonant, easily confused, etc. For example, the near-homophone of '上' is '双', and the character '上' can be modified to '双'.

[0026] (4)Characters with similar form. According to the form code part of the phonetic shape code, characters with relatively high similarity are selected to form characters with similar form. For example, the character with similar form of '上' is '止', and the character '上' can be modified to '止'.

[0027] (5)Randomly select a character. It can be randomly changed to any character in Chinese.

[0028] In addition, homophonic words can also be replaced, that is, the geographical word in the sentence is replaced with a homophonic word. For example, '北京' and '背景' are homophonic words, and '北京' can be replaced with '背景'.

[0029] Furthermore, words with the same initial pinyin can also be replaced. That is, the geographical word in the sentence is replaced with a word with the same initial pinyin. For example, the initial pinyin of '上海' is'sh', and the words starting with'sh' include: '时候'. '上海' can be replaced with '时候'.

[0030] Preferably, among them, the structural multi-character errors in the geographical nouns include but are not limited to at least any one of the following: repeating a character; repeating a word; adding a character; adding a word with the same initial character; adding a word with the same final character. Specific examples are as follows: (1) Randomly repeating a character. For example, repeating the character "shang" in "Shanghai City", the structural error is: "shang Shanghai City".

[0031] (2) Randomly repeating a word. For example, repeating the word "Shanghai" currently, the structural error is: "Shanghai Shanghai".

[0032] (3) Randomly adding a character in a word. For example, for "Shanghai City", the structural error is: "shang Shanghai City".

[0033] (4) Randomly adding a word with the same initial character in a word. For example, for "Shanghai City", randomly find a character in a geographical word, such as the character "shang", and words starting with the same character "shang" are: "shangtai", the structural error can be: "shangtai Shanghai City".

[0034] (5) Randomly adding a word with the same final character in a word. For example, for "Shanghai City", randomly find a character in a geographical word, such as the character "city", and words ending with the same character "city" are: "city", the structural error can be: "Shanghai city". Here, the above-mentioned ways of misspelling construction are only examples, and other existing ways applicable to this application should also be included in the protection scope of this application.

[0035] Continuing in this embodiment, in the step S13, based on the correct corpus and the misspelled character corpus, model training is performed to determine a trained error correction model, where the error correction model includes geographical noun recognition and geographical noun error correction.

[0036] Preferably, the model training can use a Transformer encoder, which is composed of multiple identical layers stacked, and each layer has two sub-layers. The first is a multi-head self-attention aggregation, and the second sub-layer is a position-based feed-forward neural network. Each sub-layer uses a residual link.

[0037] During training, in addition to the basic error correction training task, this solution also adds a task of recognizing geographical nouns in a sentence.

[0038] Continuing in this embodiment, in the step S14, based on the trained error correction model, geographical noun error correction is performed on a sentence containing geographical nouns.

[0039] Such as Figure 2The schematic diagram of the output result of error correction based on the trained error correction model is shown. Among them, in this solution, the grammatical category of each word in a given sentence is determined, and the process of determining its word nature and marking it is carried out. That is, after the sentence is segmented, the nature of the word such as noun (n), verb (v), etc. is marked after the segmented words in the sentence. Among them, in this embodiment, the Model adopts a Transformer encoder.

[0040] Specifically, input the sentence to be corrected, and obtain the word nature marking result and the error correction result. As Figure 2 shown, when the word nature of a word is LOC (corresponding to a geographical noun), correct the words with error correction actions (referring to those whose error correction label is not KEEP), so as to obtain the correct sentence after error correction and return it. Among them, keep refers to the words that do not need error correction. In this embodiment, the seventh word "尚" is corrected to "上".

[0041] Compared with the prior art, in this application, a preset number of correct corpora containing geographical nouns are constructed, based on the preset misspelling construction rules, misspellings are constructed for the geographical nouns in each corpus to determine the misspelled corpora after construction, and model training is carried out based on the correct corpus and the misspelled corpora to determine the trained error correction model. Among them, the error correction model includes geographical noun recognition and geographical noun error correction, and then the trained error correction model is used to perform geographical noun error correction on the sentence containing geographical nouns. This method can improve the effect of geographical noun error correction, and the accuracy of error correction can be improved by assisting the text error correction function through the entity recognition function.

[0042] Figure 3 The schematic diagram of a device for geographical noun error correction according to another aspect of this application is shown. The device 1 includes: A corpus construction unit 11 for constructing a preset number of correct corpora containing geographical nouns; A misspelling construction unit 12 for constructing misspellings for the geographical nouns in each corpus based on the preset misspelling construction rules to determine the misspelled corpora after construction; An error correction training unit 13 for performing model training based on the correct corpus and the misspelled corpora to determine the trained error correction model. Among them, the error correction model includes geographical noun recognition and geographical noun error correction; An error correction execution unit 14 for performing geographical noun error correction on the sentence containing geographical nouns based on the trained error correction model.

[0043] According to another aspect of this application, a computer-readable medium is further provided. The computer-readable medium stores computer-readable instructions, and the computer-readable instructions can be executed by a processor to implement the foregoing method.

[0044] According to another aspect of this application, a device for correcting regional terminology is also provided, wherein the device includes: One or more processors; and A memory storing computer-readable instructions, which, when executed, cause the processor to perform operations as described above.

[0045] For example, when executed, computer-readable instructions cause one or more processors to: construct a preset number of correct corpora containing regional terms; construct misspellings for regional terms in each corpus based on preset misspelling rules, and determine the constructed misspelled corpus; train a model based on the correct corpus and the misspelled corpus to determine a trained error correction model, wherein the error correction model includes regional term recognition and regional term correction; and perform regional term correction on sentences containing regional terms based on the trained error correction model.

[0046] It will be apparent to those skilled in the art that this application is not limited to the details of the exemplary embodiments described above, and that it can be implemented in other specific forms without departing from the spirit or essential characteristics of this application. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of this application is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within this application. No reference numerals in the claims should be construed as limiting the scope of the claims. Furthermore, it is clear that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. Terms such as "first," "second," etc., are used to denote names and do not indicate any particular order.

Claims

1. A method for correcting errors in geographical terminology, wherein, The method includes: Construct a pre-defined corpus containing a set number of correct words and phrases related to geographic regions; Based on the preset misspelling construction rules, misspellings are constructed for regional terms in each corpus to determine the constructed misspelling corpus; The model is trained based on the correct corpus and the misspelled corpus to determine the trained error correction model, wherein the error correction model includes regional term recognition and regional term error correction; The trained error correction model is used to correct the location-related terms in sentences.

2. The method according to claim 1, wherein, The preset misspelling construction rules include at least one of the following: Replace any one character in the region name; Replace words with the same pinyin in regional terms; Replace words with the first pinyin in regional terms; There are multiple errors in the construction of regional terms.

3. The method according to claim 2, wherein, Replacing any one character in the aforementioned regional terminology includes at least one of the following: Replace with a character that begins with the same pinyin; Replace with a character that has the same pinyin; Replace with near-homophones; Replace with similar-looking characters; Replace with any character.

4. The method according to claim 2, wherein, The terminology containing multiple words in the aforementioned domain name includes at least one of the following: Repeat a word; Repeat a word; Add a word; Add a word with the same prefix; Add a word with the same ending.

5. The method according to any one of claims 1 to 4, wherein, The step of performing location-based word correction on sentences containing location-based terms based on the trained error correction model includes: Input the sentence containing the region name into the trained error correction model; The trained error correction model outputs the region name and word recognition results and the region name and word error correction results.

6. A device for correcting errors in geographical terminology, the device comprising: Corpus construction unit, used to construct a preset number of correct corpora containing geographic terms; The misspelling construction unit is used to construct misspellings for regional terms in each corpus based on preset misspelling construction rules, and to determine the constructed misspelling corpus. An error correction training unit is used to train a model based on the correct corpus and the misspelled corpus, and to determine the trained error correction model, wherein the error correction model includes regional term recognition and regional term error correction; The error correction execution unit is used to perform error correction on sentences containing regional terms based on the trained error correction model.

7. A computer-readable medium having stored thereon computer-readable instructions that can be executed by a processor to implement the method as described in any one of claims 1 to 5.

8. A device for correcting errors in geographical terminology, wherein, The device includes: One or more processors; and A memory storing computer-readable instructions, which, when executed, cause the processor to perform the operations of the method as described in any one of claims 1 to 5.

Images