An information error correction or deviation correction management system based on an intensive platform

Abstract

The invention discloses an information error correction or deviation correction management system based on an intensive platform, relates to the field of information management, and comprises a management center, wherein the management center is communicatively connected with an information collection module, an information word segmentation module and an information error correction module; the information collection module collects text information in the intensive platform, and the information word segmentation module performs word segmentation processing on the collected text information to obtain a successfully matched text information segment and a failed matched text information segment; the information error correction module obtains a text information sub-segment with abnormally failed matching in the failed matched text information segment according to the result of the word segmentation processing; the abnormally failed matched text information sub-segment is corrected according to the parts of speech of words adjacent to the abnormally failed matched text information sub-segment and the occurrence frequency of the words, thereby improving the accuracy and efficiency of error correction.

Description

Technical Field

[0001] The present invention relates to the field of information management, and in particular to an information error correction or deviation correction management system based on an intensive platform. Background Art

[0002] With the popularization of the Internet and the emergence of big data, people are faced with a large amount of text information, such as news articles, social media posts, emails, etc. However, these information often contain spelling errors, grammatical errors, ambiguity and other problems, which affect the accuracy and readability of the information. The intensive platform provides an efficient data processing and analysis framework, which decomposes large-scale data processing tasks into multiple subtasks, improving processing efficiency and performance.

[0003] How to perform word segmentation on text information to obtain different text information fragments, obtain abnormal matching failure text information sub-segments based on the results of word segmentation, correct the abnormal matching failure text information sub-segments, improve the accuracy and efficiency of error correction, and thus improve the quality and accuracy of the text, is a problem we need to solve. To this end, an information error correction or deviation correction management system based on an intensive platform is now provided. Summary of the invention

[0004] In order to solve the above technical problems, the purpose of the present invention is to provide an information error correction or deviation correction management system based on an intensive platform.

[0005] The object of the present invention can be achieved by the following technical solutions: an information error correction or deviation correction management system based on an intensive platform, comprising a management center, wherein the management center is communicatively connected with an information collection module, an information segmentation module and an information error correction module;

[0006] The information collection module is used to collect text information in the intensive platform;

[0007] The information segmentation module is used to perform segmentation processing on the collected text information to obtain different text information fragments;

[0008] The information error correction module is used to obtain and correct erroneous text information segments in the text information according to the result of word segmentation processing.

[0009] Furthermore, the process of the information collection module collecting text information in the centralized platform includes:

[0010] The information collection module is composed of several information collection terminals. A corresponding communication channel is set for each information collection terminal. The information collection terminal is connected to the intensive platform through the set communication channel. After the connection is completed, the information collection terminal obtains the text information in the intensive platform through the corresponding communication channel, and sends the obtained text information in the intensive platform to the information segmentation module.

[0011] Furthermore, the process of the information segmentation module performing segmentation processing on the collected text information includes:

[0012] Acquire a number of Chinese text information from the Internet, combine the acquired Chinese text information to obtain a corpus, preprocess the corpus and perform part-of-speech tagging on words in the corpus;

[0013] Punctuation marks and special symbols are removed from the collected text information, the text information after the punctuation marks and special symbols are removed is recorded as text information to be segmented, the text information to be segmented is copied, the copied text information to be segmented is recorded as copied text information to be segmented, a matching window is set, and the size of the matching window is set according to actual conditions;

[0014] Performing word segmentation processing on the text information to be segmented from left to right according to the obtained corpus and the set matching window, and performing word segmentation processing on the copied text information to be segmented from right to left according to the obtained corpus and the set matching window;

[0015] Obtain the number of words segmented from left to right for the word segmentation text information to be segmented and the number of words segmented from right to left for the word segmentation of the copied word segmentation text information to be segmented,

[0016] The number of words segmented from left to right is compared with the number of words segmented from right to left, and the final word segmentation result is obtained according to the comparison result.

[0017] Furthermore, the process of preprocessing the corpus includes:

[0018] Remove punctuation marks and special symbols in the corpus, obtain the occurrence frequency of each word in the corpus, set the word occurrence frequency threshold according to the actual situation, and delete the words whose occurrence frequency is lower than the set occurrence frequency threshold.

[0019] Furthermore, the process of part-of-speech tagging of words in the corpus includes:

[0020] Constructing a part-of-speech tagging model based on the dictionary, wherein the part-of-speech tagging model is composed of a plurality of part-of-speech nodes, each of which contains all words of the corresponding part-of-speech in the dictionary;

[0021] The words in the corpus are input into the part-of-speech tagging model, and the words in the corpus are compared with the words in each part-of-speech node. If the comparison results are the same, the part-of-speech tagging model outputs the corresponding part-of-speech, and the words in the corpus are tagged according to the part-of-speech output by the part-of-speech tagging model.

[0022] Furthermore, the process of performing word segmentation processing on the word segmentation text information from left to right includes:

[0023] Starting from the left end of the text information to be segmented, a text information segment of the same length as the matching window size is obtained through the matching window, and the words matching the text information segment are searched through the corpus. If a matching word is found, the text information segment is cut out from the text information to be segmented and recorded as a successfully matched text information segment, and the matching window is moved to the right. If a matching word is not found, the matching window is moved one character to the right to obtain a text information segment of the same length as the matching window size, and the words matching the text information segment are searched through the corpus, and so on, until a matching word is found, the corresponding text information segment is cut out from the text information to be segmented, recorded as a successfully matched text information segment, and the text information segment that failed to match is cut out from the text information to be segmented.

[0024] Furthermore, the process of obtaining the erroneous text information fragment in the text information according to the result of the word segmentation processing by the information error correction module includes:

[0025] Obtaining a text information segment that fails to match, segmenting the text information segment that fails to match once for every character, and obtaining a plurality of segmented text information segments that fail to match, which are recorded as sub-segments of the text information that fails to match;

[0026] If a word in the corpus contains a matching failure text information sub-segment, the matching failure text information sub-segment is recorded as a normal matching failure text information sub-segment; if a word in the corpus does not contain a matching failure text information sub-segment, the matching failure text information sub-segment is recorded as an abnormal matching failure text information sub-segment.

[0027] Furthermore, the process of correcting the obtained erroneous text information fragment includes:

[0028] Combine the normal matching failure text information sub-segment, the abnormal matching failure text information sub-segment, the matching success text information segment, and the removed punctuation marks and special symbols to obtain the original text information;

[0029] Obtaining a text information segment adjacent to a sub-segment of the abnormally failed matching text information in the original text information; if the text information segment adjacent to the sub-segment of the abnormally failed matching text information obtained is a sub-segment of the abnormally failed matching text information, merging the two sub-segments of the abnormally failed matching text information to obtain a text information segment adjacent to the sub-segment of the merged text information; if the text information segment adjacent to the sub-segment of the merged text information obtained is a sub-segment of the abnormally failed matching text information, continuing to merge until the text information segment adjacent to the sub-segment of the merged text information is a successfully matched text information segment or a normally failed matching text information sub-segment;

[0030] If the text information segment adjacent to the merged text information sub-segment is a successfully matched text information segment, then the word corresponding to the successfully matched text information segment in the corpus is obtained; if the text information segment adjacent to the merged text information sub-segment is a normally matched failed text information sub-segment, then the word corresponding to the normally matched failed text information sub-segment in the corpus is obtained, and the number of occurrences and part of speech of the word adjacent to the corresponding word are obtained;

[0031] Obtaining the text information segment adjacent to the abnormal matching failure text information sub-segment in the original text information, as above, until the text information segment adjacent to the merged text information sub-segment is a matching success text information segment or a normal matching failure text information sub-segment;

[0032] If the text information segment adjacent to the merged text information sub-segment is a successfully matched text information segment, then the word corresponding to the successfully matched text information segment in the corpus is obtained; if the text information segment adjacent to the merged text information sub-segment is a normally matched failed text information sub-segment, then the word corresponding to the normally matched failed text information sub-segment in the corpus is obtained, and the number of occurrences and part of speech of the word adjacent to the corresponding word are obtained;

[0033] According to the obtained number of occurrences and parts of speech of the words adjacent to the corresponding word behind and the number of occurrences and parts of speech of the words adjacent to the corresponding word in front, the corresponding word and the abnormal matching failure text information sub-segment are selected for replacement.

[0034] Compared with the prior art, the beneficial effects of the present invention are: collecting text information in an intensive platform, performing word segmentation processing on the collected text information to obtain successfully matched text information segments and unmatched text information segments, obtaining text information sub-segments with abnormally failed matching in the unmatched text information segments according to the results of the word segmentation processing, correcting the abnormally failed matching text information sub-segments according to the parts of speech of adjacent words and the frequency of occurrence of words in the unmatched unmatched text information sub-segments, thereby improving the accuracy and efficiency of error correction, thereby improving the quality and accuracy of the text. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0036] Figure 1 It is a schematic diagram of the present invention. DETAILED DESCRIPTION

[0037] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] like Figure 1 As shown, an information error correction or deviation correction management system based on an intensive platform includes a management center, and the management center is communicatively connected to an information collection module, an information segmentation module, and an information error correction module;

[0039] The information collection module is used to collect text information in the intensive platform;

[0040] It should be further explained that, in the specific implementation process, the process of the information collection module collecting text information in the intensive platform includes:

[0041] The information collection module is composed of several information collection terminals. A corresponding communication channel is set for each information collection terminal. The information collection terminal is connected to the intensive platform through the set communication channel. After the connection is completed, the information collection terminal obtains the text information in the intensive platform through the corresponding communication channel, and sends the obtained text information in the intensive platform to the information segmentation module.

[0042] The information segmentation module is used to perform segmentation processing on the collected text information to obtain different text information fragments;

[0043] It should be further explained that, in the specific implementation process, the process of the information segmentation module performing segmentation processing on the collected text information includes:

[0044] Acquire a number of Chinese text information from the Internet, combine the acquired Chinese text information to obtain a corpus, preprocess the corpus, remove punctuation marks and special symbols in the corpus, obtain the occurrence frequency of each word in the corpus, set a word occurrence frequency threshold according to actual conditions, and delete words whose occurrence frequency is lower than the set occurrence frequency threshold;

[0045] Perform part-of-speech tagging on the words in the corpus, including nouns, verbs, adjectives, numerals, quantifiers, pronouns, adverbs, prepositions, conjunctions, auxiliary words, interjections, onomatopoeia, etc.;

[0046] A part-of-speech tagging model is constructed according to the dictionary, and the words in the corpus are used as the input of the part-of-speech tagging model, and the parts of speech of the words in the corpus are used as the output of the part-of-speech tagging model, wherein the part-of-speech tagging model is composed of noun nodes, verb nodes, adjective nodes, numeral nodes, quantifier nodes, pronoun nodes, adverb nodes, preposition nodes, conjunction nodes, particle nodes, interjection nodes and onomatopoeia nodes, wherein the noun nodes include all noun words in the dictionary, the verb nodes include all verb words in the dictionary, the adjective nodes include all adjective words in the dictionary, the numeral nodes include all numeral words in the dictionary, the quantifier nodes include all quantifier words in the dictionary, the pronoun nodes include all pronoun words in the dictionary, the adverb nodes include all adverb words in the dictionary, the preposition nodes include all preposition words in the dictionary, the conjunction nodes include all conjunction words in the dictionary, the particle nodes include all particle words in the dictionary, the interjection nodes include all interjection words in the dictionary, and the onomatopoeia nodes include all onomatopoeia words in the dictionary;

[0047] Input the words in the corpus to the part-of-speech tagging model, the words in the corpus reach the noun node, compare the words in the corpus with the words in the noun node, if there is a word in the noun node that is the same as the word in the corpus, the part-of-speech tagging model outputs a noun, and marks the word in the corpus as a noun, if there is no word in the noun node that is the same as the word in the corpus, then transfer the word in the corpus to the verb node, compare the words in the corpus with the words in the verb node, if there is a word in the verb node that is the same as the word in the corpus, then the part-of-speech tagging model outputs a verb, and marks the word in the corpus as a verb, if there is no word in the verb node that is the same as the word in the corpus, then transfer the word in the corpus to the adjective node, and so on, until the part-of-speech tagging model outputs;

[0048] Regularly update and maintain the corpus to reflect the latest word usage;

[0049] It should be further explained that, in the specific implementation process, words with a frequency lower than the set frequency threshold are not representative of the corpus;

[0050] Preprocessing the collected text information, removing punctuation marks and special symbols in the collected text information, recording the text information after removing the punctuation marks and special symbols as text information to be segmented, copying the text information to be segmented, recording the copied text information to be segmented as copied text information to be segmented, setting a matching window, and setting the size of the matching window according to actual conditions, wherein the size of the matching window is used to limit the maximum length of each match;

[0051] The text information to be segmented is segmented from left to right, starting from the left end of the text information to be segmented, using the matching window to obtain a text information segment of the same length as the matching window size, searching the longest word matching the text information segment through the corpus, if a matching word is found, the text information segment is cut out from the text information to be segmented, and recorded as a successfully matched text information segment, the matching window is moved to the right, if no matching word is found, the matching window is moved to the right by one character, a text information segment of the same length as the matching window size is obtained, searching the longest word matching the text information segment through the corpus, and so on, until a matching word is found, the corresponding text information segment is cut out from the text information to be segmented, recorded as a successfully matched text information segment, and the text information segment that fails to match is cut out from the text information to be segmented;

[0052] Perform word segmentation processing on the copied text information to be segmented from right to left, starting from the right end of the copied text information to be segmented, using the matching window to obtain a text information segment of the same length as the matching window size, searching the longest word matching the text information segment through the corpus, if a matching word is found, the text information segment is cut out from the copied text information to be segmented, and recorded as a successfully matched text information segment, and the matching window is moved to the left, if no matching word is found, the matching window is moved to the left by one character to obtain a text information segment of the same length as the matching window size, searching the longest word matching the text information segment through the corpus, and so on, until a matching word is found, the corresponding text information segment is cut out from the copied text information to be segmented, recorded as a successfully matched text information segment, and the text information segment that fails to match is cut out from the copied text information to be segmented;

[0053] The number of words segmented from left to right for the word segmentation processing of the text information to be segmented is obtained, and the number of words segmented from right to left for the copied text information to be segmented is obtained. If the number of words segmented from left to right for the word segmentation processing of the text information to be segmented is greater than the number of words segmented from right to left for the copied text information to be segmented, the result of the word segmentation processing of the copied text information to be segmented from right to left is used as the result of the final word segmentation processing. If the number of words segmented from left to right for the word segmentation processing of the text information to be segmented is less than or equal to the number of words segmented from right to left for the copied text information to be segmented, the result of the word segmentation processing of the text information to be segmented from left to right is used as the result of the final word segmentation processing.

[0054] The information error correction module is used to obtain and correct erroneous text information segments in the text information according to the result of word segmentation processing;

[0055] It should be further explained that, in a specific implementation process, the process of the information error correction module obtaining the erroneous text information fragment in the text information according to the result of the word segmentation processing includes:

[0056] Obtaining a text information segment that fails to match, segmenting the obtained text information segment that fails to match, segmenting the text information segment that fails to match once every other character, obtaining a plurality of segmented text information segments that fail to match, recording the obtained segmented text information segment that fails to match as a sub-segment of failed matching text information, comparing the sub-segment of failed matching text information with words in a corpus, if the words in the corpus contain the sub-segment of failed matching text information, recording the sub-segment of failed matching text information as a normal sub-segment of failed matching text information, if the words in the corpus do not contain the sub-segment of failed matching text information, recording the sub-segment of failed matching text information as an abnormal sub-segment of failed matching text information, wherein the abnormal sub-segment of failed matching text information is an erroneous sub-segment of failed matching text information;

[0057] It should be further explained that, in the specific implementation process, the process of correcting the obtained erroneous text information fragment includes:

[0058] Combine the normal matching failure text information sub-segment, the abnormal matching failure text information sub-segment, the matching success text information segment, and the removed punctuation marks and special symbols to obtain the original text information;

[0059] Obtaining a text information segment adjacent to a sub-segment of the abnormally failed matching text information in the original text information; if the text information segment adjacent to the sub-segment of the abnormally failed matching text information obtained is a sub-segment of the abnormally failed matching text information, merging the two sub-segments of the abnormally failed matching text information to obtain a text information segment adjacent to the sub-segment of the merged text information; if the text information segment adjacent to the sub-segment of the merged text information obtained is a sub-segment of the abnormally failed matching text information, continuing to merge until the text information segment adjacent to the sub-segment of the merged text information is a successfully matched text information segment or a normally failed matching text information sub-segment;

[0060] If the text information segment adjacent to the merged text information sub-segment is a successfully matched text information segment, then the word corresponding to the successfully matched text information segment in the corpus is obtained; if the text information segment adjacent to the merged text information sub-segment is a normally matched failed text information sub-segment, then the word corresponding to the normally matched failed text information sub-segment in the corpus is obtained, and the number of occurrences and part of speech of the words adjacent to the back of the corresponding words are obtained, and the words adjacent to the back are sorted from high to low according to the number of occurrences of the words adjacent to the back, and are recorded as the first back word, the second back word, ... the nth back word respectively;

[0061] Obtaining a text information segment adjacent to the abnormal matching failure text information sub-segment in the original text information; if the obtained text information segment adjacent to the abnormal matching failure text information sub-segment is an abnormal matching failure text information sub-segment, merging the two abnormal matching failure text information sub-segments to obtain a text information segment adjacent to the merged text information sub-segment; if the obtained text information segment adjacent to the merged text information sub-segment is an abnormal matching failure text information sub-segment, continuing to merge until the text information segment adjacent to the merged text information sub-segment is a successfully matched text information segment or a normally matched failure text information sub-segment;

[0062] If the text information segment adjacent to the merged text information sub-segment is a successfully matched text information segment, then the word corresponding to the successfully matched text information segment in the corpus is obtained; if the text information segment adjacent to the merged text information sub-segment is a normally matched failed text information sub-segment, then the word corresponding to the normally matched failed text information sub-segment in the corpus is obtained, and the number of occurrences and part of speech of the word adjacent to the corresponding word are obtained, and the words adjacent to the words in front of the words are sorted from high to low according to the number of occurrences of the words adjacent to the words in front, and are recorded as the first front word, the second front word, ... the nth front word respectively;

[0063] If the first rear word is the same as the first front word, and the part of speech of the first rear word is different from the part of speech of the corresponding words in the corpus of the text information segments in front and behind the abnormal matching failure text information sub-segment, then the first rear word is recorded as a smooth state, otherwise it is recorded as an incoherent state, and the first rear word is replaced with the merged text information sub-segment;

[0064] If the first rear word and the first front word are different, the word that appears more times and is in a fluent state is replaced with the merged text information sub-segment; if both the first rear word and the first front word are not in a fluent state, the status of the second rear word and the second front word is obtained, and the word that appears more times and is in a fluent state is replaced with the merged text information sub-segment; if both the second rear word and the second front word are not in a fluent state, the status of the third rear word and the third front word is obtained, and so on, until the replacement is completed.

[0065] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. An information error correction or deviation correction management system based on an intensive platform, including a management center, characterized in that: The management center is communicatively connected to an information collection module, an information segmentation module, and an information error correction module; The information collection module is used to collect text information in the intensive platform; the information word segmentation module is used to perform word segmentation processing on the collected text information to obtain different text information fragments; the information error correction module is used to obtain erroneous text information fragments in the text information according to the result of the word segmentation processing and correct them; The process of the information segmentation module performing segmentation processing on the collected text information includes: Acquire a number of Chinese text information from the Internet, combine the acquired Chinese text information to obtain a corpus, preprocess the corpus and perform part-of-speech tagging on words in the corpus; Punctuation marks and special symbols are removed from the collected text information, and the text information after the punctuation marks and special symbols are removed is recorded as text information to be segmented, and the text information to be segmented is copied, and the copied text information to be segmented is recorded as copied text information to be segmented, and a matching window is set; the text information to be segmented is segmented from left to right according to the obtained corpus and the set matching window, and the copied text information to be segmented is segmented from right to left according to the obtained corpus and the set matching window; Obtain the number of words segmented from left to right by performing word segmentation processing on the text information to be segmented and the number of words segmented from right to left by performing word segmentation processing on the copied text information to be segmented, compare the number of words segmented from left to right with the number of words segmented from right to left, and obtain the final word segmentation processing result according to the comparison result; The process of obtaining the erroneous text information fragment in the text information according to the result of word segmentation processing by the information error correction module includes: Obtaining a text information segment that fails to match, segmenting the text information segment that fails to match once every character, and obtaining a number of segmented text information segments that fail to match, which are recorded as failed matching text information sub-segments; if a word in the corpus contains a failed matching text information sub-segment, the failed matching text information sub-segment is recorded as a normal failed matching text information sub-segment; if a word in the corpus does not contain a failed matching text information sub-segment, the failed matching text information sub-segment is recorded as an abnormal failed matching text information sub-segment; The process of correcting the obtained erroneous text information fragment includes: Combine the normal matching failure text information sub-segment, the abnormal matching failure text information sub-segment, the matching success text information segment, and the removed punctuation marks and special symbols to obtain the original text information; Obtaining a text information segment adjacent to a sub-segment of the abnormally failed matching text information in the original text information; if the text information segment adjacent to the sub-segment of the abnormally failed matching text information obtained is a sub-segment of the abnormally failed matching text information, merging the two sub-segments of the abnormally failed matching text information to obtain a text information segment adjacent to the sub-segment of the merged text information; if the text information segment adjacent to the sub-segment of the merged text information obtained is a sub-segment of the abnormally failed matching text information, continuing to merge until the text information segment adjacent to the sub-segment of the merged text information is a successfully matched text information segment or a normally failed matching text information sub-segment; If the text information segment adjacent to the merged text information sub-segment is a successfully matched text information segment, then the word corresponding to the successfully matched text information segment in the corpus is obtained; if the text information segment adjacent to the merged text information sub-segment is a normally matched failed text information sub-segment, then the word corresponding to the normally matched failed text information sub-segment in the corpus is obtained, and the number of occurrences and part of speech of the word adjacent to the corresponding word are obtained; Obtaining a text information segment adjacent to the abnormal matching failure text information sub-segment in the original text information; if the obtained text information segment adjacent to the abnormal matching failure text information sub-segment is an abnormal matching failure text information sub-segment, merging the two abnormal matching failure text information sub-segments to obtain a text information segment adjacent to the merged text information sub-segment; if the obtained text information segment adjacent to the merged text information sub-segment is an abnormal matching failure text information sub-segment, continuing to merge until the text information segment adjacent to the merged text information sub-segment is a successfully matched text information segment or a normally matched failure text information sub-segment; If the text information segment adjacent to the merged text information sub-segment is a successfully matched text information segment, then the word corresponding to the successfully matched text information segment in the corpus is obtained; if the text information segment adjacent to the merged text information sub-segment is a normally matched failed text information sub-segment, then the word corresponding to the normally matched failed text information sub-segment in the corpus is obtained, and the number of occurrences and part of speech of the word adjacent to the corresponding word are obtained; According to the obtained number of occurrences and parts of speech of the words adjacent to the corresponding word behind and the number of occurrences and parts of speech of the words adjacent to the corresponding word in front, the corresponding word and the abnormal matching failure text information sub-segment are selected for replacement.

2. According to claim 1, an information error correction or deviation correction management system based on an intensive platform is characterized in that: The process of the information collection module collecting text information in the intensive platform includes: The information collection module is composed of several information collection terminals. A corresponding communication channel is set for each information collection terminal. The information collection terminal is connected to the intensive platform through the set communication channel. After the connection is completed, the information collection terminal obtains the text information in the intensive platform through the corresponding communication channel, and sends the obtained text information in the intensive platform to the information segmentation module.

3. The information error correction or deviation correction management system based on an intensive platform according to claim 2 is characterized in that: The process of preprocessing the corpus includes: Remove punctuation marks and special symbols in the corpus, obtain the occurrence frequency of each word in the corpus, set the word occurrence frequency threshold according to the actual situation, and delete the words whose occurrence frequency is lower than the set occurrence frequency threshold.

4. The information error correction or deviation correction management system based on an intensive platform according to claim 2 is characterized in that: The process of part-of-speech tagging of words in a corpus includes: Constructing a part-of-speech tagging model based on the dictionary, wherein the part-of-speech tagging model is composed of a plurality of part-of-speech nodes, each of which contains all words of the corresponding part-of-speech in the dictionary; The words in the corpus are input into the part-of-speech tagging model, and the words in the corpus are compared with the words in each part-of-speech node. If the comparison results are the same, the part-of-speech tagging model outputs the corresponding part-of-speech, and the words in the corpus are tagged according to the part-of-speech output by the part-of-speech tagging model.

5. The information error correction or deviation correction management system based on an intensive platform according to claim 2 is characterized in that: The process of word segmentation from left to right includes: Starting from the left end of the text information to be segmented, a text information segment of the same length as the matching window size is obtained through the matching window, and the words matching the text information segment are searched through the corpus. If a matching word is found, the text information segment is cut out from the text information to be segmented and recorded as a successfully matched text information segment, and the matching window is moved to the right. If a matching word is not found, the matching window is moved one character to the right to obtain a text information segment of the same length as the matching window size, and the words matching the text information segment are searched through the corpus, and so on, until a matching word is found, the corresponding text information segment is cut out from the text information to be segmented, recorded as a successfully matched text information segment, and the text information segment that failed to match is cut out from the text information to be segmented.

Images