Information error correction method, device, computer equipment and storage medium
By combining word segmentation dictionaries and similar-looking dictionaries with grammatical structure and semantic analysis, abnormal information in OCR recognition is identified and corrected, solving the problems of low recognition accuracy and error correction efficiency in traditional methods, and achieving efficient and accurate information error correction.
Patent Information
- Application Number
- CN202311668330.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-07
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2043-12-07
AI Technical Summary
Existing OCR recognition technology has errors in recognizing paper documents when converting them into electronic documents. Traditional error correction methods judge whether the information is abnormal at the character composition level, resulting in low recognition accuracy and low error correction efficiency.
Word segmentation is performed using a word segmentation dictionary. By combining character-level and word-level analysis of grammatical structure and semantic information, the text information to be corrected is identified. Similar text information is then replaced in a similarity dictionary. The degree of confusion is calculated using a language model, and the text information with the lowest degree of confusion is selected as the correction result.
It improves the accuracy of abnormal information identification and the efficiency of error correction, ensuring the reliability and accuracy of error correction results.
Smart Images

Figure CN117371445B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image recognition technology, and in particular to an information error correction method, apparatus, computer equipment, and storage medium. Background Art
[0002] With the development of image recognition technology, OCR recognition technology has emerged, which is used to convert paper documents into electronic documents. However, since the current OCR recognition technology still produces incorrect recognition results, it is necessary to correct the recognition results.
[0003] The traditional method mainly regards the scattered string characters after word segmentation as abnormal information and performs error correction on the abnormal information.
[0004] However, this traditional method only judges whether the information is abnormal at the level of character composition, and is prone to misjudging normal information as abnormal information, thereby reducing the accuracy of identifying abnormal information, and further reducing the efficiency and accuracy of error correction for abnormal information. Summary of the Invention
[0005] Based on this, it is necessary to provide an information error correction method, device, computer equipment, computer-readable storage medium and computer program product that can efficiently and accurately identify and correct abnormal information in response to the above technical problems.
[0006] In a first aspect, the present application provides an information error correction method, comprising:
[0007] Acquire a target image and identify initial text information corresponding to the target image;
[0008] The initial text information is segmented using a segmentation dictionary to obtain a plurality of segmentation information corresponding to the initial text information, the grammatical structure and semantic information corresponding to each segmentation information are analyzed according to the character granularity and the word granularity, and each segmentation information is judged, and if it is judged that the current segmentation information does not meet the preset conditions, the current segmentation information is used as the text information to be corrected; wherein the character granularity represents the segmentation unit at the character level, and the word granularity represents the segmentation unit at the word level;
[0009] Matching the text information to be corrected with a plurality of corresponding similar text information in a similarity dictionary, replacing each similar text information with the text information to be corrected in the initial text information in sequence, and using the replaced initial text information as the candidate text information;
[0010] The degree of confusion corresponding to each candidate text information is calculated, and the candidate text information with the smallest degree of confusion is used as the corrected text information.
[0011] In one embodiment, the acquiring of the target image and the identification of the initial text information corresponding to the target image include: matching the book title marks in the initial text information and the text information within the book title marks according to the left and right boundaries of the book title marks based on regular expressions, and inputting the text information within the book title marks into the word segmentation dictionary.
[0012] In one embodiment, the analyzing the grammatical structure and semantic information corresponding to each word segmentation information according to character granularity and word granularity includes: analyzing the grammatical structure and semantic information corresponding to each word segmentation information based on the standard rules for grammatical structure and semantic information carried by the word segmentation dictionary; wherein the standard rules include part-of-speech rules, phrase collocation rules, polysemy rules, proper noun recognition rules and grammatical component rules.
[0013] In one embodiment, if it is determined that the current word segmentation information does not meet the preset conditions, the current word segmentation information is used as text information to be corrected, including: if the current word segmentation information is a single character, the current word segmentation information is analyzed based on the standard rules; if the current word segmentation information does not meet the standard rules, the current word segmentation information is used as text information to be corrected.
[0014] In one embodiment, the text information to be corrected is matched with corresponding multiple similar text information in a similar dictionary, and each similar text information is replaced with the text information to be corrected in the initial text information in sequence, including: if there are at least two text information to be corrected, each text information to be corrected is matched with corresponding multiple similar text information in a similar dictionary, the similar text information corresponding to different text information to be corrected are combined with each other to obtain multiple similar text combination information, and the multiple similar text combination information are replaced with the corresponding text information to be corrected in the initial text information in sequence.
[0015] In one embodiment, the calculation of the degree of confusion corresponding to each candidate text information includes: inputting each candidate text information into a trained model, and the model obtains the degree of confusion of the corresponding candidate text information based on the conditional probability of each character in each candidate text information in the corresponding candidate text information; wherein the conditional probability refers to the probability of the next word appearing given the current word.
[0016] In a second aspect, the present application further provides an information error correction device, comprising:
[0017] An acquisition module, configured to acquire a target image and identify initial text information corresponding to the target image;
[0018] A word segmentation module is configured to perform word segmentation processing on the initial text information through a word segmentation dictionary to obtain a plurality of word segmentation information corresponding to the initial text information, analyze the grammatical structure and semantic information corresponding to each word segmentation information according to the character granularity and the word granularity, and judge each word segmentation information. If it is judged that the current word segmentation information does not meet the preset conditions, the current word segmentation information is used as the text information to be corrected; wherein the character granularity represents the word segmentation unit at the character level, and the word granularity represents the word segmentation unit at the word level;
[0019] a matching module configured to match the text information to be corrected with corresponding multiple similar text information in a similarity dictionary, replace each similar text information with the text information to be corrected in the initial text information in sequence, and use the replaced initial text information as candidate text information;
[0020] The calculation module is used to calculate the degree of confusion corresponding to each candidate text information, and take the candidate text information with the smallest degree of confusion as the corrected text information.
[0021] In one embodiment, the matching module is also used to: if there are at least two text information to be corrected, each text information to be corrected is matched with a corresponding plurality of similar text information in the similarity dictionary, the similar text information corresponding to different text information to be corrected are combined with each other to obtain a plurality of similar text combination information, and the plurality of similar text combination information are sequentially replaced with the corresponding text information to be corrected in the initial text information.
[0022] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:
[0023] Acquire a target image and identify initial text information corresponding to the target image;
[0024] The initial text information is segmented using a segmentation dictionary to obtain a plurality of segmentation information corresponding to the initial text information, the grammatical structure and semantic information corresponding to each segmentation information are analyzed according to the character granularity and the word granularity, and each segmentation information is judged, and if it is judged that the current segmentation information does not meet the preset conditions, the current segmentation information is used as the text information to be corrected; wherein the character granularity represents the segmentation unit at the character level, and the word granularity represents the segmentation unit at the word level;
[0025] Matching the text information to be corrected with a plurality of corresponding similar text information in a similarity dictionary, replacing each similar text information with the text information to be corrected in the initial text information in sequence, and using the replaced initial text information as the candidate text information;
[0026] The degree of confusion corresponding to each candidate text information is calculated, and the candidate text information with the smallest degree of confusion is used as the corrected text information.
[0027] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:
[0028] Acquire a target image and identify initial text information corresponding to the target image;
[0029] The initial text information is segmented using a segmentation dictionary to obtain a plurality of segmentation information corresponding to the initial text information, the grammatical structure and semantic information corresponding to each segmentation information are analyzed according to the character granularity and the word granularity, and each segmentation information is judged, and if it is judged that the current segmentation information does not meet the preset conditions, the current segmentation information is used as the text information to be corrected; wherein the character granularity represents the segmentation unit at the character level, and the word granularity represents the segmentation unit at the word level;
[0030] Matching the text information to be corrected with a plurality of corresponding similar text information in a similarity dictionary, replacing each similar text information with the text information to be corrected in the initial text information in sequence, and using the replaced initial text information as the candidate text information;
[0031] The degree of confusion corresponding to each candidate text information is calculated, and the candidate text information with the smallest degree of confusion is used as the corrected text information.
[0032] The above-mentioned information error correction method, device, computer equipment and storage medium identify the corresponding initial text information through the target image, segment the initial text information through a word segmentation dictionary, obtain corresponding multiple word segmentation information, analyze the grammatical structure and semantic information corresponding to each word segmentation information according to the character granularity and word granularity, and comprehensively and accurately identify the text information to be corrected in the word segmentation information. Based on this, the accuracy of identifying abnormal information is improved, thereby improving the efficiency and accuracy of error correction for abnormal information. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0034] Figure 1 1 is a flow chart of an information error correction method according to an embodiment;
[0035] Figure 2 1 is a flow chart of an information error correction method in another embodiment;
[0036] Figure 3 is a structural block diagram of an information error correction device in one embodiment;
[0037] Figure 4 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION
[0038] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.
[0039] In one embodiment, Figure 1 As shown, an information error correction method is provided. This embodiment uses the method applied to a server as an example for illustration. It is understandable that the method can also be applied to a terminal, and can also be applied to a system including a terminal and a server, and implemented through the interaction between the terminal and the server. In this embodiment, the method includes steps S102 to S108. Among them:
[0040] Step S102: acquiring a target image and identifying initial text information corresponding to the target image.
[0041] For example, the text information corresponding to the target image may be recognized by using OCR (Optical Character Recognition) technology.
[0042] In step S104, the initial text information is segmented through a word segmentation dictionary to obtain multiple word segmentation information corresponding to the initial text information. The grammatical structure and semantic information corresponding to each word segmentation information are analyzed according to the character granularity and word granularity, and each word segmentation information is judged. If it is judged that the current word segmentation information does not meet the preset conditions, the current word segmentation information is used as the text information to be corrected; wherein the character granularity represents the word segmentation unit at the character level, and the word granularity represents the word segmentation unit at the word level.
[0043] The word segmentation dictionary refers to a dictionary containing Chinese words and their corresponding information, and is used to handle the task of word segmentation for Chinese text.
[0044] Exemplarily, in the word segmentation dictionary, word information corresponding to the words in the initial text information is retrieved and matched, and the initial text information is segmented accordingly according to the word information in the matched word segmentation dictionary. The multiple word segmentation information obtained after segmentation corresponds to the word information matched in the word segmentation dictionary.
[0045] Exemplarily, the grammatical structure and semantic information corresponding to each word segmentation information are analyzed at the character level and the word level respectively. If a word segmentation information does not meet the preset conditions, that is, the grammatical structure and semantic information corresponding to the word segmentation information do not match the grammatical structure and semantic information corresponding to the word information retrieved in the word segmentation dictionary, then the word segmentation information is used as text information to be corrected, that is, it is judged that the word segmentation information is obtained by incorrectly identifying the information in the process of recognizing the initial text information based on the image.
[0046] Optionally, if each word segmentation information satisfies a preset condition, each word segmentation information and the corresponding initial text information are normal information that does not require error correction.
[0047] Step S106 , matching the text information to be corrected with corresponding multiple similar text information in the similarity dictionary, replacing each similar text information with the text information to be corrected in the initial text information in turn, and using the replaced initial text information as the candidate text information.
[0048] Among them, the similar-looking dictionary refers to a dictionary containing Chinese characters with similar shapes, structures or spellings, and is used to handle the task of distinguishing and calling confusing Chinese characters.
[0049] Exemplarily, in a similarity dictionary, similar text information corresponding to the text information to be corrected in shape or structure is retrieved and matched, and in the initial text information, the text information to be corrected is replaced with the similar text information, and the initial text information after the replacement is used as candidate text information; if multiple similar text information is matched, the text information to be corrected is replaced with different similar text information in turn, and the multiple initial text information after the replacement is used as different candidate text information.
[0050] For example, the initial text information containing the text information to be corrected can also be directly used as candidate text information, that is, the text information to be corrected is regarded as matching the exact same text in the similarity dictionary, and the exact same text is used to replace the text information to be corrected in the initial text information.
[0051] Optionally, if no similar text information corresponding to the text information to be corrected in shape or structure is retrieved and matched in the similarity dictionary, the initial text information containing the text information to be corrected is directly used as the candidate text information.
[0052] Step S108 , calculating the degree of confusion corresponding to each candidate text information, and taking the candidate text information with the smallest degree of confusion as the corrected text information.
[0053] The degree of confusion refers to the grammatical, structural or semantic complexity of text information, which is used to represent the difficulty of understanding text information.
[0054] Exemplarily, each candidate text information can be input into a trained language model, which calculates the degree of confusion corresponding to each candidate text information. Based on the calculation result, the candidate text information with the lowest degree of confusion is used as the corrected text information, that is, it is judged that the candidate text information is obtained by correctly identifying the information in the process of recognizing the initial text information based on the image.
[0055] Optionally, the length of the text can be used as an indicator to measure the degree of confusion, that is, text information that is too long or too short may make it difficult to understand the text information; the grammatical structure of the text information can be used as an indicator to measure the degree of confusion, that is, the use of complex grammatical structures or unconventional grammatical structures may increase the difficulty of understanding the text information; the ambiguity of the text information can be used as an indicator to measure the degree of confusion, that is, a specific word or phrase may have multiple interpretations, which increases the ambiguity of the text information and increases the degree of confusion.
[0056] In the above-mentioned information error correction method, the initial text information corresponding to the target image is identified, and the initial text information is segmented through a word segmentation dictionary to obtain corresponding multiple word segmentation information. The grammatical structure and semantic information corresponding to each word segmentation information are analyzed according to the character granularity and word granularity, and the text information to be corrected is comprehensively and accurately identified in the word segmentation information. Based on this, the accuracy of identifying abnormal information is improved, thereby improving the efficiency and accuracy of error correction for abnormal information.
[0057] In an exemplary embodiment, obtaining a target image and identifying initial text information corresponding to the target image includes step S202, wherein:
[0058] Step S202 : Based on the regular expression, the book title marks in the initial text information and the text information within the book title marks are matched according to the left and right boundaries of the book title marks, and the text information within the book title marks is input into the word segmentation dictionary.
[0059] Regular expressions are tools used to match, search, or replace text that conforms to specific text patterns.
[0060] Exemplarily, the left and right boundaries of the book title marks are input into the regular expression, so that the function of the regular expression is set to match the text with book title marks. Based on the regular expression, the book title marks and the text information within the book title marks are matched in the initial text information, so that the text information within the book title marks can be extracted separately and word segmentation processing can be performed.
[0061] Optionally, based on a regular expression containing quotation marks, the title part in the text information can be quickly identified; based on a regular expression containing question marks, the question part in the text information can be quickly identified; based on a regular expression containing exclamation marks, the key marked part in the text information can be quickly identified.
[0062] In this embodiment, based on regular expressions, text information of a specific text pattern is quickly matched, so that information can be adaptively and specifically identified and corrected in different application scenarios.
[0063] In an exemplary embodiment, analyzing the grammatical structure and semantic information corresponding to each word segmentation information according to the character granularity and the word granularity includes step S302, wherein:
[0064] Step S302: Analyze the grammatical structure and semantic information corresponding to each word segmentation information based on the standard rules for grammatical structure and semantic information carried by the word segmentation dictionary; the standard rules include part-of-speech rules, phrase collocation rules, polysemy rules, proper noun recognition rules, and grammatical component rules.
[0065] Among them, part-of-speech rules refer to the rules formulated for the part-of-speech information corresponding to vocabulary, which help to determine the part of speech of words, such as nouns, verbs, adjectives, adverbs, etc.; phrase collocation rules refer to the rules defined based on the collocation between words, which help to determine the fixed collocation of phrases and the customary expression of texts; polysemy rules refer to the rules formulated for words with multiple meanings, which help to identify and distinguish words with multiple meanings in the text; proper noun recognition rules refer to the rules formulated for proper nouns, which are used to identify proper nouns in the text; grammatical component rules refer to the rules formulated for the arrangement and combination of different components in a sentence, which help to determine the grammatical position of words in the text by determining the grammatical components of words, such as subject, predicate, object, etc., to ensure that the text has a legal grammatical structure.
[0066] For example, the word segmentation dictionary carries standard rules formulated for grammatical structure or semantic information, such as part of speech rules, phrase collocation rules, polysemy rules, proper noun recognition rules, and grammatical component rules. Based on these standard rules, the grammatical structure and semantic information corresponding to each word segmentation information are analyzed.
[0067] In this embodiment, based on the standard rules for grammatical structure and semantic information carried by the word segmentation dictionary, each word segmentation information is analyzed in terms of grammatical structure and semantic information, thereby comprehensively and accurately identifying the text information to be corrected.
[0068] In an exemplary embodiment, if it is determined that the current word segmentation information does not meet the preset condition, the current word segmentation information is used as text information to be corrected, including step S402, wherein:
[0069] Step S402: If the current word segmentation information is a single character, the current word segmentation information is analyzed based on the standard rules. If the current word segmentation information does not meet the standard rules, the current word segmentation information is used as text information to be corrected.
[0070] For example, after the initial text information is segmented, multiple segmentation information is obtained. If one of the segmentation information is a single character, it means that the single character does not match the corresponding word information in the segmentation dictionary. The segmentation information corresponding to the single character may be abnormal information, and the segmentation information needs to be analyzed according to the standard rules of the segmentation dictionary. If the segmentation information does not meet the standard rules, the segmentation information is used as text information to be corrected.
[0071] Optionally, if one of the word segmentation information is a single character, that is, the single character does not match the corresponding word information in the word segmentation dictionary, but the word segmentation information meets the standard rules of the word segmentation dictionary, then it indicates that the word segmentation information of the single character is normal information, for example, the single character exists as an independent word.
[0072] In this embodiment, the word segmentation information of a single character that may be abnormal information is further analyzed using standard rules, thereby avoiding determining normal information as abnormal information and improving the accuracy of identifying abnormal information.
[0073] In an exemplary embodiment, the text information to be corrected is matched with corresponding multiple similar text information in the similarity dictionary, and each similar text information is sequentially replaced with the text information to be corrected in the initial text information, including step S502, wherein:
[0074] Step S502: If there are at least two text information to be corrected, each text information to be corrected is matched with the corresponding multiple similar text information in the similarity dictionary, and the similar text information corresponding to different text information to be corrected are combined with each other to obtain multiple similar text combination information, and the multiple similar text combination information are replaced with the corresponding text information to be corrected in the initial text information in sequence.
[0075] Exemplarily, if the text information includes at least two text information to be corrected, then in the similarity dictionary, the corresponding similar text information is matched with different text information to be corrected, and the similar text information corresponding to different text information to be corrected are combined with each other to obtain multiple similar text combination information, and the multiple similar text combination information are sequentially replaced with the corresponding text information to be corrected in the initial text information, thereby finally generating candidate text information corresponding to the number of similar text combination information.
[0076] Optionally, A, C and D are Chinese characters with similar glyphs, B, E and F are Chinese characters with similar glyphs, and the text information includes two text information to be corrected, A and B. In the similarity dictionary, the text information A to be corrected is matched to the similar text information A, C and D, and the text information B to be corrected is matched to the similar text information B, E and F; the similar text information corresponding to different text information to be corrected are combined with each other to obtain nine types of similar text combination information AB, AE, AF, CB, CE, CF, DB, DE and DF; in the initial text information, AB is replaced in turn with different similar text combination information, and finally nine candidate text information containing AB, AE, AF, CB, CE, CF, DB, DE and DF are generated.
[0077] In this embodiment, by combining the similar text information corresponding to different text information to be corrected, multiple similar text combination information are obtained, and the similar text combination information replaces the corresponding text information to be corrected, thereby providing a comprehensive and complete set of candidate text information to improve the accuracy of information correction.
[0078] In an exemplary embodiment, calculating the degree of confusion corresponding to each candidate text information includes step S602, wherein:
[0079] In step S602, each candidate text information is input into the trained model. The model obtains the confusion level of the corresponding candidate text information based on the conditional probability of each character in each candidate text information in the corresponding candidate text information; wherein the conditional probability refers to the probability of the next word appearing given the current word.
[0080] Among them, the model can be expressed as a language model, which can calculate the rationality and probability distribution of a given text by learning the language structure and contextual relationship of large-scale text data, thereby obtaining the degree of confusion of the given text.
[0081] Exemplarily, each candidate text information is input into a trained model, which calculates the conditional probability of each character in each candidate text information in the corresponding candidate text information, that is, in a candidate text information, the model calculates the probability corresponding to the appearance of the current character, and then calculates the probability corresponding to the simultaneous appearance of the next character when the current character appears, thereby calculating the probability of appearance of each character in sequence to obtain the conditional probability corresponding to the simultaneous appearance of all characters in the candidate text information.
[0082] Optionally, the probability corresponding to the candidate text information is inversely proportional to the degree of confusion. The candidate text information with the highest probability indicates that the candidate text information has the highest probability of being normal information and the lowest degree of confusion, so it is easier to understand than other candidate text information.
[0083] Alternatively, the probability of each character appearing can be derived based on the semantic relationships between the characters and their grammatical positions in the text. The semantic relationships between the characters can be expressed as the meaning relationships between the characters in the text, such as the influence between adjacent characters, character combinations, and arrangement. The grammatical positions of the characters in the text can be expressed as the grammatical roles of the characters in the text, such as subject, predicate, or object.
[0084] In this embodiment, the probability of occurrence of each character in the candidate text information is calculated through a model to obtain the probability corresponding to the candidate text information, thereby efficiently and accurately obtaining the degree of confusion of the candidate text information.
[0085] In an exemplary embodiment, Figure 2 As shown, the following steps are included:
[0086] In a business scenario where the title content of an advertising poster is identified and corrected, OCR technology is used to identify the initial text information corresponding to the advertising poster, namely, "The popular short drama "Du Bu Kuang Long" is free for a limited time." Based on a regular expression containing quotation marks, the quotation marks and the text information within the quotation marks are matched in the initial text information, and subsequent correction processing is performed on the text information within the quotation marks, namely, the title content of the advertising poster, "Du Bu Kuang Long."
[0087] Perform word segmentation on the above "Doubu狂龙" through a word segmentation dictionary, and obtain three word segmentation information: "Dou", "Bu", and "狂龙". Analyze the corresponding grammatical structures and semantic information of each word segmentation information according to character granularity and word granularity. Among them, the word segmentation information of "Bu" does not meet the standard rules of the word segmentation dictionary for grammatical structures and semantic information, and the word segmentation information of "Bu" is used as the text information to be corrected. Among them, the word segmentation dictionary can adopt the jieba library, or a custom dictionary that collects specific vocabulary based on specific business scenarios, so that the word segmentation dictionary is adapted to different business scenarios and can meet iterative business requirements; in addition, a homograph dictionary can also be applied to the above custom operations.
[0088] Match the text information to be corrected of "Bu" to four homograph text information: "Bu", "City", "Coin", and "Towel" through an approximate dictionary, and replace "Bu" in "Doubu狂龙" with the four homograph text information in turn, and obtain four candidate text information: "Doubu狂龙", "DouShi狂龙", "DouCoin狂龙", and "DouTowel狂龙".
[0089] Input the four candidate text information into the natural language model respectively, and calculate the perplexity corresponding to each candidate text information. That is, the perplexity of "Doubu狂龙" is 0.5, the perplexity of "DouShi狂龙" is 0.01, the perplexity of "DouCoin狂龙" is 0.3, and the perplexity of "DouTowel狂龙" is 0.3. Sort different candidate text information according to the perplexity from low to high. Therefore, "DouShi狂龙" with the lowest perplexity is used as the corrected text information. Among them, the perplexity is equivalent to the degree of confusion.
[0090] In this embodiment, in the case of low pixels, strong interference, deformed text, etc., the text information recognized from the image may be incorrect. Based on this method, the incorrect text information can be corrected efficiently and accurately, thereby ensuring the reliability of the result.
[0091] It should be understood that although the steps in the flowcharts involved in the above embodiments are shown in sequence according to the indications of the arrows, these steps do not necessarily need to be executed in the order indicated by the arrows. Unless there is a clear statement in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages. These steps or stages do not necessarily need to be executed at the same time, but can be executed at different times. The execution order of these steps or stages does not necessarily need to be sequential, but can be executed alternately or alternately with at least a part of other steps or steps or stages in other steps.
[0092] Based on the same inventive concept, the embodiments of the present application also provide an information error correction device for implementing the information error correction method involved above. The implementation solution provided by this device is similar to the implementation solution described in the above method. Therefore, the specific limitations in one or more information error correction device embodiments provided below can be referred to the limitations of the information error correction method above and will not be repeated here.
[0093] In an exemplary embodiment, Figure 3 As shown, an information error correction device is provided, including: an acquisition module 702, a word segmentation module 704, a matching module 706 and a calculation module 708, wherein:
[0094] The acquisition module 702 is used to acquire a target image and identify initial text information corresponding to the target image.
[0095] The word segmentation module 704 is used to segment the initial text information through a word segmentation dictionary to obtain multiple word segmentation information corresponding to the initial text information, analyze the grammatical structure and semantic information corresponding to each word segmentation information according to the character granularity and word granularity, and judge each word segmentation information. If it is judged that the current word segmentation information does not meet the preset conditions, the current word segmentation information is used as the text information to be corrected; wherein the character granularity represents the word segmentation unit at the character level, and the word granularity represents the word segmentation unit at the word level.
[0096] The matching module 706 is used to match the text information to be corrected with corresponding multiple similar text information in the similarity dictionary, replace the text information to be corrected in the initial text information with each similar text information in turn, and use the replaced initial text information as candidate text information.
[0097] The calculation module 708 is configured to calculate the degree of confusion corresponding to each candidate text information, and take the candidate text information with the smallest degree of confusion as the corrected text information.
[0098] In an exemplary embodiment, the acquisition module 702 is further configured to match the book title marks in the initial text information and the text information within the book title marks based on the left and right boundaries of the book title marks based on regular expressions, and input the text information within the book title marks into the word segmentation dictionary.
[0099] In an exemplary embodiment, the word segmentation module 704 is also used to analyze the grammatical structure and semantic information corresponding to each word segmentation information based on the standard rules for grammatical structure and semantic information carried by the word segmentation dictionary; wherein the standard rules include part of speech rules, phrase collocation rules, polysemy rules, proper noun recognition rules and grammatical component rules.
[0100] In an exemplary embodiment, the word segmentation module 704 is also used to analyze the current word segmentation information based on standard rules if the current word segmentation information is a single character, and to use the current word segmentation information as text information to be corrected if the current word segmentation information does not meet the standard rules.
[0101] In an exemplary embodiment, the matching module 706 is also used to match each text information to be corrected with corresponding multiple similar text information in the similarity dictionary if at least two text information to be corrected are included, combine the similar text information corresponding to different text information to be corrected with each other to obtain multiple similar text combination information, and replace the corresponding text information to be corrected in the initial text information with the multiple similar text combination information in sequence.
[0102] In an exemplary embodiment, the calculation module 708 is also used to input each candidate text information into a trained model, and the model obtains the confusion level of the corresponding candidate text information based on the conditional probability of each character in each candidate text information in the corresponding candidate text information; wherein the conditional probability refers to the probability of the next word appearing given the current word.
[0103] Each module in the above-mentioned information error correction device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each module.
[0104] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 4 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store images, various text information, word segmentation dictionary data, morphological dictionary data and language model data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an information error correction method is implemented.
[0105] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.
[0106] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:
[0107] Obtain a target image and identify the initial text information corresponding to the target image;
[0108] The initial text information is segmented through a word segmentation dictionary to obtain multiple word segmentation information corresponding to the initial text information. The grammatical structure and semantic information corresponding to each word segmentation information are analyzed according to the character granularity and word granularity, and each word segmentation information is judged. If it is judged that the current word segmentation information does not meet the preset conditions, the current word segmentation information is used as the text information to be corrected; the character granularity represents the word segmentation unit at the character level, and the word granularity represents the word segmentation unit at the word level;
[0109] Matching the text information to be corrected with corresponding multiple similar text information in the similarity dictionary, replacing each similar text information with the text information to be corrected in the initial text information in turn, and using the replaced initial text information as the candidate text information;
[0110] The degree of confusion corresponding to each candidate text information is calculated, and the candidate text information with the smallest degree of confusion is used as the corrected text information.
[0111] In one embodiment, when the processor executes the computer program, it also implements the following steps: based on a regular expression, according to the left and right boundaries of the book title marks, matching the book title marks in the initial text information and the text information within the book title marks, and inputting the text information within the book title marks into the word segmentation dictionary.
[0112] In one embodiment, when the processor executes the computer program, it also implements the following steps: based on the standard rules for grammatical structure and semantic information carried by the word segmentation dictionary, the grammatical structure and semantic information corresponding to each word segmentation information are analyzed; wherein the standard rules include part of speech rules, phrase collocation rules, polysemy rules, proper noun recognition rules and grammatical component rules.
[0113] In one embodiment, when the processor executes the computer program, the following steps are also implemented: if the current word segmentation information is a single character, the current word segmentation information is analyzed based on standard rules; if the current word segmentation information does not meet the standard rules, the current word segmentation information is used as text information to be corrected.
[0114] In one embodiment, when the processor executes the computer program, the following steps are also implemented: if there are at least two text information to be corrected, each text information to be corrected is matched with corresponding multiple similar text information in the similarity dictionary, the similar text information corresponding to different text information to be corrected are combined with each other to obtain multiple similar text combination information, and the multiple similar text combination information are replaced with the corresponding text information to be corrected in the initial text information in sequence.
[0115] In one embodiment, when the processor executes the computer program, the following steps are also implemented: each candidate text information is input into a trained model, and the model obtains the confusion level of the corresponding candidate text information based on the conditional probability of each character in each candidate text information in the corresponding candidate text information; wherein the conditional probability refers to the probability of the next word appearing given the current word.
[0116] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:
[0117] Obtain a target image and identify the initial text information corresponding to the target image;
[0118] The initial text information is segmented through a word segmentation dictionary to obtain multiple word segmentation information corresponding to the initial text information. The grammatical structure and semantic information corresponding to each word segmentation information are analyzed according to the character granularity and word granularity, and each word segmentation information is judged. If it is judged that the current word segmentation information does not meet the preset conditions, the current word segmentation information is used as the text information to be corrected; the character granularity represents the word segmentation unit at the character level, and the word granularity represents the word segmentation unit at the word level;
[0119] Matching the text information to be corrected with corresponding multiple similar text information in the similarity dictionary, replacing each similar text information with the text information to be corrected in the initial text information in turn, and using the replaced initial text information as the candidate text information;
[0120] The degree of confusion corresponding to each candidate text information is calculated, and the candidate text information with the smallest degree of confusion is used as the corrected text information.
[0121] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: based on regular expressions, according to the left and right boundaries of the book title marks, the book title marks in the initial text information and the text information within the book title marks are matched, and the text information within the book title marks is input into the word segmentation dictionary.
[0122] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: based on the standard rules for grammatical structure and semantic information carried by the word segmentation dictionary, the grammatical structure and semantic information corresponding to each word segmentation information are analyzed; wherein the standard rules include part of speech rules, phrase collocation rules, polysemy rules, proper noun recognition rules and grammatical component rules.
[0123] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: if the current word segmentation information is a single character, the current word segmentation information is analyzed based on standard rules; if the current word segmentation information does not meet the standard rules, the current word segmentation information is used as text information to be corrected.
[0124] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: if it includes at least two text information to be corrected, each text information to be corrected is matched with corresponding multiple similar text information in the similarity dictionary, the similar text information corresponding to different text information to be corrected are combined with each other to obtain multiple similar text combination information, and the multiple similar text combination information are replaced with the corresponding text information to be corrected in the initial text information in sequence.
[0125] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: each candidate text information is input into a trained model, and the model obtains the confusion level of the corresponding candidate text information based on the conditional probability of each character in each candidate text information in the corresponding candidate text information; wherein the conditional probability refers to the probability of the next word appearing given the current word.
[0126] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.
[0127] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), data processing logic devices based on quantum computing, and the like.
[0128] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0129] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.
Claims
1. An information error correction method, characterized in that: The method comprises: Acquiring a target image and identifying initial text information corresponding to the target image, including: matching text information corresponding to the target symbol in the initial text information based on a target symbol included in a regular expression, and inputting the text information corresponding to the target symbol into a word segmentation dictionary, wherein the target symbol includes a book title mark, a question mark, and an exclamation mark; The initial text information is segmented through a segmentation dictionary to obtain a plurality of segmentation information corresponding to the initial text information, the grammatical structure and semantic information corresponding to each segmentation information are analyzed according to the character granularity and the word granularity, and each segmentation information is judged, and if it is judged that the current segmentation information does not meet the preset conditions, the current segmentation information is used as the text information to be corrected, including: if the grammatical structure and semantic information corresponding to the current segmentation information do not match the grammatical structure and semantic information corresponding to the word information retrieved in the segmentation dictionary, the current segmentation information is used as the text information to be corrected; wherein the character granularity represents the segmentation unit at the character level, and the word granularity represents the segmentation unit at the word level; Matching the text information to be corrected with corresponding multiple similar text information in a similar dictionary, replacing the text information to be corrected in the initial text information with each similar text information in turn, and using the replaced initial text information as candidate text information; Calculating the degree of confusion corresponding to each candidate text information, and taking the candidate text information with the smallest degree of confusion as the corrected text information; wherein the text length, grammatical structure and text ambiguity of each candidate text information are used as indicators to measure the degree of confusion; the degree of confusion represents the complexity of the candidate text information in terms of grammar, structure or semantics, and is used to characterize the difficulty of understanding the candidate text information; The grammatical structure and semantic information corresponding to each word segmentation information is analyzed according to the character granularity and word granularity, including: Based on the standard rules for grammatical structure and semantic information carried by the word segmentation dictionary, the grammatical structure and semantic information corresponding to each word segmentation information are analyzed; wherein the standard rules include part-of-speech rules, phrase collocation rules, polysemous word rules, proper noun recognition rules and grammatical component rules; If it is determined that the current word segmentation information does not meet the preset condition, the current word segmentation information is used as text information to be corrected, including: If the current word segmentation information is a single character, the current word segmentation information is analyzed based on the standard rule; if the current word segmentation information does not meet the standard rule, the current word segmentation information is used as text information to be corrected.
2. The method according to claim 1, characterized in that The acquiring of the target image and identifying the initial text information corresponding to the target image includes: Based on regular expressions, the book title marks in the initial text information and the text information in the book title marks are matched according to the left and right boundaries of the book title marks, and the text information in the book title marks is input into the word segmentation dictionary.
3. The method according to claim 1, characterized in that The step of matching the text information to be corrected with corresponding multiple similar text information in the similarity dictionary, and sequentially replacing each similar text information with the text information to be corrected in the initial text information, includes: If at least two text information to be corrected are included, each text information to be corrected is matched with corresponding multiple similar text information in the similarity dictionary, and the similar text information corresponding to different text information to be corrected are combined with each other to obtain multiple similar text combination information, and the multiple similar text combination information are replaced in sequence with the corresponding text information to be corrected in the initial text information.
4. The method according to claim 1, characterized in that: The calculating the confusion degree corresponding to each candidate text information includes: Each candidate text information is input into a trained model, and the model obtains the confusion degree of the corresponding candidate text information according to the conditional probability of each character in each candidate text information in the corresponding candidate text information; wherein the conditional probability refers to the probability of the next word appearing given the current word.
5. The method according to claim 4, characterized in that The step of inputting each candidate text information into a trained model, wherein the model obtains the confusion degree of the corresponding candidate text information according to the conditional probability of each character in each candidate text information in the corresponding candidate text information, including: In the current candidate text information, the probability corresponding to the appearance of the current character is calculated. When the current character appears, the probability corresponding to the simultaneous appearance of the next character is calculated. The probability of the appearance of each character is calculated in sequence to obtain the conditional probability corresponding to the simultaneous appearance of all characters in the current candidate text information.
6. The method according to claim 4, characterized in that The probability of each character appearing is determined by the semantic association between the characters and the grammatical position of the characters in the text. The semantic association between characters is represented by the meaning relationship between characters in the text, and the meaning relationship between characters includes the influence between adjacent characters, the combination and arrangement of characters; The grammatical position relationship of characters in the text is represented by the grammatical role of each character in the text. The grammatical role of each character in the text includes subject, predicate and object.
7. An information error correction device, characterized in that: The device comprises: An acquisition module is used to acquire a target image and identify initial text information corresponding to the target image, including: matching text information corresponding to the target symbol in the initial text information based on a target symbol included in a regular expression, and inputting the text information corresponding to the target symbol into a word segmentation dictionary, wherein the target symbol includes a book title mark, a question mark, and an exclamation mark; The word segmentation module is used to perform word segmentation processing on the initial text information through a word segmentation dictionary to obtain multiple word segmentation information corresponding to the initial text information, analyze the grammatical structure and semantic information corresponding to each word segmentation information according to the character granularity and the word granularity, and judge each word segmentation information. If it is judged that the current word segmentation information does not meet the preset conditions, the current word segmentation information is used as the text information to be corrected, including: if the grammatical structure and semantic information corresponding to the current word segmentation information do not match the grammatical structure and semantic information corresponding to the word information retrieved in the word segmentation dictionary, the current word segmentation information is used as the text information to be corrected; wherein the word granularity represents the word segmentation unit at the character level, and the word granularity represents the word segmentation unit at the word level; a matching module, used for matching the text information to be corrected with corresponding multiple similar text information in a similarity dictionary, replacing the text information to be corrected in the initial text information with each similar text information in turn, and using the replaced initial text information as candidate text information; A calculation module, used to calculate the degree of confusion corresponding to each candidate text information, and take the candidate text information with the smallest degree of confusion as the corrected text information; wherein the text length, grammatical structure and text ambiguity of each candidate text information are used as indicators to measure the degree of confusion; the degree of confusion represents the complexity of the candidate text information in terms of grammar, structure or semantics, and is used to characterize the difficulty of understanding the candidate text information; The word segmentation module is also used to: analyze the grammatical structure and semantic information corresponding to each word segmentation information based on the standard rules for grammatical structure and semantic information carried by the word segmentation dictionary; wherein the standard rules include part of speech rules, phrase collocation rules, polysemous word rules, proper noun recognition rules and grammatical component rules; The word segmentation module is also used to: if the current word segmentation information is a single character, analyze the current word segmentation information based on the standard rule; if the current word segmentation information does not meet the standard rule, use the current word segmentation information as text information to be corrected.
8. The device according to claim 7, characterized in that The matching module is also used for: If at least two text information to be corrected are included, each text information to be corrected is matched with corresponding multiple similar text information in the similarity dictionary, and the similar text information corresponding to different text information to be corrected are combined with each other to obtain multiple similar text combination information, and the multiple similar text combination information are replaced in sequence with the corresponding text information to be corrected in the initial text information.
9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
