Image restoration method based on image detection and region extraction

Abstract

The present application relates to the technical field of data processing, and more particularly to an image restoration method based on image detection and region extraction, comprising S1, establishing a Chinese character pattern library; S2, an AI processing module acquires the binary grayscale coordinates of a target image and identifies the content in the target image; S3, the AI processing module performs correlation determination on the Chinese character content in the acquired target image to confirm the rationality of the Chinese character content in the target image; S4, the AI processing module outputs a new electronic document according to the content encoding in the target image; and S5, the AI processing module performs deep learning on the comparison process and comparison result. The present application can make the paper version file directly output a clear digital signal version document after scanning and archiving, and improve the processing efficiency, by converting the paper version file into an electronic digital signal output, and then re-identifying, correcting, supplementing and re-entering the Chinese character content.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to an image restoration method based on image detection and region extraction. Background Technology

[0002] With the development of the Internet era and the popularization of electronic archives, the recognition of Chinese characters in electronic documents has become increasingly important. In the recognition of electronic documents, scanning and archiving the original paper documents has become the main means of converting paper documents into electronic documents. However, related problems have also arisen. Some paper documents are old and have been copied many times, resulting in abnormally blurry electronic documents with a lot of noise and distorted content after scanning. Even worse, some Chinese characters are no longer recognizable, which reduces the value of the scanned digital documents. Therefore, there is an urgent need for a method to re-recognize, correct, fill in, and re-enter Chinese characters in documents after scanning.

[0003] Chinese Patent Publication No. CN106548169A discloses a method and apparatus for enhancing fuzzy Chinese characters based on deep neural networks. The method includes: establishing a reference database; acquiring test images containing Chinese characters; dividing the test images into multiple test image blocks according to image block partitioning rules; using each test image block as a target search index in the reference database to select multiple preset image blocks most similar to the test image blocks; weightedly fusing the multiple most similar preset image blocks into restored image blocks according to a fusion coefficient; and weightedly fusing adjacent restored image blocks to obtain the restored image. This invention also provides a device for enhancing fuzzy Chinese characters based on deep neural networks. By introducing deep neural network features when establishing the reference data and retrieving image blocks, the robustness of the image blocks is improved; even in offline mode, images containing fuzzy Chinese characters can be restored into clear images through a trained database, facilitating the display or recognition of Chinese characters in the image and improving the recognition and clarity of Chinese characters in the image. In the deep neural network-based fuzzy Chinese character enhancement method, the target image is normalized. Although the influence weights of each feature dimension on the target are uniform, the method itself has poor robustness, and the maximum and minimum values ​​are very easily affected by outliers. Summary of the Invention

[0004] To address this, the present invention provides an image restoration method based on image detection and region extraction, which overcomes the problem in the prior art that Chinese characters in old paper documents cannot be re-identified, corrected, supplemented, and re-entered.

[0005] To achieve the above objectives, the present invention provides an image restoration method based on image detection and region extraction, comprising:

[0006] Step S1: Establish a Chinese character image library;

[0007] In step S2, the AI ​​processing module obtains the binarized grayscale coordinates of the target image and identifies the Chinese characters, font size, font and content in the target image;

[0008] Step S3: The AI ​​processing module performs correlation determination on the Chinese characters in the acquired target image to confirm the rationality of the Chinese characters in the target image, and determines the coordinates, format and encoding of the corresponding Chinese characters in the target image.

[0009] In step S4, the AI ​​processing module outputs a new electronic document based on the relative coordinates of the Chinese characters in the target image, the Chinese character encoding, and the format encoding.

[0010] In step S5, the AI ​​processing module performs deep learning based on the comparison process and results of comparing the Chinese character image library with the target image, in order to improve the accuracy and efficiency of the AI ​​processing module in recognizing the target image.

[0011] Specifically, the method for obtaining the target image can be scanning an electronic version with a photocopier, faxing an electronic version, or taking an electronic version with a mobile phone or camera. As long as it can meet the requirements for obtaining the target image in this embodiment, it will not be elaborated further.

[0012] In step S1, the Chinese character graphic library includes a Chinese character library and a punctuation character library. The Chinese character library contains Chinese characters in several fonts, and the punctuation character library includes letters, numbers, operators, punctuation marks, and functional symbols. The Chinese character library assigns a number to any Chinese character it contains, and the punctuation character library assigns a number to any symbol it contains. The Chinese character library assigns a number to any computer font for any Chinese character it contains, and marks the relative coordinates of the features of any computer font for any Chinese character it contains in a two-dimensional coordinate system, thus obtaining a two-dimensional coordinate model of any Chinese character and any font. The punctuation character library assigns a number to any computer font for any symbol it contains, and marks the relative coordinates of the features of any computer font for any symbol it contains in a two-dimensional coordinate system, thus obtaining a two-dimensional coordinate model of any symbol and any font.

[0013] Specifically, the Chinese character database contains 100,000 characters, including simplified Chinese characters, traditional Chinese characters, ancient written Chinese characters, and oracle bone script.

[0014] Specifically, the punctuation character set includes English and Chinese letters, Roman letters, and Cyrillic letters, and numbers including Roman numerals and Arabic numerals.

[0015] The Chinese character image library also includes word relationships and phrase relationships for all Chinese characters in the library. Based on these word relationships and phrase relationships, the degree of association between characters, between characters and words, between words, and between phrases is graded to obtain the first degree of association, the second degree of association, the third degree of association, and the fourth degree of association. Among them, the degree of association is ranked according to the frequency of use as follows: first degree of association > second degree of association > third degree of association > fourth degree of association. The degree of association is used to determine the Chinese characters that cannot be identified by relying on the simplified Chinese character library and the traditional Chinese character library based on the frequency of use of words.

[0016] The Chinese character image library also includes a typesetting library and a typesetting training library, which are used to store the original format typesetting corresponding to the target image, including the version without Chinese characters and the version with Chinese characters. It is used to store the Chinese character typesetting type associated with the target image and its corresponding number, and to establish a two-dimensional coordinate model corresponding to any Chinese character typesetting type in the two-dimensional coordinate system.

[0017] In step S1, the Chinese character image library assigns character levels to the Chinese characters it contains. The character levels include Level 1, Level 2, Level 3, and Level 4 characters. The character levels are ordered by frequency of use as follows: Level 1 > Level 2 > Level 3 > Level 4. The Chinese character image library also assigns font levels to the fonts it contains. The font levels include Level 1, Level 2, and Level 3 fonts. The font levels are ordered by frequency of use as follows: Level 1 > Level 2 > Level 3. The Chinese character image library also assigns symbol levels to the symbols it contains. The symbol levels include Level 1, Level 2, and Level 3 symbols. The symbol levels are ordered by frequency of use as follows: Level 1 > Level 2 > Level 3.

[0018] In step S2, the AI ​​processing module performs binarization on the target image to obtain the coordinates of each point in the target image in two-dimensional coordinates and their corresponding gray values, and then segments the target image according to the gray value of each coordinate point.

[0019] When the grayscale value of any coordinate point is less than the first grayscale value preset by the AI ​​processing module, the AI ​​processing module determines that the coordinate point corresponding to the current grayscale value is an invalid feature point. When the grayscale value of any coordinate point is greater than the second grayscale value preset by the AI ​​processing module, the AI ​​processing module determines that the coordinate point corresponding to the current grayscale value is an important feature point. When the grayscale value of any coordinate point is less than the second grayscale value but greater than the first grayscale value, the AI ​​processing module determines that the coordinate point corresponding to the current grayscale value is a valid feature point. Here, the first grayscale value < the second grayscale value, and the feature points are ranked according to their importance in terms of feature representation as follows: important feature point > valid feature point > invalid feature point.

[0020] The AI ​​processing module performs proportional distance judgment based on the coordinate spacing between effective feature points in the target image and the coordinate spacing between effective feature points in the Chinese character and symbol library. This is used to determine the cutting range of a single Chinese character and symbol in the target image. Based on the cutting range, the module cuts the single Chinese character and symbol in the target image to obtain the two-dimensional coordinates of the single Chinese character and symbol in the target image. Finally, based on the absolute value of the two-dimensional coordinates of the single Chinese character and symbol in the target image, the module obtains the font size of the single Chinese character and symbol in the target image.

[0021] In step S2, when the AI ​​processing module segments the target image, it segments the target image by taking each recognizable single Chinese character and single symbol as a minimum segmentation unit, resulting in segmented graphics of several single Chinese character groups and single symbol groups. The remaining unrecognizable Chinese characters and symbols are retained as blurred graphics. The AI ​​processing module marks all segmented graphics and blurred graphics according to the original coordinate order of the target image, and determines the relative positions of all segmented graphics and blurred graphics to facilitate subsequent re-layout and re-entry.

[0022] In step S2, the AI ​​processing module proportionally reduces the two-dimensional coordinate models in the Chinese character library and punctuation character library based on the coordinate values ​​corresponding to the font sizes of the Chinese characters and symbols in the target image, obtaining Chinese character graphic library font sizes consistent with the font sizes of the Chinese characters and symbols in the target image. Furthermore, based on the comparison results between the important and effective feature points of the Chinese characters and symbols in the target image and the important and effective feature points of the Chinese characters and symbols in the Chinese character graphic library, the module determines the font of a single Chinese character and the font of a single symbol in the target image.

[0023] When the AI ​​processing module determines the font of a single Chinese character in a target image, it sorts them in ascending order of frequency of use and compares them with the fonts in the Chinese character library level by level. That is, it first compares the first level fonts. If the font of the Chinese character still cannot be determined after the first level font comparison is completed, it then compares the second level fonts and the third level fonts in turn, until the font of the Chinese character is determined.

[0024] When the AI ​​processing module determines the content of a single Chinese character in a target image, it sorts the characters by their frequency of use in ascending order and compares them with the characters in the character library level by level. That is, it first compares the first-level characters. If the content of the Chinese character still cannot be determined after the comparison of the first-level characters, it then compares the second-level characters, the third-level characters, and the fourth-level characters in turn, until the content of the Chinese character is determined.

[0025] In step S2, the AI ​​processing module determines the blurred image. Specifically, it separates overlapping key and valid feature points to obtain one overlapping Chinese character image and two non-overlapping Chinese character images. The key and valid feature points of the overlapping image are then restored to the missing parts of the characters in the two non-overlapping images. Based on the comparison results between the two restored non-overlapping images and the key and valid feature points of characters in the Chinese character image library, the module determines the corresponding Chinese character content for each of the two restored non-overlapping images.

[0026] When the AI ​​processing module determines the Chinese characters corresponding to the two restored non-overlapping Chinese character images, it compares them with the Chinese characters in the character library in ascending order of the frequency of use of the Chinese characters, from high to low. That is, it first compares the first level of Chinese characters. If it still cannot determine the Chinese character after the comparison of the first level of Chinese characters, it then compares the second level of Chinese characters, the third level of Chinese characters, and the fourth level of Chinese characters in turn, until the Chinese character is determined.

[0027] In step S2, when the AI ​​processing module determines the Chinese character, font, or symbol in the target image that corresponds to the Chinese character image library, if there is no corresponding Chinese character, font, or symbol in the Chinese character image library, the AI ​​processing module determines that the Chinese character image library needs to be supplemented. It adds a corresponding color mark to the position of the two-dimensional coordinates in the target image of the Chinese character, so as to add a prompt background color when the Chinese character is output and re-entered, and outputs a prompt that the Chinese character image library needs to be supplemented, so as to facilitate manual verification and timely supplementation of the Chinese character image library.

[0028] In step S3, the AI ​​processing module performs a correlation determination on the Chinese characters in the acquired target image to confirm the rationality of the Chinese characters in the target image. When determining the correlation level of adjacent Chinese characters in the acquired target image, the AI ​​processing module compares the correlation levels with the Chinese characters in the character library in ascending order of frequency of use. That is, it first compares the first correlation level. After the first level of correlation comparison is completed, if the correlation of the Chinese characters still cannot be determined, it then compares the second, third, and fourth correlation levels in sequence until the correlation of the Chinese characters is determined. If the AI ​​processing module cannot determine the correlation of any Chinese character, it determines that there may be a typo or recognition error, and adds a corresponding color mark to the position of the Chinese character in the two-dimensional coordinates of the target image. This color mark is used to add a prompt background color when the Chinese character is output and re-entered, so as to facilitate manual verification.

[0029] In step S3, when the AI ​​processing module determines the coordinates, format, and encoding of the corresponding Chinese characters in the acquired target image, it outputs digital signals to the output unit based on the coordinates, format, and corresponding Chinese character coordinates, character number, and symbol number from the Chinese character image library in the acquired target image. The AI ​​processing module also outputs digital signals to the output unit based on the corresponding coordinate order and image ratio of the unrecognized Chinese character images in the target image.

[0030] The AI ​​processing module determines the layout format of the current target image based on the corresponding layout format in the layout library, and outputs the corresponding layout format digital signal. The layout format includes document type, table type, and specific layout coordinate position information. When the AI ​​processing module determines that there is no layout in the layout library that is the same as the layout format of the current target image, it determines the layout of the target image based on the edge range of the gray value of the target image, and selects the document type or table type format based on the layout of the target image to output the layout format digital signal with the corresponding proportion and tilt.

[0031] In step S4, the AI ​​processing module outputs digital signals based on the relative coordinates of Chinese characters, Chinese character encoding, and format encoding in the target image to form a new electronic document. The digital signals include tilt adjustment values ​​and corresponding coordinate signals, scale adjustment values ​​and corresponding coordinate signals, font and corresponding coordinate signals, symbols and corresponding coordinate signals, Chinese characters and corresponding coordinate signals, font size and corresponding coordinate signals, Chinese character images and corresponding coordinate signals, and color symbols and their corresponding coordinate signals.

[0032] Specifically, this application uses existing technologies for target image recognition, comparison of icon images with the training library, and deep learning methods. The specific selection of these technologies is based on preset feature requirements and the discriminability of the target image. The preset values ​​of the parameters are also set according to the user and usage requirements. This application protects a method for image recognition, therefore, it does not disclose the specific parameter setting requirements, nor does it affect the specific implementation of this application in practical applications. The existing technologies can be unsupervised machine learning methods, supervised machine learning methods, or deep learning algorithms such as convolutional neural networks, as long as they can meet the needs of image recognition, comparison, and learning in this embodiment. This will not be elaborated further.

[0033] Compared with the prior art, the beneficial effects of the present invention are as follows: First, through the provided Chinese character image library, a large number of training samples of Chinese characters and symbols under different fonts can be stored, and the feature values ​​of relevant Chinese characters and symbols under different fonts can be extracted to quickly identify the font of Chinese characters and symbols in the target image.

[0034] Furthermore, by setting up a typesetting library within the Chinese character graphic library, deep learning can be performed on the original typesetting electronic archives of the target image to better identify the typesetting, file format and other information corresponding to the target image, and output digital signals to form new electronic documents, thereby enhancing the recognizability and clarity of scanned and archived paper documents.

[0035] Furthermore, by assigning corresponding levels to the relationships between Chinese characters, symbols, fonts, and files within the Chinese character image library, the identification of target images can be simplified and made to be identified step by step, reducing the large amount of computational work in image recognition and making the recognition work faster.

[0036] Secondly, by binarizing the target image, the range and tilt of the Chinese characters in the target image can be obtained, which facilitates the subsequent target image cutting operation. It can better cut out single Chinese characters and single symbols, and perform step-by-step recognition for single Chinese characters and single symbols, increasing the efficiency of feature point acquisition, reducing the related computational load, and improving the processing speed.

[0037] Third, by segmenting the target image, we focus on obtaining unclear or overlapping Chinese characters and symbols, and perform segmentation and restoration of the unclear or overlapping Chinese characters and symbols using preset feature points, thereby determining the specific content expressed by the Chinese characters and symbols and optimizing computing power.

[0038] Furthermore, by performing fuzzy and correlation determination on unclear or overlapping Chinese characters and symbols, it is possible to correct and supplement them to a certain extent. Meanwhile, for unclear or overlapping Chinese characters and symbols that cannot be corrected or supplemented, by using the sequential numbering of the coordinate system, their segmented proportional images can be displayed in a new electronic document with a bottom highlight color. This can be used for manual verification, supplementing the Chinese character image library, or manually correcting the unclear Chinese characters and symbols.

[0039] Fourth, by sorting all the segmented images of the target image, a virtual permutation and combination of the corresponding content can be obtained. The rationality of the Chinese character content in the target image can be confirmed by using the correlation of Chinese characters. Based on the virtual permutation and combination of the corresponding content, a digital signal of the corresponding order is output to form a new electronic document.

[0040] Fifth, through the established target image layout library, the system can learn from relatively clear layout files or layout template files without Chinese characters in advance based on the layout method of the target image. This allows for faster acquisition of the target image's layout format and tilt, and corresponding corrections can be made on the coordinate system. At the same time, the standardized layout format of the target image is perfectly presented in the new electronic document. Additionally, unrecognizable content is highlighted in the new electronic document, facilitating the addition of new related logic and learning methods by the user to improve the image recognition, correction, and output methods. Attached Figure Description

[0041] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0042] Figure 1 This is a schematic diagram of the image restoration method based on image detection and region extraction according to an embodiment of the present invention; Detailed Implementation

[0043] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0044] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0045] It should be noted that in the description of this invention, the terms "upper", "lower", "left", "right", "inner", "outer", etc., which indicate directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and is not intended to indicate or imply that the device or element must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of this invention.

[0046] Furthermore, it should be noted that, in the description of this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0047] Please see Figure 1As shown, it is a structural schematic diagram of the image restoration method based on image detection and region extraction according to an embodiment of the present invention. This embodiment includes:

[0048] Step S1: Establish a Chinese character image library;

[0049] In step S2, the AI ​​processing module obtains the binarized grayscale coordinates of the target image and identifies the Chinese characters, font size, font and content in the target image;

[0050] Step S3: The AI ​​processing module performs correlation determination on the Chinese characters in the acquired target image to confirm the rationality of the Chinese characters in the target image, and determines the coordinates, format and encoding of the corresponding Chinese characters in the target image.

[0051] In step S4, the AI ​​processing module outputs a new electronic document based on the relative coordinates of the Chinese characters in the target image, the Chinese character encoding, and the format encoding.

[0052] In step S5, the AI ​​processing module performs deep learning based on the comparison process and results of comparing the Chinese character image library with the target image, in order to improve the accuracy and efficiency of the AI ​​processing module in recognizing the target image.

[0053] In this embodiment, the method for obtaining the target image can be scanning an electronic version with a photocopier, faxing an electronic version, or taking an electronic version with a mobile phone or camera. As long as it can meet the requirements for obtaining the target image in this embodiment, it will not be elaborated further.

[0054] 1. In step S1, the Chinese character graphic library includes a Chinese character library and a punctuation character library. The Chinese character library contains Chinese characters in several fonts. The punctuation character library includes letters, numbers, operators, punctuation marks, and functional symbols. The Chinese character library assigns a number to any Chinese character it contains, and the punctuation character library assigns a number to any symbol it contains. The Chinese character library assigns a number to any computer font for any Chinese character it contains, and marks the relative coordinates of the features of any computer font for any Chinese character it contains in a two-dimensional coordinate system, thus obtaining a two-dimensional coordinate model of any Chinese character and any font. The punctuation character library assigns a number to any computer font for any symbol it contains, and marks the relative coordinates of the features of any computer font for any symbol it contains in a two-dimensional coordinate system, thus obtaining a two-dimensional coordinate model of any symbol and any font.

[0055] In this embodiment, the Chinese character library contains 100,000 characters, including simplified Chinese characters, traditional Chinese characters, ancient written Chinese characters, and oracle bone script.

[0056] In this embodiment, the letters in the punctuation character library include English and Chinese letters, Roman letters, and Cyrillic letters, and the numbers include Roman numerals and Arabic numerals.

[0057] The Chinese character image library also includes word relationships and phrase relationships for all Chinese characters in the library. Based on these word relationships and phrase relationships, the degree of association between characters, between characters and words, between words, and between phrases is graded to obtain the first degree of association, the second degree of association, the third degree of association, and the fourth degree of association. Among them, the degree of association is ranked according to the frequency of use as follows: first degree of association > second degree of association > third degree of association > fourth degree of association. The degree of association is used to determine the Chinese characters that cannot be identified by relying on the simplified Chinese character library and the traditional Chinese character library based on the frequency of use of words.

[0058] The Chinese character image library also includes a typesetting library and a typesetting training library, which are used to store the original format typesetting corresponding to the target image, including the version without Chinese characters and the version with Chinese characters. It is used to store the Chinese character typesetting type associated with the target image and its corresponding number, and to establish a two-dimensional coordinate model corresponding to any Chinese character typesetting type in the two-dimensional coordinate system.

[0059] 2. In step S1, the Chinese character image library assigns Chinese character levels to the Chinese characters it contains. The Chinese character levels include Level 1, Level 2, Level 3, and Level 4 Chinese characters. The order of Chinese character levels by frequency of use is Level 1 > Level 2 > Level 3 > Level 4. The Chinese character image library also assigns font levels to the fonts it contains. The font levels include Level 1, Level 2, and Level 3 fonts. The order of font levels by frequency of use is Level 1 > Level 2 > Level 3. The Chinese character image library also assigns symbol levels to the symbols it contains. The symbol levels include Level 1, Level 2, and Level 3 symbols. The order of symbol levels by frequency of use is Level 1 > Level 2 > Level 3.

[0060] 3. In step S2, the AI ​​processing module performs binarization on the target image to obtain the coordinates of each point in the target image in two-dimensional coordinates and their corresponding gray values, and then segments the target image according to the gray value of each coordinate point.

[0061] When the grayscale value of any coordinate point is less than the first grayscale value preset by the AI ​​processing module, the AI ​​processing module determines that the coordinate point corresponding to the current grayscale value is an invalid feature point. When the grayscale value of any coordinate point is greater than the second grayscale value preset by the AI ​​processing module, the AI ​​processing module determines that the coordinate point corresponding to the current grayscale value is an important feature point. When the grayscale value of any coordinate point is less than the second grayscale value but greater than the first grayscale value, the AI ​​processing module determines that the coordinate point corresponding to the current grayscale value is a valid feature point. Here, the first grayscale value < the second grayscale value, and the feature points are ranked according to their importance in terms of feature representation as follows: important feature point > valid feature point > invalid feature point.

[0062] The AI ​​processing module performs proportional distance judgment based on the coordinate spacing between effective feature points in the target image and the coordinate spacing between effective feature points in the Chinese character and symbol library. This is used to determine the cutting range of a single Chinese character and symbol in the target image. Based on the cutting range, the module cuts the single Chinese character and symbol in the target image to obtain the two-dimensional coordinates of the single Chinese character and symbol in the target image. Finally, based on the absolute value of the two-dimensional coordinates of the single Chinese character and symbol in the target image, the module obtains the font size of the single Chinese character and symbol in the target image.

[0063] 4. In step S2, when the AI ​​processing module cuts the target image, it uses each recognizable single Chinese character and single symbol as a minimum cutting unit to cut the target image, resulting in several cut graphics of single Chinese character groups and single symbol groups. The remaining unrecognizable Chinese characters and symbols are retained as blurred graphics. The AI ​​processing module marks all cut graphics and blurred graphics according to the original coordinate order of the target image and determines the relative positions of all cut graphics and blurred graphics to facilitate subsequent re-layout and re-entry.

[0064] 5. In step S2, the AI ​​processing module proportionally reduces the two-dimensional coordinate models in the Chinese character library and punctuation character library based on the coordinate values ​​corresponding to the font sizes of the Chinese characters and symbols in the target image, obtaining Chinese character graphic library font sizes consistent with the font sizes of the Chinese characters and symbols in the target image. Furthermore, based on the comparison results between the important and effective feature points of the Chinese characters and symbols in the target image and the important and effective feature points of the Chinese characters and symbols in the Chinese character graphic library, the module determines the font of a single Chinese character and the font of a single symbol in the target image.

[0065] When the AI ​​processing module determines the font of a single Chinese character in a target image, it sorts them in ascending order of frequency of use and compares them with the fonts in the Chinese character library level by level. That is, it first compares the first level fonts. If the font of the Chinese character still cannot be determined after the first level font comparison is completed, it then compares the second level fonts and the third level fonts in turn, until the font of the Chinese character is determined.

[0066] When the AI ​​processing module determines the content of a single Chinese character in a target image, it sorts the characters by their frequency of use in ascending order and compares them with the characters in the character library level by level. That is, it first compares the first-level characters. If the content of the Chinese character still cannot be determined after the comparison of the first-level characters, it then compares the second-level characters, the third-level characters, and the fourth-level characters in turn, until the content of the Chinese character is determined.

[0067] 6. In step S2, the AI ​​processing module determines the blurred image. Specifically, the AI ​​processing module separates the overlapping important and effective feature points, obtaining one overlapping Chinese character image and two non-overlapping Chinese character images. The important and effective feature points of the overlapping image are then restored to the missing parts of the characters in the two non-overlapping images. Based on the comparison results between the two restored non-overlapping images and the important and effective feature points of characters in the Chinese character image library, the module determines the corresponding Chinese character content for each of the two restored non-overlapping images.

[0068] When the AI ​​processing module determines the Chinese characters corresponding to the two restored non-overlapping Chinese character images, it compares them with the Chinese characters in the character library in ascending order of the frequency of use of the Chinese characters, from high to low. That is, it first compares the first level of Chinese characters. If it still cannot determine the Chinese character after the comparison of the first level of Chinese characters, it then compares the second level of Chinese characters, the third level of Chinese characters, and the fourth level of Chinese characters in turn, until the Chinese character is determined.

[0069] 7. In step S2, when the AI ​​processing module determines the Chinese character, font, or symbol in the target image that corresponds to the Chinese character image library, if there is no corresponding Chinese character, font, or symbol in the Chinese character image library, the AI ​​processing module determines that the Chinese character image library needs to be supplemented. It adds a corresponding color mark to the position of the two-dimensional coordinates in the target image of the Chinese character, so as to add a prompt background color when the Chinese character is output and re-entered, and outputs a prompt that the Chinese character image library needs to be supplemented, so as to facilitate manual verification and timely supplementation of the Chinese character image library.

[0070] 8. In step S3, the AI ​​processing module performs a correlation determination on the Chinese characters in the acquired target image to confirm the rationality of the Chinese characters in the target image. When the AI ​​processing module determines the correlation level of adjacent Chinese characters in the acquired target image, it compares the correlation levels with the correlation of Chinese characters in the Chinese character library in ascending order of frequency of use. That is, it first compares the first correlation level. After the first level of correlation comparison is completed, if the correlation of Chinese characters still cannot be determined, it compares the second, third and fourth correlation levels in sequence until the correlation of Chinese characters is determined. If the AI ​​processing module cannot determine the correlation of any Chinese character, the AI ​​processing module determines that there may be a typo or recognition error, and adds a corresponding color mark to the position of the Chinese character in the two-dimensional coordinates of the target image. This is used to add a prompt background color when the Chinese character is output and re-entered, so as to facilitate manual verification.

[0071] 9. In step S3, when the AI ​​processing module determines the coordinates, format, and encoding of the corresponding Chinese characters in the acquired target image, the AI ​​processing module outputs digital signals to the output unit based on the coordinates, format, and corresponding Chinese character coordinates, character number, and symbol number from the Chinese character image library in the acquired target image. The AI ​​processing module also outputs digital signals to the output unit based on the corresponding coordinate order and image ratio of the unrecognized Chinese character images in the target image.

[0072] The AI ​​processing module determines the layout format of the current target image based on the corresponding layout format in the layout library, and outputs the corresponding layout format digital signal. The layout format includes document type, table type, and specific layout coordinate position information. When the AI ​​processing module determines that there is no layout in the layout library that is the same as the layout format of the current target image, it determines the layout of the target image based on the edge range of the gray value of the target image, and selects the document type or table type format based on the layout of the target image to output the layout format digital signal with the corresponding proportion and tilt.

[0073] 10. In step S4, the AI ​​processing module outputs digital signals based on the relative coordinates of Chinese characters, Chinese character encoding, and format encoding in the target image to form a new electronic document. The digital signals include digital signals of tilt adjustment values ​​and corresponding coordinate points, digital signals of scale adjustment values ​​and corresponding coordinate points, digital signals of fonts and corresponding coordinate points, digital signals of symbols and corresponding coordinate points, digital signals of Chinese characters and corresponding coordinate points, digital signals of font size and corresponding coordinate points, digital signals of Chinese character images and corresponding coordinate points, and digital signals of color symbols and their corresponding coordinate points.

[0074] In this embodiment, the identification of the target image, the comparison of the icon image with the training library, and the deep learning method all use existing technical means. The specific selection of the technical means is determined according to the preset feature requirements and the distinguishability of the target image. The preset values ​​of the parameters are also set according to the user and usage requirements. This application protects an image identification method, therefore the specific parameter setting requirements are not disclosed, nor does it affect the specific implementation of this application in practical applications. The existing technical means can be unsupervised machine learning methods, supervised machine learning methods, or deep learning algorithms such as convolutional neural networks, as long as they can meet the needs of image recognition, comparison, and learning in this embodiment. This will not be elaborated further.

[0075] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

[0076] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An image restoration method based on image detection and region extraction, characterized by, Comprise: Step S1, establish a Chinese character pattern library; Step S2, the AI processing module obtains the binary grayscale coordinates of the target image, and identifies the Chinese characters, font size and font in the target image; Step S3, the AI processing module determines the relevance of the Chinese character content in the target image obtained to confirm the rationality of the Chinese character content in the target image, and determines the coordinates, format and encoding of the corresponding Chinese characters in the target image; Step S4, the AI processing module outputs according to the corresponding Chinese character coordinates, Chinese character encoding and format encoding in the target image to form a new electronic document; Step S5, the AI processing module performs deep learning according to the comparison process and comparison result of comparing the Chinese character pattern library with the target image, so as to improve the accuracy and efficiency of the AI processing module in identifying the target image; In step S3, the AI processing module determines the relevance of the Chinese character content in the target image obtained to confirm the rationality of the Chinese character content in the target image, wherein when the AI processing module determines the relevance level of the adjacent Chinese character content in the target image obtained, the relevance degrees in the Chinese character library are compared in ascending order from high to low according to the commonly used degree of the relevance degrees, wherein the first relevance degree is compared first, if the Chinese character relevance degree cannot be determined after the first relevance degree comparison, the second relevance degree, the third relevance degree and the fourth relevance degree are sequentially compared, until the Chinese character relevance degree is determined, if the AI processing module cannot determine any Chinese character relevance degree, the AI processing module determines that there may be a wrong character or an identification error, and adds the corresponding color mark to the position mark of the two-dimensional coordinates of the Chinese character in the target image, so as to add a prompt background color when the Chinese character is output and reentered, facilitating manual verification; In step S3, when the AI processing module determines the coordinates, format and encoding of the corresponding Chinese characters in the target image obtained, the AI processing module outputs the digital signals of the Chinese character coordinates, Chinese character number and symbol number in the Chinese character pattern library corresponding to the Chinese character coordinates and format in the target image obtained to the output unit, and forms the digital signals of the corresponding coordinate order and image proportion according to the Chinese character image in the target image which is not identified and outputs them to the output unit, wherein the AI processing module determines the layout format of the current target image according to the corresponding layout format in the layout library, and outputs the digital signals of the corresponding layout format according to the layout format of the current target image, the layout format includes document type, table type and specific layout coordinate position information, when the AI processing module determines that there is no layout format same as the layout format of the current target image in the layout library, it determines the layout of the target image according to the edge range of the grayscale value of the target image, and selects the format of the document type or the table type based on the layout of the target image to output the digital signals of the corresponding proportion and inclination of the layout format.

2. The image restoration method based on image detection and region extraction according to claim 1, wherein in step S1, the Chinese character pattern library comprises a Chinese character library and a punctuation character library, the Chinese character library records Chinese characters of several fonts, the punctuation character library records symbols including alphabets, numbers, operation symbols, punctuation symbols and functional symbols, the Chinese character library numbers each recorded Chinese character, the punctuation character library numbers each recorded symbol, the Chinese character library numbers each font of each recorded Chinese character and marks the relative coordinates of each font of each recorded Chinese character in a two-dimensional coordinate system to obtain a two-dimensional coordinate model of each font of each Chinese character, the punctuation character library numbers each font of each recorded symbol and marks the relative coordinates of each font of each recorded symbol in a two-dimensional coordinate system to obtain a two-dimensional coordinate model of each font of each symbol; The Chinese character pattern library further comprises word relations and phrase relations of all Chinese characters in the Chinese character library, and the hierarchical degrees of association between characters and characters, between characters and words, between words and words, and between phrases are obtained according to the word relations and phrase relations of the Chinese characters, respectively, to obtain a first degree of association, a second degree of association, a third degree of association, and a fourth degree of association, wherein, the correlation degrees are sorted according to the frequency of use as first correlation degree > second correlation degree > third correlation degree > fourth correlation degree, and the correlation degrees are used to determine the Chinese characters that cannot be recognized by the Chinese character library according to the frequency of use; the Chinese character pattern library further comprises a layout library, the layout library is used to store original format layouts corresponding to the target image, including a layout without Chinese characters and a layout with Chinese characters, and is used to store the layout types of Chinese characters associated with the target image and the corresponding numbers and establish a two-dimensional coordinate model corresponding to the layout types of Chinese characters in a two-dimensional coordinate system; the Chinese character pattern library assigns a Chinese character level to each recorded Chinese character, the Chinese character level comprises a first level Chinese character, a second level Chinese character, a third level Chinese character and a fourth level Chinese character, the Chinese character level is sorted according to the frequency of use as first level Chinese character > second level Chinese character > third level Chinese character > fourth level Chinese character, the Chinese character pattern library further assigns a font level to each recorded font, the font level comprises a first level font, a second level font and a third level font, and the font level is sorted according to the frequency of use as first level font > second level font > third level font, the Chinese character pattern library further assigns a symbol level to each recorded symbol, the symbol level comprises a first level symbol, a second level symbol and a third level symbol, and the symbol level is sorted according to the frequency of use as first level symbol > second level symbol > third level symbol.

3. The image restoration method based on image detection and region extraction according to claim 2, characterized in that, In step S2, the AI processing module binarizes the target image, obtains each coordinate point and its corresponding gray value in the two-dimensional coordinates of the target image, and cuts the target image according to the gray value of each coordinate point. When the gray value of any coordinate point is less than the first gray value preset by the AI processing module, the AI processing module determines that the coordinate point corresponding to the current gray value is an invalid feature point. When the gray value of any coordinate point is greater than the second gray value preset by the AI processing module, the AI processing module determines that the coordinate point corresponding to the current gray value is an important feature point. When the gray value of any coordinate point is less than the second gray value and greater than the first gray value, the AI processing module determines that the coordinate point corresponding to the current gray value is a valid feature point. The first gray value < the second gray value, and the important feature points, the valid feature points, and the invalid feature points are ranked according to the importance of feature expression in descending order. The AI processing module respectively judges the cutting range of single Chinese characters and single symbols in the target image according to the coordinate distance between the valid feature points of the target image and the coordinate distance between the valid feature points of Chinese characters and symbols in the Chinese character graph library, cuts the single Chinese characters and single symbols in the target image according to the cutting range, obtains the two-dimensional coordinates of the single Chinese characters and single symbols in the target image, and obtains the font size of the single Chinese characters and single symbols in the target image according to the absolute value of the two-dimensional coordinates of the single Chinese characters and single symbols in the target image.

4. The image restoration method based on image detection and region extraction according to claim 3, characterized in that, In step S2, when the AI processing module cuts the target image, the recognizable single Chinese characters and single symbols are taken as a minimum cutting unit to cut the target image, and a plurality of cutting graphs of single Chinese character groups and single symbol groups are obtained. The remaining Chinese characters and symbols that cannot be recognized are taken as fuzzy graphs and are reserved. The AI processing module labels all cutting graphs and fuzzy graphs according to the original coordinate order of the target image, and determines the relative positions of all cutting graphs and fuzzy graphs.

5. The image restoration method based on image detection and region extraction according to claim 4, characterized in that, In step S2, the AI processing module reduces the two-dimensional coordinate models in the Chinese character library and the punctuation character library by the same proportion according to the coordinate values corresponding to the font sizes of Chinese characters and symbols in the target image, obtains the font sizes of the Chinese character graph library consistent with the font sizes of Chinese characters and symbols in the target image, and judges the font of single Chinese characters and the font of single symbols in the target image according to the comparison results of the important feature points and the valid feature points in the target image and the important feature points and the valid feature points in the Chinese character graph library. When the AI processing module judges the font of single Chinese characters in the target image according to the font level, the fonts in the Chinese character library are compared in ascending order of usage frequency from high to low. First, the first-level font is compared. If the font of the Chinese character cannot be determined after the comparison of the first-level font, the second-level font and the third-level font are compared in turn until the font of the Chinese character is determined. When the AI processing module determines the single Chinese character content in the target image according to the Chinese character level, the Chinese characters in the Chinese character library are compared in ascending order of the commonly used degree from high to low, and the first-level Chinese characters are compared first. If the Chinese character content cannot be determined after the comparison of the first-level Chinese characters, the second-level Chinese characters, the third-level Chinese characters, and the fourth-level Chinese characters are sequentially compared until the Chinese character content is determined.

6. The image restoration method based on image detection and region extraction according to claim 5, characterized in that, In step S2, the AI processing module determines the fuzzy graph, wherein the AI processing module splits the overlapped important feature points and effective feature points to obtain one Chinese character overlapping image and two non-overlapping Chinese character images. The important feature points and the effective feature points of the Chinese character overlapping image are restored in the missing parts of the Chinese characters of the two non-overlapping Chinese character images, respectively. The Chinese character contents corresponding to the two restored non-overlapping Chinese character images are determined according to the comparison results of the important feature points and the effective feature points of the two restored non-overlapping Chinese character images with the Chinese characters in the Chinese character graph library. When the AI processing module determines the Chinese characters corresponding to the two restored non-overlapping Chinese character images, the Chinese characters in the Chinese character library are compared in ascending order of the commonly used degree of the Chinese character level from high to low. The first-level Chinese characters are compared first. If the Chinese character content cannot be determined after the comparison of the first-level Chinese characters, the second-level Chinese characters, the third-level Chinese characters, and the fourth-level Chinese characters are sequentially compared until the Chinese character content is determined.

7. The image restoration method based on image detection and region extraction according to claim 6, characterized in that, In step S2, when the AI processing module determines the Chinese characters, fonts, and symbols corresponding to the Chinese character graph library in the target image, if there is no corresponding Chinese character, font, or symbol in the Chinese character graph library, the AI processing module determines that the Chinese character graph library needs to be supplemented. The position of the Chinese character in the two-dimensional coordinates of the target image is added with a corresponding color mark to add a prompt background color when the Chinese character is output and re-entered. The prompt that the Chinese character graph library needs to be supplemented is output to facilitate manual verification and timely supplement of the Chinese character graph library.

8. The image restoration method based on image detection and region extraction according to claim 7, characterized in that, In step S4, the AI processing module outputs the digital signals corresponding to the Chinese character coordinates, Chinese character encoding, and format encoding in the target image to form a new electronic document. The digital signals include the inclination adjustment value and the corresponding coordinate point digital signal, the proportion adjustment value and the corresponding coordinate point digital signal, the font and the corresponding coordinate point digital signal, the symbol and the corresponding coordinate point digital signal, the Chinese character and the corresponding coordinate point digital signal, the font size and the corresponding coordinate point digital signal, the Chinese character image and the corresponding coordinate point digital signal, and the color mark and the corresponding coordinate point digital signal.

Images