Super-resolution reconstruction method and device, electronic equipment and storage medium

Abstract

The invention relates to a super-resolution reconstruction method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring an original image, and performing feature extraction on original characters contained in the original image to obtain a position feature vector and a semantic feature vector of the original characters; target characters matched with the semantic feature vectors are retrieved in candidate characters contained in a dictionary library, and target character structure feature vectors corresponding to the target characters and recorded in the dictionary library are determined; and inputting the original image, the position feature vector and the target character structure feature vector into a pre-trained super-resolution reconstruction model, and obtaining a reconstructed image output by the super-resolution reconstruction model, the reconstructed image including an optimized character obtained by performing super-resolution reconstruction on the original character. By means of the technical scheme, the accuracy and readability of the optimized characters obtained through reconstruction can be effectively improved.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to a super-resolution reconstruction method and device, electronic equipment, and storage medium. Background Art

[0002] Scene text understanding plays a crucial role in computer vision tasks. Currently, the cameras used in terminal devices are limited by hardware size and cost, resulting in low-resolution images. The shapes and outlines of text in such low-resolution images are often blurred, hindering scene text understanding.

[0003] In the related art, a text prior branch is added to the image super-resolution branch to solve the above problem. Among them, the text prior branch is used to extract the sequence feature vector of the text in the original image. The sequence feature vector is fused with the image feature vector output by the image super-resolution branch to improve the super-resolution reconstruction effect of the text area in the original image. However, the text structure information contained in the sequence feature vector of the above text is not necessarily accurate. For example, for the adhesion text contained in the original image, the text with partial structural missing and the text with low clarity, the text structure information contained in the extracted sequence feature vector is fuzzy and incomplete. Therefore, super-resolution reconstruction based on the fuzzy and incomplete sequence feature vector is likely to lead to the inaccuracy of the optimized text after reconstruction, which greatly reduces the readability of the optimized text, which is not conducive to achieving a better scene text understanding effect. Summary of the Invention

[0004] The present disclosure provides a super-resolution reconstruction method and device, an electronic device, and a storage medium to address the deficiencies in the related art.

[0005] According to a first aspect of the embodiments of the present disclosure, a super-resolution reconstruction method is proposed, comprising:

[0006] Acquire an original image, and perform feature extraction on original text contained in the original image to obtain a position feature vector and a semantic feature vector of the original text;

[0007] Retrieving a target character that matches the semantic feature vector from candidate characters contained in a dictionary library, and determining a target character structural feature vector corresponding to the target character recorded in the dictionary library;

[0008] The original image, the position feature vector, and the target text structure feature vector are input into a pre-trained super-resolution reconstruction model, and a reconstructed image output by the super-resolution reconstruction model is obtained. The reconstructed image includes optimized text obtained by super-resolution reconstruction of the original text.

[0009] Optionally, the method further includes: enhancing the contrast of the original image to obtain a contrast-enhanced image; and extracting features of the original text contained in the original image includes: extracting features of the original text contained in the contrast-enhanced image.

[0010] Optionally, the feature extraction of the original text contained in the original image to obtain the position feature vector of the original text includes: performing positioning detection on the original text to obtain the positioning point feature vector of the original text; connecting the positioning point feature vectors to obtain a closed positioning frame feature vector, and the position feature vector includes the positioning frame feature vector.

[0011] Optionally, the method further includes: detecting whether the positioning box represented by the positioning box feature vector covers the original text, and when detecting that the positioning box does not cover the original text, correcting the positioning point feature vector so that the positioning box corresponding to the corrected positioning point feature vector covers the original text.

[0012] Optionally, before extracting the semantic feature vector of the original text, the method further includes: performing a scale transformation on the original text so that the scale of the transformed text meets the preset scale requirement; and / or, when the direction of the original text is an inclined direction, correcting the direction of the original text.

[0013] Optionally, the dictionary library also includes a candidate text style feature vector corresponding to the candidate text, and the method also includes: extracting the text style feature vector of the original text; searching for a target text that matches the semantic feature vector from the candidate text contained in the dictionary library includes: searching for a target text that matches both the semantic feature vector and the text style feature vector in the dictionary library.

[0014] Optionally, the method further includes: converting the target text structural feature vector to obtain verification text, and calculating the similarity between the verification text and the original text; inputting the original image, the position feature vector, and the target text structural feature vector into a pre-trained super-resolution reconstruction model, including: when the similarity is greater than a similarity threshold, inputting the original image, the position feature vector, and the target text structural feature vector into the pre-trained super-resolution reconstruction model.

[0015] Optionally, the method further includes: extracting a clarity feature vector of the original text, and inputting the clarity feature vector into the super-resolution reconstruction model, so that the super-resolution reconstruction model performs super-resolution reconstruction on the original text by performing a processing operation corresponding to the clarity feature vector; wherein, when the clarity feature vector indicates that the clarity of the original text is greater than or equal to a first clarity threshold, the processing operation includes improving the clarity of the original text to obtain the optimized text; when the clarity feature vector indicates that the clarity of the original text is less than the first clarity threshold and greater than a second clarity threshold, the processing operation includes generating the optimized text according to the target text structure feature vector; when the clarity feature vector indicates that the clarity of the original text is less than or equal to the second clarity threshold, the processing operation includes improving the clarity and sharpness of the original text to obtain the optimized text.

[0016] Optionally, the super-resolution reconstruction model includes a text processing branch and an image processing branch, wherein the text processing branch is used to perform super-resolution reconstruction on the original text and output the optimized text; the image processing branch is used to perform super-resolution reconstruction on the area other than the original text in the original image and output the reconstructed area; the reconstructed image is obtained by combining the reconstructed area and the optimized text.

[0017] Optionally, the method further includes: performing background compensation on the text area where the optimized text is located, so that the background attributes of the compensated text area are the same as the background attributes of the non-text area in the reconstructed image.

[0018] According to a second aspect of the embodiments of the present disclosure, a super-resolution reconstruction apparatus is provided, comprising:

[0019] An extraction unit, configured to obtain an original image and perform feature extraction on original text contained in the original image to obtain a position feature vector and a semantic feature vector of the original text;

[0020] a matching unit, configured to search for a target character that matches the semantic feature vector from candidate characters contained in a dictionary library, and determine a target character structural feature vector corresponding to the target character recorded in the dictionary library;

[0021] An input unit is used to input the original image, the position feature vector, and the target text structure feature vector into a pre-trained super-resolution reconstruction model, and obtain a reconstructed image output by the super-resolution reconstruction model, wherein the reconstructed image includes optimized text obtained by super-resolution reconstruction of the original text.

[0022] According to a third aspect of the embodiments of the present disclosure, an electronic device is provided, including:

[0023] processor;

[0024] a memory for storing processor-executable instructions;

[0025] The processor is configured to implement the method described in the embodiment of the first aspect above.

[0026] According to a fourth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the steps of the method described in the embodiments of the first aspect are implemented.

[0027] The technical solutions provided by the embodiments of the present disclosure may have the following beneficial effects:

[0028] As can be seen from the above embodiments, the present disclosure extracts the semantic feature vector of the original text in the original image, then searches for the target text that matches the semantic feature vector in a dictionary library containing multiple candidate texts, and then uses the target text structural feature vector corresponding to the target text recorded in the dictionary library to perform super-resolution reconstruction. Since the candidate texts contained in the dictionary library are clear and accurate standard texts, and the dictionary library records the complete and accurate candidate text structural feature vectors corresponding to the candidate texts (the candidate text structural feature vectors can fully and accurately reflect the structural information of the candidate texts), the above super-resolution reconstruction method extracts the complete and accurate text structural feature vectors from the dictionary library and performs super-resolution reconstruction based on the complete and accurate text structural feature vectors, thereby effectively improving the accuracy of the reconstructed optimized text and thus improving the readability of the optimized text. Moreover, for original text with special cases such as adhesion, partial structural loss, and similar text structures, the accurate target text can also be matched in the dictionary library based on the semantic feature vector of the original text, and then super-resolution reconstruction can be performed based on the target text structural feature vector corresponding to the target text to obtain accurate optimized text, thereby effectively improving the super-resolution reconstruction effect of the original text in the above special cases.

[0029] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0031] Figure 1 is a schematic diagram of a super-resolution reconstruction system according to an embodiment of the present disclosure.

[0032] Figure 2 is a schematic flowchart of a super-resolution reconstruction method according to an embodiment of the present disclosure.

[0033] Figure 3 This is a schematic diagram showing a method of correcting a positioning frame according to an embodiment of the present disclosure.

[0034] Figure 4 The figure is a flowchart showing a method of searching in a dictionary according to an embodiment of the present disclosure.

[0035] Figure 5 It is a schematic diagram showing multiple text styles of the same text according to an embodiment of the present disclosure.

[0036] Figure 6 The figure is a flow chart showing a method for realizing super-resolution reconstruction based on a diffusion model according to an embodiment of the present disclosure.

[0037] Figure 7a It is a flowchart of a super-resolution reconstruction method according to an embodiment of the present disclosure.

[0038] Figure 7b is a schematic diagram showing an original image according to an embodiment of the present disclosure.

[0039] Figure 7c is a schematic diagram showing a reconstructed image according to an embodiment of the present disclosure.

[0040] Figure 8 This is a schematic block diagram of a super-resolution reconstruction device according to an embodiment of the present disclosure.

[0041] Figure 9 is a schematic block diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0042] The following will be combined with the accompanying drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the embodiments described are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of the present disclosure.

[0043] The terms used in the embodiments of the present disclosure are only for the purpose of describing specific embodiments, and are not intended to limit the embodiments of the present disclosure. The singular forms "a" and "the" used in the embodiments of the present disclosure and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0044] It should be understood that although the terms first, second, third, etc. may be used in the embodiments of the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the embodiments of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to determining".

[0045] For the purposes of simplicity and ease of understanding, the terms "greater than" or "less than", "higher than" or "lower than" are used herein to characterize size relationships. However, for those skilled in the art, it can be understood that the term "greater than" also encompasses the meaning of "greater than or equal to", and "less than" also encompasses the meaning of "less than or equal to"; the term "higher than" encompasses the meaning of "higher than or equal to", and "lower than" also encompasses the meaning of "lower than or equal to".

[0046] In the related art, a text prior branch is added outside the image super-resolution branch to achieve super-resolution reconstruction of text. Among them, the text prior branch is used to extract the sequence feature vector of the text in the original image. The sequence feature vector is fused with the image feature vector output by the image super-resolution branch to improve the super-resolution reconstruction effect of the text area in the original image. However, the text structure information contained in the above sequence feature vector of the text is not necessarily accurate. For example, for the adhesive text, the text with partial structure missing, and the text with low clarity contained in the original image, the text structure information contained in the extracted sequence feature vector is fuzzy and incomplete. For example, if the two horizontal lines in the middle of the original text "目" in the original image are adhered together, it is easy to cause the extracted text structure information to be the text structure information of the character "日". Obviously, the extracted text structure information is incorrect. It can be seen that based on such fuzzy, incomplete or even incorrect sequence feature vectors for super-resolution reconstruction, it is easy to cause the optimized text obtained by the reconstruction to be inaccurate, thereby greatly reducing the readability of the optimized text and being unfavorable for achieving a good scene text understanding effect.

[0047] Based on this, the present disclosure proposes a super-resolution reconstruction scheme, which can effectively improve the accuracy and readability of the optimized text.

[0048] Next, one or more embodiments of the present disclosure are described in detail.

[0049] The embodiments of the present disclosure can be applied to electronic devices, which may include but are not limited to smartphones, desktop computers, tablet computers, laptop computers, e-book readers, smart watches, smart bracelets, etc. One or more embodiments of the present disclosure are not limited to this. During operation, the electronic device can run a super-resolution reconstruction system to achieve the purpose of super-resolution reconstruction. Among them, the application of the super-resolution reconstruction system can be pre-installed on the electronic device, so that the super-resolution reconstruction system can be started and run on the electronic device; of course, when using technologies such as HTML5, there is no need to install the corresponding application on the electronic device to obtain and run the super-resolution reconstruction system.

[0050] In another embodiment, the super-resolution reconstruction system may include Figure 1 The server 11, network 12, and electronic device 13 are shown. During operation, the server 11 can run the server-side program of the super-resolution reconstruction system to implement related data processing and other functions; while the electronic device 13 can run the client-side program of the super-resolution reconstruction system to implement related display, human-computer interaction, and other functions, thereby cooperating between the server 11 and the electronic device 13 to implement the super-resolution reconstruction solution. It should be noted that the server 11 can also communicate with multiple electronic devices to implement super-resolution reconstruction on multiple electronic devices.

[0051] The server 11 may be a physical server comprising an independent host, or the server 11 may be a virtual server hosted by a host cluster. The electronic device 13 may be any of the following types of devices: PCs, tablet devices, laptops, PDAs (Personal Digital Assistants), mobile phones, wearable devices (such as smart glasses, smart watches, etc.), etc., and one or more embodiments of this specification are not limited to this. The network 12 for interaction between the electronic device 13 and the server 11 may include various types of wired or wireless networks. In one embodiment, the network 12 may include a public switched telephone network (PSTN) and the Internet. The client application of the super-resolution reconstruction system may be pre-installed on the electronic device so that the client can be started and run on the electronic device; of course, when an online "client" such as HTML5 technology is used, the client can be obtained and run without installing the corresponding application on the electronic device.

[0052] Figure 2 is a schematic flowchart of a super-resolution reconstruction method according to an embodiment of the present disclosure.

[0053] like Figure 2 As shown, the method may include the following steps:

[0054] S201: Acquire an original image, and perform feature extraction on original text contained in the original image to obtain a position feature vector and a semantic feature vector of the original text.

[0055] In this embodiment, the original image is an image containing text. The original image needs to be super-resolution reconstructed. After acquiring the original image, feature extraction is performed on the original text contained in the original image to obtain the position feature vector and semantic feature vector of the original text. Among them, the position feature vector can reflect the specific position of the original text in the original image. Super-resolution reconstruction based on the position feature vector can ensure that the position of the optimized text obtained by reconstruction in the reconstructed image is consistent with the position of the original text in the original image, thereby improving the display effect of the reconstructed image. There are many methods for extracting position feature vectors, such as extracting the coordinate information of the text boundary box as the position feature vector, or extracting the edge and texture information of the text as the position feature vector, etc., and the present disclosure is not limited to this.

[0056] After extracting the position feature vector, the position of the original text can be determined based on the position feature vector, and then the semantic feature vector of the original text can be extracted at the determined position. The semantic feature vector is used to reflect the specific meaning of the original text and the connection between the original text. The semantic feature vector of the original text can be extracted through models such as BERT (Bidirectional Encoder Representations from Transformers), RNN (Recurrent Neural Network) or CNN (Convolution Neural Network), and the present disclosure is not limited to this. In addition, when using a neural network to extract features from the original text, you can choose to use one neural network to extract features to simultaneously output the position feature vector and semantic feature vector of the original text, or you can choose to use different neural networks to output the position feature vector and semantic feature vector of the original text respectively.

[0057] In one embodiment, before extracting features from the original text, the original image may be preprocessed, namely, the contrast of the original image may be enhanced to obtain a contrast-enhanced image. By increasing the contrast of the original image, the contrast difference between the text area containing the original text and the non-text area (excluding the original text) in the contrast-enhanced image can be increased, thereby highlighting the text area. This facilitates subsequent feature extraction of the original text within the text area based on the contrast, helping to improve the efficiency and accuracy of feature extraction and avoid missing or incorrect detection of the original text.

[0058] In one embodiment, a specific method for extracting the position feature vector of the original text may include: performing positioning detection on the original text, determining multiple positioning points, and converting the information of the multiple positioning points into a positioning point feature vector. The positioning point feature vectors are then connected to obtain a closed positioning frame feature vector. The positioning frame corresponding to the positioning frame feature vector is used to cover the original text. In other words, the positioning frame feature vector can reflect the position information of the original text. The positioning frame described in this embodiment can be a rectangular positioning frame (which can be determined by four positioning points), or a triangular positioning frame (which can be determined by three positioning points), or a positioning frame of other shapes. Those skilled in the art can set it themselves according to actual needs, and this disclosure is not limited to this.

[0059] In this embodiment, by performing positioning detection on the original text and using the positioning frame feature vector composed of the positioning point feature vector as the position feature vector of the original text, it can be achieved that each original text corresponds to a positioning frame (that is, one positioning frame covers one original text), thereby improving the accuracy of the extracted position feature vector of each original text and avoiding missed or incorrect detection of the original text.

[0060] In one embodiment, after obtaining the positioning frame feature vector of the original text, the positioning frame feature vector can be verified. Specifically, it is detected whether the positioning frame represented by the positioning frame feature vector covers the original text. If it is detected that the positioning frame does not cover the original text, the positioning point feature vector needs to be corrected. The positioning point feature vector can be corrected based on the color and texture distribution around the original text to ensure that the positioning frame corresponding to the corrected positioning point feature vector can cover the original text. The positioning frame corresponding to the corrected positioning point feature vector can be determined based on the corrected positioning frame feature vector composed of the corrected positioning point feature vectors. Figure 3 This is a schematic diagram showing a method of correcting a positioning frame according to an embodiment of the present disclosure. Figure 3The left diagram in shows the positioning box A corresponding to the positioning point feature vector obtained by initial positioning. This positioning box A (represented by a dashed box in the figure) is determined by four positioning points. Obviously, the positioning box A truncates the original character "生" and fails to completely cover the original character "生". Therefore, it is necessary to correct the positioning point feature vector in the left diagram according to the color and texture distribution around the original character "生". It should be noted that the positioning point feature vector corresponding to one or more positioning points in the left diagram can be corrected, as long as the positioning box corresponding to the corrected positioning point feature vector can cover the original character. Figure 3 The right diagram in shows the positioning box B corresponding to a corrected positioning point feature vector. Obviously, the positioning box B can cover the original character "生", that is, the original character "生" can be completely contained within the positioning box B.

[0061] In this embodiment, when it is detected that the positioning box does not cover the original character, the positioning point feature vector is corrected so that the positioning box corresponding to the corrected positioning point feature vector can cover the original character, thereby ensuring the integrity of the original character within the positioning box, which helps to extract complete and accurate semantic feature vectors based on the complete original character, and then perform the subsequent super-resolution reconstruction process based on the complete and accurate semantic feature vectors, which can effectively improve the accuracy of the optimized characters obtained by reconstruction.

[0062] In one embodiment, after extracting the position feature vector of the original character, the original character can be corrected first, and then the semantic feature vector of the corrected original character can be extracted. Correcting the original character can include: performing scale transformation on the original character and / or correcting the direction of the original character. Among them, performing scale transformation on the original character can include transforming the height and width of the original character, so that the height and width of the transformed original character can meet the preset scale requirements. For example, transforming the original character of 256×256 into a character of 16×64 helps to improve the efficiency and accuracy of subsequent semantic feature vector extraction.

[0063] In addition, the direction of some original characters in the original image is an inclined direction. If the semantic feature vector of such an inclined original character is directly extracted, it is easy to cause errors in the extracted semantic feature vector. Therefore, when it is detected that the direction of the original character is an inclined direction, the direction of the original character needs to be corrected so that the direction of the original character can be parallel to the horizontal direction or the vertical direction. Furthermore, based on the corrected original character, the extraction of the semantic feature vector can ensure the accuracy of the extracted semantic feature vector.

[0064] S202: Retrieve the target character that matches the semantic feature vector from the candidate characters contained in the dictionary library, and determine the target character structure feature vector recorded in the dictionary library corresponding to the target character.

[0065] The dictionary library contains multiple candidate characters and the corresponding candidate character structure feature vectors. Among them, the candidate characters are all clear and accurate standard characters. Therefore, the candidate character structure feature vectors are also complete and accurate character structure feature vectors, which can accurately reflect the structure information of the candidate characters. The structure information of the characters may include the stroke structure of the characters, etc. In this embodiment, first, a target character that matches the semantic feature vector of the original character is retrieved from the candidate characters contained in the dictionary library, that is, the target character that matches is determined according to the semantics of the original character first, and then the target character structure feature vector corresponding to the target character is determined from the candidate character structure feature vectors.

[0066] Figure 4 It is a schematic flowchart of a retrieval process in a dictionary library shown according to an embodiment of the present disclosure. As Figure 4 shown, there are partial missing parts in the original character, and the extracted semantic feature vector is semantic feature vector a. According to the semantic feature vector a, it can be determined that the semantics of the original character belong to "sheng". Therefore, when retrieving in the dictionary library according to the semantic feature vector a, the candidate character "sheng" can be retrieved to match the semantic feature vector a. That is to say, "sheng" is the target character. Then, the target character structure feature vector b corresponding to "sheng" is determined from the candidate character structure feature vectors contained in the dictionary library. Since the target character structure feature vector b corresponding to "sheng" is complete, clear, and accurate, super-resolution reconstruction based on the target character structure feature vector b can effectively improve the accuracy of the optimized character obtained by reconstruction.

[0067] In one embodiment, the same character can have multiple different character styles. The character styles may include but are not limited to character color, character size, character font, etc. Figure 5 It is a schematic diagram of multiple character styles of the same character shown according to an embodiment of the present disclosure. Figure 5 shows the character "sheng" in three different character styles. Among them, character style 1 is bold + 10pt, character style 2 is Song typeface + 21pt, and character style 3 is artistic word.

[0068] Based on this, the dictionary library can also include candidate characters in multiple character styles and the corresponding candidate character style feature vectors for each character style. When extracting features of the original character, the character style feature vector of the original character can also be extracted. Then, a target character that simultaneously matches the semantic feature vector and the character style feature vector of the original character is retrieved from the candidate characters contained in the dictionary library. Combined with Figure 5, assume that the dictionary library contains three Chinese characters '生' in font styles 1-3 (these three Chinese characters '生' are all candidate characters). The font style feature vector c of the extracted original text indicates that the font style of the original text is Song typeface + Initial size. Then, when retrieving in the dictionary library, first retrieve the three Chinese characters '生' in Figure 5 all match the semantic feature vector a, and then determine the Chinese character '生' corresponding to font style 2 as the target character according to the font style feature vector c of the original text.

[0069] In addition, since the font style feature vector obtained by feature extraction of the original text is usually complete and accurate, even if there are special situations such as partial missing, adhesion, and similar structures in the original text, it usually does not affect the accuracy of the font style feature vector of the original text. Therefore, the dictionary library can contain only candidate characters of one font style. In this case, there is no need to retrieve the target character in the dictionary library that matches the font style feature vector of the original text. Instead, the extracted font style feature vector and the target character structure feature vector determined in the dictionary library can be directly input into the subsequent super-resolution reconstruction model, so that the super-resolution reconstruction model generates optimized text that matches both the font style feature vector and the target character structure feature vector. For example, the dictionary library only contains the candidate character '生' in the font style of Bold typeface + Small No. 2. And the extracted font style feature vector indicates that the font style of the original text is Song typeface. In this case, first retrieve the target character '生' (font style is Bold typeface + Small No. 2) according to the semantic feature vector of the original text, and determine the target character structure feature vector corresponding to the target character '生'. Then input the extracted font style feature vector (i.e., Song typeface) and the target character structure feature vector of the target character '生' into the super-resolution reconstruction model. The super-resolution reconstruction model can output the Chinese character '生' in the font style of Song typeface + Small No. 2.

[0070] In this embodiment, by extracting the font style feature vector of the original text and then performing super-resolution reconstruction based on this font style feature vector and the target character structure feature vector. On the one hand, it can ensure that the font style of the optimized text obtained by super-resolution reconstruction is consistent with the font style of the original text, that is, it can accurately generate optimized text with the same color, size, and font as the original text, thus improving the display effect of the optimized text. On the other hand, it realizes super-resolution reconstruction of original text with different font styles, effectively improving the generality of the super-resolution reconstruction method.

[0071] In one embodiment, after searching a dictionary for a target text that matches both the semantic feature vector and the text style feature vector of the original text, the target text structural feature vector corresponding to the target text, recorded in the dictionary, is determined. The target text structural feature vector is then converted to obtain a verification text. The similarity between the verification text and the original text is then calculated. If the similarity between the verification text and the original text exceeds a similarity threshold, the target text structural feature vector corresponding to the verification text is determined to be accurate. The original image, the target text structural feature vector, and the original text positional feature vector can then be input into a super-resolution reconstruction model for reconstruction.

[0072] If the similarity between the verification text and the original text is less than or equal to the similarity threshold, the target text structural feature vector corresponding to the verification text can be determined to be incorrect, and subsequent super-resolution reconstruction is unnecessary. This situation often occurs when the target text structural feature vector is incorrect for original text with extremely low clarity, which can be understood as text that is unrecognizable to the human eye or optical character recognition (OCR). Because the original text has extremely low clarity, it is impossible to identify and extract relatively accurate semantic features, which can easily lead to the retrieval of an incorrect target text structural feature vector from the dictionary library.

[0073] In this embodiment, after determining the target text structural feature vector, the target text structural feature vector is further verified, that is, the similarity between the verification text obtained by converting the target text structural feature vector and the original text is calculated, thereby avoiding super-resolution reconstruction of the erroneous target text structural feature vector and further improving the accuracy of the reconstructed optimized text.

[0074] S203: Input the original image, the position feature vector, and the target text structure feature vector into a pre-trained super-resolution reconstruction model, and obtain a reconstructed image output by the super-resolution reconstruction model, wherein the reconstructed image includes optimized text obtained by super-resolution reconstruction of the original text.

[0075] The super-resolution reconstruction model can perform super-resolution reconstruction on the input image, reconstructing a low-resolution image into a high-resolution image, thereby improving image quality and display quality. In this embodiment, the super-resolution reconstruction model can output a reconstructed image based on the input original image, the target text structural feature vector, and the original text position feature vector. The reconstructed image includes the optimized text reconstructed based on the target text structural feature vector, and the position of the optimized text in the reconstructed image matches the original text position feature vector.

[0076] There are many super-resolution reconstruction models, such as SRGAN (Super-Resolution Generative Adversarial Network), EDSR (Enhanced Deep Super-Resolution), and super-resolution reconstruction methods based on diffusion models, etc., which are not limited in this disclosure. Figure 6 FIG. 1 is a flow chart showing a method for realizing super-resolution reconstruction based on a diffusion model according to an embodiment of the present disclosure. Figure 6 As shown, when super-resolution reconstruction is achieved based on the diffusion model, the original image input to the diffusion model (represented by Unet in the figure) needs to be denoised. Specifically, the encoder in the VAE (Variational Auto-Encoder) is used to encode the original image to obtain the corresponding feature map, and then noise is added to the feature map to obtain a noisy feature map. The noisy feature map, the target text structure feature vector and the position feature vector of the original text are then input into the diffusion model so that the diffusion model can perform super-resolution reconstruction and denoising on the feature map. That is, the diffusion model will output a denoised reconstructed feature map. The decoder in the VAE is then used to decode the denoised reconstructed feature map to obtain a reconstructed image. The reconstructed image includes the optimized text reconstructed according to the target text structure feature vector.

[0077] In the above embodiment, the semantic feature vector of the original text in the original image is extracted, and then a target text that matches the semantic feature vector is searched in a dictionary library containing multiple candidate texts, and then super-resolution reconstruction is performed using the target text structural feature vector corresponding to the target text recorded in the dictionary library. Since the candidate texts contained in the dictionary library are clear and accurate standard texts, and the dictionary library records complete and accurate candidate text structural feature vectors corresponding to the candidate texts (the candidate text structural feature vectors can fully and accurately reflect the structural information of the candidate texts), the above super-resolution reconstruction method effectively improves the accuracy of the reconstructed optimized text by extracting complete and accurate text structural feature vectors from the dictionary library and performing super-resolution reconstruction based on the complete and accurate text structural feature vectors, thereby improving the readability of the optimized text. Moreover, for original text with special cases such as adhesion, partial structural loss, and similar text structures, the accurate target text can also be matched in the dictionary library based on the semantic feature vector of the original text, and then super-resolution reconstruction can be performed based on the target text structural feature vector corresponding to the target text to obtain accurate optimized text, thereby effectively improving the super-resolution reconstruction effect of the original text in the above special cases.

[0078] In one embodiment, different original texts have different clarity, and the original texts can be divided into three categories according to the clarity: ① original texts with clarity greater than or equal to a first clarity threshold (such original texts have higher clarity); ② original texts with clarity less than the first clarity threshold and greater than a second clarity threshold (such original texts can be recognized by the human eye or OCR technology); ③ original texts with clarity less than or equal to the second clarity threshold (such original texts cannot be recognized by the human eye or OCR technology).

[0079] The super-resolution reconstruction model performs different processing operations for original text of varying clarity. Therefore, the original text's clarity feature vector can be extracted, and then the original image, the original text's clarity feature vector and position feature vector, and the target text's structural feature vector can be input into the super-resolution reconstruction model. The super-resolution reconstruction model can perform super-resolution reconstruction on the original text using processing operations corresponding to the clarity feature vectors. For the aforementioned first category of original text, because this type of original text originally has high clarity, the super-resolution reconstruction model can enhance the clarity of the original text while simultaneously eliminating background noise within the text area where the original text resides, without requiring regeneration of the text. The enhanced clarity text is then output as the optimized text. For the aforementioned second category of original text, the super-resolution reconstruction model can generate optimized text based on the target text's structural feature vector and replace the original text with the optimized text. For the aforementioned third category of original text, because this type of original text originally has extremely low clarity, which can easily result in erroneous text generation results, the super-resolution reconstruction model can appropriately enhance the clarity and sharpness of the original text without requiring regeneration of the text, and output the enhanced text as the optimized text.

[0080] In this embodiment, by extracting the clarity feature vector of the original text, the super-resolution reconstruction model can take processing measures corresponding to the clarity feature vector to perform super-resolution reconstruction on the original text, thereby taking different processing measures to reconstruct original text of different clarity, effectively improving the super-resolution reconstruction effect of original text of different clarity, and helping to improve the readability of the optimized text.

[0081] In one embodiment, a super-resolution reconstruction model may include a text processing branch and an image processing branch. The original image, the positional feature vector of the original text, and the target text structural feature vector are input into a pre-trained super-resolution reconstruction model. The text processing branch in the super-resolution reconstruction model may perform super-resolution reconstruction on the original text in the original image based on the positional feature vector and the target text structural feature vector, and output the optimized text. The image processing branch may perform super-resolution reconstruction on the region of the original image excluding the original text and output the reconstructed region. The reconstructed region and the optimized text are then combined to obtain a reconstructed image, which is then used as the output of the super-resolution reconstruction model.

[0082] In another embodiment, the original image can be first subjected to region extraction to extract the text region containing the original text and the non-text region excluding the original text. The text region, the positional feature vector of the original text, and the structural feature vector of the target text are then input into a first super-resolution reconstruction model to obtain the optimized text output by the first super-resolution reconstruction model. The non-text region is then input into a second super-resolution reconstruction model to obtain the reconstructed region output by the second super-resolution reconstruction model. The reconstructed region and the optimized text are then combined to obtain a reconstructed image, wherein the position of the optimized text in the reconstructed image remains consistent with the position of the original text in the original image.

[0083] In one embodiment, during the super-resolution reconstruction process, the background properties of the text area and non-text area in the original image may change. Background properties may include, but are not limited to, brightness, contrast, saturation, etc. Therefore, after obtaining the reconstructed image, background compensation can be performed on the text area containing the optimized text in the reconstructed image, so that the background properties of the compensated text area are the same as those of the non-text area in the reconstructed image. This ensures the consistency of the background of the reconstructed image and helps improve the overall display effect of the reconstructed image.

[0084] Figure 7a FIG. 1 is a flow chart of a super-resolution reconstruction method according to an embodiment of the present disclosure. Figure 7a As shown, the process may include the following steps:

[0085] S701: Acquire an original image.

[0086] The original image obtained in this embodiment is as follows Figure 7b As shown, the original image contains original text a and original text b, wherein the original text a (text style is Songti) is partially missing, and the original text b (text style is bold) is adhered.

[0087] S702: Extract features of the original text in the original image to obtain the position feature vector of the original text.

[0088] In this embodiment, perform positioning detection on the original text to obtain the positioning point feature vector of the original text. Then connect the positioning point feature vectors to obtain a closed positioning frame feature vector, and the position feature vector includes the positioning frame feature vector.

[0089] S703: Detect whether the positioning frame represented by the positioning frame feature vector obtained in S702 covers the original text.

[0090] If so, jump to S705 for scale transformation and / or orientation correction; if not, jump to S704 for correction of the positioning point feature vector.

[0091] S704: Correct the positioning point feature vector so that the positioning frame corresponding to the corrected positioning point feature vector covers the original text.

[0092] S705: Perform scale transformation and / or orientation correction on the original text.

[0093] In this embodiment, perform scale transformation on the original text so that the scale of the transformed text meets the preset scale requirement, and / or, when the orientation of the original text is an inclined orientation, correct the orientation of the original text. Combined Figure 7b , the orientations of the original text a and the original text b are not inclined orientations, so there is no need to correct the orientations of the original text a and the original text b.

[0094] S706: Extract features of the original text obtained in S705 to obtain the clarity feature vector, text style feature vector, and semantic feature vector of the original text.

[0095] S707: Retrieve the target text that simultaneously matches the semantic feature vector and text style feature vector of the original text from the candidate texts included in the dictionary library, and determine the target text structure feature vector corresponding to the target text recorded in the dictionary library.

[0096] Combined Figure 7b , it can be retrieved that the target text that simultaneously matches the semantic feature vector and text style feature vector of the original text a is the character '生' in Song typeface, and the target text that simultaneously matches the semantic feature vector and text style feature vector of the original text b is the character '目' in Bold typeface. Then respectively determine the target text structure feature vector m corresponding to the target text '生' and the target text structure feature vector n corresponding to the target text '目'.

[0097] S708: Convert the target text structure feature vector obtained in S707 to obtain the verification text, and calculate the similarity between the verification text and the original text.

[0098] S709: Determine whether the similarity obtained in S708 is greater than the similarity threshold.

[0099] If so, jump to S710; if not, jump to S711 and do not perform super-resolution reconstruction on the original text.

[0100] S710: Input the original image, the position feature vector and clarity feature vector of the original text, and the target text structure feature vector into a pre-trained super-resolution reconstruction model.

[0101] The super-resolution reconstruction model can perform super-resolution reconstruction on the original text by taking processing operations corresponding to the clarity feature vector of the original text. Combining Figure 7b , the clarity of both the original text a and the original text b is less than the first clarity threshold and greater than the second clarity threshold. Therefore, the super-resolution reconstruction model can generate the optimized text "sheng" according to the target text structure feature vector m, and generate the optimized text "mu" according to the target text structure feature vector n.

[0102] S712: Obtain the reconstructed image output by the super-resolution reconstruction model, and the reconstructed image includes the optimized text obtained by performing super-resolution reconstruction on the original text.

[0103] In this embodiment, for Figure 7b perform super-resolution reconstruction, and the obtained reconstructed image is as Figure 7c shown. Comparing Figure 7b and Figure 7c it can be seen that the super-resolution reconstruction method in this embodiment realizes the complementation of the original text with partial missing parts, and the reconstruction of clear optimized text for the original text with adhesion, greatly improving the readability of the optimized text.

[0104] Corresponding to the foregoing embodiments of the super-resolution reconstruction method, the present disclosure also provides an embodiment of a super-resolution reconstruction apparatus.

[0105] Please refer to Figure 8 , Figure 8 is a schematic block diagram of a super-resolution reconstruction apparatus provided by an exemplary embodiment. The apparatus may include: an extraction unit 802, a matching unit 804, and an input unit 806. Among them:

[0106] The extraction unit 802 is configured to obtain the original image and perform feature extraction on the original text included in the original image to obtain the position feature vector and semantic feature vector of the original text;

[0107] The matching unit 804 is configured to search for a target character that matches the semantic feature vector from candidate characters contained in a dictionary, and determine a target character structural feature vector corresponding to the target character recorded in the dictionary;

[0108] The input unit 806 is configured to input the original image, the position feature vector, and the target text structure feature vector into a pre-trained super-resolution reconstruction model, and obtain a reconstructed image output by the super-resolution reconstruction model, wherein the reconstructed image includes the optimized text obtained by super-resolution reconstruction of the original text.

[0109] Optionally, the apparatus further includes: a contrast enhancement unit 808 configured to enhance the contrast of the original image to obtain a contrast-enhanced image;

[0110] The extraction unit 802 is specifically configured to perform feature extraction on the original text contained in the contrast-enhanced image.

[0111] Optionally, the extraction unit 802 is specifically used to: perform positioning detection on the original text to obtain a positioning point feature vector of the original text; connect the positioning point feature vectors to obtain a closed positioning frame feature vector, and the position feature vector includes the positioning frame feature vector.

[0112] Optionally, the device further includes:

[0113] The positioning frame detection unit 810 is configured to detect whether the positioning frame represented by the positioning frame feature vector covers the original text, and if it is detected that the positioning frame does not cover the original text, correct the positioning point feature vector so that the positioning frame corresponding to the corrected positioning point feature vector covers the original text.

[0114] Optionally, the device further includes:

[0115] The text transformation unit 812 is configured to perform scale transformation on the original text before extracting the semantic feature vector of the original text so that the scale of the transformed text meets the preset scale requirement, and / or correct the direction of the original text when the direction of the original text is tilted.

[0116] Optionally, the dictionary library further includes a candidate text style feature vector corresponding to the candidate text, and the extraction unit 802 is further configured to: extract the text style feature vector of the original text;

[0117] The matching unit 804 is specifically configured to search the dictionary for target text that matches both the semantic feature vector and the text style feature vector.

[0118] Optionally, the apparatus further includes: a similarity calculation unit 814 configured to convert the target text structural feature vector to obtain a verification text, and calculate the similarity between the verification text and the original text;

[0119] The input unit 806 is specifically configured to: when the similarity is greater than a similarity threshold, input the original image, the position feature vector, and the target text structure feature vector into the pre-trained super-resolution reconstruction model.

[0120] Optionally, the extraction unit 802 is further used to extract a clarity feature vector of the original text; the input unit 806 is further used to input the clarity feature vector into the super-resolution reconstruction model, so that the super-resolution reconstruction model takes a processing operation corresponding to the clarity feature vector to perform super-resolution reconstruction on the original text; wherein, when the clarity feature vector indicates that the clarity of the original text is greater than or equal to a first clarity threshold, the processing operation includes improving the clarity of the original text to obtain the optimized text; when the clarity feature vector indicates that the clarity of the original text is less than the first clarity threshold and greater than a second clarity threshold, the processing operation includes generating the optimized text according to the target text structure feature vector; when the clarity feature vector indicates that the clarity of the original text is less than or equal to the second clarity threshold, the processing operation includes improving the clarity and sharpness of the original text to obtain the optimized text.

[0121] Optionally, the super-resolution reconstruction model includes a text processing branch and an image processing branch, wherein the text processing branch is used to perform super-resolution reconstruction on the original text and output the optimized text; the image processing branch is used to perform super-resolution reconstruction on the area other than the original text in the original image and output the reconstructed area; the reconstructed image is obtained by combining the reconstructed area and the optimized text.

[0122] Optionally, the device further includes:

[0123] The background compensation unit 816 is configured to perform background compensation on the text area where the optimized text is located, so that the background attributes of the compensated text area are the same as the background attributes of the non-text area in the reconstructed image.

[0124] Regarding the apparatus in the above embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments of the relevant methods and will not be elaborated on here.

[0125] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are merely illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the disclosed solution. Those of ordinary skill in the art can understand and implement it without paying any creative work.

[0126] Figure 9 9 is a schematic block diagram of an electronic device according to an embodiment of the present disclosure. For example, the electronic device 900 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0127] Reference Figure 9 , the electronic device 900 may include one or more of the following components: a processing component 902 , a memory 904 , a power component 906 , a multimedia component 908 , an audio component 910 , an input / output (I / O) interface 912 , a sensor component 914 , and a communication component 916 .

[0128] The processing component 902 generally controls the overall operation of the electronic device 900, such as operations associated with display, phone calls, data communications, camera operation, and recording operations. The processing component 902 may include one or more processors 920 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 902 may include one or more modules to facilitate interaction between the processing component 902 and other components. For example, the processing component 902 may include a multimedia module to facilitate interaction between the multimedia component 908 and the processing component 902.

[0129] The memory 904 is configured to store various types of data to support operations on the electronic device 900. Examples of such data include instructions for any application or method operating on the electronic device 900, contact data, phone book data, messages, pictures, videos, etc. The memory 904 can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0130] The power supply component 906 provides power to the various components of the electronic device 900. The power supply component 906 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 900.

[0131] The multimedia component 908 includes a screen that provides an output interface between the electronic device 900 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensor can not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 908 includes a front camera and / or a rear camera. When the electronic device 900 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have focal length and optical zoom capabilities.

[0132] The audio component 910 is configured to output and / or input audio signals. For example, the audio component 910 includes a microphone (MIC), and when the electronic device 900 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 904 or transmitted via the communication component 916. In some embodiments, the audio component 910 also includes a speaker for outputting audio signals.

[0133] I / O interface 912 provides an interface between processing component 902 and peripheral interface modules, such as a keyboard, click wheel, buttons, etc. These buttons may include but are not limited to: a home button, volume buttons, a start button, and a lock button.

[0134] The sensor assembly 914 includes one or more sensors for providing various aspects of status assessment for the electronic device 900. For example, the sensor assembly 914 can detect the open / closed state of the electronic device 900, the relative positioning of components, such as the display and keypad of the electronic device 900. The sensor assembly 914 can also detect changes in the position of the electronic device 900 or a component of the electronic device 900, the presence or absence of user contact with the electronic device 900, the orientation or acceleration / deceleration of the electronic device 900, and temperature changes of the electronic device 900. The sensor assembly 914 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 914 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 914 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0135] The communication component 916 is configured to facilitate wired or wireless communication between the electronic device 900 and other devices. The electronic device 900 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, 4G LTE, 9G NR or a combination thereof. In an exemplary embodiment, the communication component 916 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 916 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0136] In an exemplary embodiment, the electronic device 900 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to execute the method described in any of the above embodiments.

[0137] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 904 including instructions, and the instructions can be executed by the processor 920 of the electronic device 900 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0138] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the disclosure herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0139] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A super-resolution reconstruction method, characterized in that: include: Acquire an original image, and perform feature extraction on original text contained in the original image to obtain a position feature vector and a semantic feature vector of the original text; Retrieving a target character that matches the semantic feature vector from candidate characters contained in a dictionary library, and determining a target character structural feature vector corresponding to the target character recorded in the dictionary library; The original image, the position feature vector, and the target text structure feature vector are input into a pre-trained super-resolution reconstruction model, and a reconstructed image output by the super-resolution reconstruction model is obtained. The reconstructed image includes optimized text obtained by super-resolution reconstruction of the original text.

2. The method according to claim 1, characterized in that Also includes: Improving the contrast of the original image to obtain a contrast-enhanced image; The extracting features of the original text contained in the original image includes: extracting features of the original text contained in the contrast-enhanced image.

3. The method according to claim 1, characterized in that The extracting features of the original text contained in the original image to obtain the position feature vector of the original text includes: Performing positioning detection on the original text to obtain a positioning point feature vector of the original text; The positioning point feature vectors are connected to obtain a closed positioning frame feature vector, and the position feature vector includes the positioning frame feature vector.

4. The method according to claim 3, characterized in that Also includes: Detect whether the positioning box represented by the positioning box feature vector covers the original text, and if it is detected that the positioning box does not cover the original text, correct the positioning point feature vector so that the positioning box corresponding to the corrected positioning point feature vector covers the original text.

5. The method according to claim 1, wherein Before extracting the semantic feature vector of the original text, the method further includes: The original text is scaled so that the scale of the transformed text meets the preset scale requirement; and / or, if the direction of the original text is an inclined direction, the direction of the original text is corrected.

6. The method according to claim 1, characterized in that The dictionary library also includes a candidate character style feature vector corresponding to the candidate character, and the method further includes: extracting the character style feature vector of the original character; The searching for the target text that matches the semantic feature vector from the candidate texts contained in the dictionary library includes: searching for the target text that matches both the semantic feature vector and the text style feature vector from the dictionary library.

7. The method according to claim 6, characterized in that Also includes: Converting the target text structural feature vector to obtain verification text, and calculating the similarity between the verification text and the original text; The step of inputting the original image, the position feature vector, and the target text structure feature vector into a pre-trained super-resolution reconstruction model includes: when the similarity is greater than a similarity threshold, inputting the original image, the position feature vector, and the target text structure feature vector into the pre-trained super-resolution reconstruction model.

8. The method according to claim 1, characterized in that Also includes: Extracting the clarity feature vector of the original text and inputting the clarity feature vector into the super-resolution reconstruction model so that the super-resolution reconstruction model performs super-resolution reconstruction on the original text by performing a processing operation corresponding to the clarity feature vector; wherein, When the clarity feature vector indicates that the clarity of the original text is greater than or equal to a first clarity threshold, the processing operation includes improving the clarity of the original text to obtain the optimized text; In a case where the clarity feature vector indicates that the clarity of the original text is less than the first clarity threshold and greater than a second clarity threshold, the processing operation includes generating the optimized text according to the target text structure feature vector; When the clarity feature vector indicates that the clarity of the original text is less than or equal to the second clarity threshold, the processing operation includes improving the clarity and sharpness of the original text to obtain the optimized text.

9. The method according to claim 1, characterized in that The super-resolution reconstruction model includes a text processing branch and an image processing branch, wherein: The text processing branch is used to perform super-resolution reconstruction on the original text and output the optimized text; The image processing branch is used to perform super-resolution reconstruction on the area of ​​the original image except the original text and output the reconstructed area; The reconstructed image is obtained by combining the reconstructed region and the optimized text.

10. The method according to claim 1, characterized in that Also includes: Background compensation is performed on the text area where the optimized text is located, so that the background attributes of the compensated text area are the same as the background attributes of the non-text area in the reconstructed image.

11. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor implements the method according to any one of claims 1 to 10 by running the executable instructions.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

Images