Method and apparatus for extracting text from image, and terminal and computer program product
By using region selection and automatic expansion technology, the problem of cumbersome text extraction operations within local areas of images has been solved, achieving efficient and accurate text selection and simplifying the user operation process.
Patent Information
- Application Number
- PCT/CN2026/094486
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-05-07
- Filing Date
- 2026-04-30
- Publication Date
- 2026-11-12
AI Technical Summary
In existing technologies, text extraction from local areas of an image is cumbersome and inefficient. Users need to slide or crop the image in the text extraction panel to select text, which makes the operation complex and inefficient.
By using the region selection operation, the terminal highlights the area of text that can be extracted, and only the text within the selected area is displayed in the text extraction panel. It supports automatic expansion of the selected area to ensure text integrity and improves selection accuracy and efficiency.
It simplifies user operations, improves the efficiency and accuracy of text extraction from local areas of images, conforms to user's operating intuition, and reduces the need for manual adjustments.
Smart Images

Figure CN2026094486_12112026_PF_FP_ABST
Abstract
Description
Methods, devices, terminals, and computer programs for extracting text from images.
[0001] This application claims priority to Chinese Patent Application No. 202510586360.2, filed on May 7, 2025, entitled "Method, Apparatus, Terminal and Computer Program Product for Extracting Text from Images", the entire contents of which are incorporated herein by reference. Technical Field
[0002] This application relates to the field of human-computer interaction technology, and in particular to a method, apparatus, terminal, and computer program product for extracting text from images. Background Technology
[0003] Optical Character Recognition (OCR) is a technology that determines the shape of a character by detecting its brightness and darkness, and then converts the shape into computer text using character recognition methods.
[0004] In related technologies, after OCR processing of images, users can extract and use the recognized text. Summary of the Invention
[0005] This application provides a method, apparatus, terminal, and computer program product for extracting text from images. The technical solution is as follows:
[0006] On one hand, embodiments of this application provide a method for extracting text from an image, the method being used on a terminal, the method comprising:
[0007] Highlight the extractable text region in the image, where the extractable text region is the region containing extractable text;
[0008] In response to a region selection operation on the image, the target extractable text is displayed in the text extraction panel. The target extractable text is the extractable text contained within the local image region indicated by the region selection operation.
[0009] On the other hand, embodiments of this application provide a device for extracting text from images, the device comprising:
[0010] The display module is used to highlight the extractable text area in the image, wherein the extractable text area is the area containing extractable text;
[0011] The extraction module is used to display target extractable text in the text extraction panel in response to a region selection operation on the image. The target extractable text is the extractable text contained within the local image region indicated by the region selection operation.
[0012] On the other hand, embodiments of this application provide a terminal, the terminal including a processor and a memory, the memory storing at least one computer instruction, the at least one computer instruction being loaded and executed by the processor to implement the image text extraction method as described above.
[0013] On the other hand, embodiments of this application provide a computer-readable storage medium storing at least one computer instruction, which is executed by a processor to implement the image text extraction method as described above.
[0014] On the other hand, embodiments of this application provide a computer program product, which includes computer instructions that, when executed by a processor, implement the method for extracting text from images as described above. Attached Figure Description
[0015] Figure 1 shows a flowchart of a method for extracting text from an image according to an exemplary embodiment of this application;
[0016] Figure 2 is a schematic diagram of the interface illustrating the text extraction process in a local area of an image in an exemplary embodiment of this application;
[0017] Figure 3 shows a flowchart of the automatic expansion process of a local image region provided in an exemplary embodiment of this application;
[0018] Figure 4 is a schematic diagram illustrating an implementation of an exemplary embodiment of this application of the automatic expansion process of a local image region;
[0019] Figure 5 is a schematic diagram of an interface showing a partial image and text extraction panel displayed in parallel, according to an exemplary embodiment of this application;
[0020] Figure 6 is a schematic diagram illustrating a first type of extractable text region and a second type of extractable text region in an exemplary embodiment of this application;
[0021] Figure 7 is an exemplary embodiment of this application illustrating an interface in which image text and text represented by graphic codes are displayed in a text extraction panel.
[0022] Figure 8 is a schematic diagram illustrating a third type of extractable text region in an exemplary embodiment of this application;
[0023] Figure 9 shows a flowchart of a method for extracting text from an image provided in another exemplary embodiment of this application;
[0024] Figure 10 is a schematic diagram of the interface illustrating the text extraction process in a local area of an image, as shown in another exemplary embodiment of this application.
[0025] Figure 11 shows a flowchart of a method for extracting text from an image provided in another exemplary embodiment of this application;
[0026] Figure 12 is a schematic diagram of the interface for extracting text from a local area in an image, as illustrated in another exemplary embodiment of this application.
[0027] Figure 13 is a schematic diagram of the interface illustrating the text extraction process in a local area under a graphical code scenario in an exemplary embodiment of this application;
[0028] Figure 14 is a schematic diagram of the interface illustrating the text extraction process in a local area in a text translation scenario according to an exemplary embodiment of this application;
[0029] Figure 15 shows a structural block diagram of an image text extraction device provided in another exemplary embodiment of this application;
[0030] Figure 16 shows a schematic diagram of the structure of a terminal provided in an exemplary embodiment of this application. Detailed Implementation
[0031] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.
[0032] In this article, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.
[0033] In related technologies, when a text extraction operation for an image is received, the terminal displays all extractable text in the image in the text extraction panel and performs word segmentation on the all extractable text so that the user can select according to their needs.
[0034] When a user only needs text from a portion of an image, and the image contains a significant amount of extractable text, the user needs to swipe up and down in the text extraction panel and read the extractable text displayed to locate the desired text, which can then be selected and copied. Alternatively, the user can first crop and save the image based on the area where the desired text is located, and then perform text recognition and extraction on the cropped image.
[0035] Clearly, the above text extraction method is cumbersome and inefficient in extracting text from local areas of an image.
[0036] To improve the efficiency of text extraction, this embodiment supports region selection for images containing extractable text. When extractable text from a specific area of an image is required, the user can select that area by using a region selection operation. Based on this selection, the terminal displays the extractable text located within that specific area in the text extraction panel, rather than displaying the entire extractable text of the image, allowing the user to quickly select text.
[0037] This application provides a method that can be used on a terminal with image text extraction functionality. This image text extraction functionality can be provided by the terminal system or by an application installed on the terminal (a native application or a third-party application); the terminal can be a smartphone, tablet computer, personal computer, etc.
[0038] Furthermore, when implementing image text extraction, the recognition of text in the image can be performed locally on the terminal, or by a server (the backend server of the terminal system, or the backend server of the application providing the text extraction function). For example, the terminal uploads the image to be extracted to the server, and the server returns the recognized text and its location in the image to the terminal.
[0039] For ease of explanation, the following embodiments use the method of extracting text from images as an example from the terminal.
[0040] Please refer to Figure 1, which shows a flowchart of a method for extracting text from an image according to an exemplary embodiment of this application. This embodiment uses the method applied to a terminal as an example for illustration. The method may include the following steps.
[0041] Step 102: Highlight the extractable text area in the image. The extractable text area is the area containing the extractable text.
[0042] In this embodiment of the application, the image includes at least one extractable text region, and there is no overlap between different extractable text regions.
[0043] Regarding the highlighting method of the extractable text area, in one possible implementation, the extractable text area is a rectangular area that completely contains the extractable text and has a range larger than the text range of the extractable text, and this rectangular area is highlighted.
[0044] Of course, in other possible implementations, the edges of the extractable text area can be thickened, bounded, or highlighted to achieve emphasis, and this embodiment does not limit this.
[0045] In this embodiment of the application, the extractable text contained in the extractable text region can be explicit text or implicit text. Explicit text is text that can be directly observed, while implicit text is not directly displayed in text form, but is encoded into non-text elements that require further parsing to be deciphered, such as text encoded into graphic codes.
[0046] In this embodiment of the application, the extractable text includes text in various languages, such as Chinese, English, Japanese, Korean, French, etc.; and / or, various punctuation marks; and / or, various mathematical characters.
[0047] As illustrated in Figure 2, the various extractable text regions 22 in image 21 are highlighted.
[0048] Step 104: In response to the region selection operation on the image, display the target extractable text in the text extraction panel. The target extractable text is the extractable text contained within the local image region indicated by the region selection operation.
[0049] The region selection operation refers to selecting a specific area within an image. The selected area is a closed region, which can be a regular shape (such as a rectangle created by dragging) or an irregular shape (such as a manually selected circular area).
[0050] Furthermore, the local image area indicated by the region selection operation can be exactly the same as the area selected by the region selection operation, or it can be expanded based on the region selection operation (i.e., the local image area indicated by the region selection operation can be larger than the area selected by the region selection operation).
[0051] The text extraction panel displays extractable text from images, and this text can be selected for application purposes. For example, the extractable text displayed in the text extraction panel supports partial copying or full copying.
[0052] In some embodiments, the text extraction panel is triggered by word segmentation, and the extractable text displayed in the text extraction panel has been processed by word segmentation.
[0053] Unlike related technologies where the text extraction panel displays all extractable text in an image, in this embodiment, when a region selection operation is received for an image, the terminal only displays the extractable text contained within the local image region indicated by the region selection operation in the text extraction panel. That is, extractable text contained outside the local image region will not be displayed in the text extraction panel, thus avoiding interference with the user's text selection.
[0054] As illustrated in Figure 2, the terminal displays a region selection box 23 on top of image 21 based on the received region selection operation. When the region selection operation is completed, the terminal displays the extractable text 25 contained within the local image area within the region selection box 23 in the text extraction panel 24. Extractable text contained within the image area outside the region selection box 23 (such as "Meet Again", "More Cities Coming Soon", etc.) will not be displayed in the text extraction panel 24.
[0055] In summary, in this embodiment of the application, when it is necessary to extract extractable text from a local image region, the user can select the desired local image region by performing a region selection operation. The terminal then displays the extractable text located within that local image region in the text extraction panel, facilitating text selection. Compared to locating the desired text from the full extractable text displayed in the text extraction panel, or first cropping the image and then performing text recognition and extraction on the cropped image, directly selecting a region on the image is more efficient and more intuitive for the user.
[0056] In some embodiments, in response to a region selection operation on an image, the target extractable text is displayed in a text extraction panel, including:
[0057] Receives a region selection operation on an image;
[0058] If the selected area of a local image is missing, the local image area will be automatically expanded to obtain the expanded local image area.
[0059] The Text Extraction panel displays the target extractable text contained within the expanded local image area.
[0060] In some embodiments, when there is a missing selection area within the local image region indicated by the region selection operation, the local image region is automatically expanded to obtain an expanded local image region, including:
[0061] Determine the intersection between the local image area indicated by the region selection operation and each extractable text area;
[0062] If there is an intersection with the extractable text region, and the intersection region is smaller than the extractable text region, the local image region is automatically expanded based on the extractable text region to obtain the expanded local image region. The intersection region between the expanded local image region and the extractable text region is the extractable text region.
[0063] In some embodiments, when there is an intersection with the extractable text region, and the intersection region is smaller than the extractable text region, the local image region is automatically expanded based on the extractable text region to obtain the expanded local image region, including:
[0064] If there is an intersection with the extractable text region, and the intersection region is smaller than the extractable text region, determine the overflow text contained outside the intersection region in the extractable text region;
[0065] In cases where overflowing text cannot be discarded, the local image region is automatically expanded based on the extractable text region to obtain the expanded local image region.
[0066] The method also includes:
[0067] If the overflowing text is discardable, the target extractable text contained within a local image area is displayed in the Text Extraction panel.
[0068] In some embodiments, the method includes:
[0069] Determine the text relevance between the text within the intersection area and the overflow text;
[0070] If the text relevance is greater than the relevance threshold, the overflowing text must not be discarded.
[0071] If the text relevance is less than the relevance threshold, the overflowing text can be discarded.
[0072] In some embodiments, the method further includes:
[0073] The local image corresponding to the local image area is displayed in the image display area outside the text extraction panel, and the local image is magnified to the maximum ratio of the image display area.
[0074] In some embodiments, the method further includes:
[0075] In response to a region selection operation on a local image, update the local image displayed in the image display area, as well as the target extractable text displayed in the text extraction panel.
[0076] In some embodiments, the extractable text contained in the extractable text region includes at least one of the following:
[0077] Image text in an image, translated text corresponding to the image text in an image, and text represented by graphic codes in an image;
[0078] The translated text is displayed on top of the corresponding image text.
[0079] In some embodiments, highlighting an extractable text region in an image includes at least one of the following:
[0080] In response to the text recognition operation on the image, the first type of extractable text region and / or the second type of extractable text region in the image are highlighted. The extractable text contained in the first type of extractable text region is image text, and the extractable text contained in the second type of extractable text region is text represented by a graphic code.
[0081] In response to the text translation operation on the image, the third type of extractable text region in the image is highlighted. The extractable text contained in the third type of extractable text region is the translated text.
[0082] In some embodiments, in response to a region selection operation on an image, the target extractable text is displayed in a text extraction panel, including:
[0083] In response to a region selection operation on an image, the local image region indicated by the region selection operation is highlighted;
[0084] Once a local image area is selected, the target extractable text is displayed in the text extraction panel in response to the text extraction operation.
[0085] In some embodiments, when a partial image region is selected, in response to a text extraction operation, the target extractable text is displayed in the text extraction panel, including:
[0086] Once a local image area has been selected, the enlarged local image corresponding to that area will be displayed.
[0087] In response to a text extraction operation, the target extractable text is displayed in the text extraction panel.
[0088] In some embodiments, after displaying the local image corresponding to the magnified local image region, the method further includes:
[0089] In response to the undo operation of the selection box, stop displaying the partial image and redisplay the image.
[0090] In some embodiments, the method further includes:
[0091] In response to the text extraction operation, the full extractable text contained in the image is displayed in the text extraction panel, and the image is displayed in the image display area outside the text extraction panel;
[0092] In response to a region selection operation on an image, the target extractable text is displayed in the text extraction panel, including:
[0093] In response to a region selection operation on an image, the local image region indicated by the region selection operation is highlighted in the image display area;
[0094] Once a local image area is selected, the target text that can be extracted will be displayed in the text extraction panel.
[0095] In some embodiments, the method further includes:
[0096] In response to a region selection operation on an image, the extractable text contained within a specific area of the image in the text extraction panel is highlighted.
[0097] Due to limitations in terminal screen size and operational precision, users may experience incomplete selection when performing area selection operations (the text the user expects is not included in the selection), requiring users to repeatedly adjust the selection area.
[0098] To improve the accuracy and efficiency of text selection, in this embodiment, the terminal automatically identifies whether there are missing selection areas within the local image area indicated by the region selection operation, and automatically expands the local image area indicated by the region selection operation if missing selection areas exist. As shown in Figure 3, step 104 above may include the following sub-steps:
[0099] Step 104A: Receive the region selection operation for the image.
[0100] In one possible implementation, after receiving a region selection operation, the terminal determines whether there is a missing selection area within the local image region indicated by the region selection operation.
[0101] In some embodiments, when the terminal detects the end of a single region selection operation, it determines whether there is a missing selection area within the local image region indicated by the current region selection operation. The single region selection operation may be the first region selection operation and / or a region selection adjustment operation performed based on the previous region selection operation.
[0102] If there are no missing selections within the local image area indicated by the region selection operation, the terminal displays the target extractable text contained within that local image area in the text extraction panel. If there are missing selections within the local image area indicated by the region selection operation, the terminal automatically expands the local image area through step 104B as described below.
[0103] It should be noted that regardless of whether automatic expansion is performed later, users can continue to manually select and adjust the selections.
[0104] Step 104B: If there is a missing selection area within the local image area indicated by the region selection operation, the local image area is automatically expanded to obtain the expanded local image area.
[0105] Regarding the method for identifying whether there is a missing selection box, in one possible implementation, the terminal determines the intersection between the local image area indicated by the region selection operation and each extractable text area.
[0106] Optionally, for any extractable text region, the terminal determines the intersection between the local image region and the extractable text region based on their respective area ranges. This intersection can be either present or absent. If an intersection exists, the local image region may completely contain the extractable text region, or it may partially contain the extractable text region.
[0107] To illustrate, when both the local image region and the extractable text region are rectangular regions, and the coordinates of the lower left corner of the local image region are (x1, y1) and the upper right corner are (x2, y2), and the coordinates of the lower left corner of the extractable text region are (x3, y3) and the upper right corner are (x4, y4), then if x2 < x3, or x1 > x4, or y2 < y3, or y1 > y4, then there is no intersection between the local image region and the extractable text region; if x1 ≤ x4, and x3 ≤ x2, and y1 ≤ y4, and y2 ≤ y3, then there is no intersection between the local image region and the extractable text region.
[0108] Of course, the terminal can also use other methods to determine whether two regions have an intersection, which is not limited in this embodiment.
[0109] Furthermore, for each extractable text region, if there is an intersection between the local image region and the extractable text region, and the intersection region is smaller than the extractable text region, the terminal automatically expands the local image region based on the extractable text region to obtain the expanded local image region.
[0110] In this case, the intersection area between the local image region and the extractable text region is smaller than the extractable text region, indicating that the extractable text region is not completely contained within the local image region. At this time, extractable text may exist in the part of the extractable text region located outside the local image region.
[0111] Correspondingly, the intersection area between the expanded local image area and the extractable text area is the extractable text area. That is, after automatic expansion, the extractable text area is completely located inside the local image area.
[0112] Regarding the method of automatically expanding the local image region based on the extractable text region, in one possible implementation, the terminal determines a first boundary in the extractable text region located outside the local image region, and extends a second boundary of the local image region to the first boundary.
[0113] Illustratively, when both the partial image area and the extractable text area are rectangular areas, if the right boundary of the extractable text area is located outside the partial image area, the terminal extends the right boundary of the partial image area to the right boundary of the extractable text area; if the left boundary and the upper boundary of the extractable text area are located outside the partial image area, the terminal extends the left boundary of the partial image area to the left boundary of the extractable text area, and extends the upper boundary of the partial image area to the upper boundary of the extractable text area.
[0114] In some embodiments, to allow the user to intuitively perceive the automatic expansion effect, the terminal displays a frame selection box change animation of expanding the original partial image area (i.e., the partial image area before expansion) to the target partial image area (i.e., the partial image area after expansion).
[0115] Illustratively, as shown in FIG. 4, the user performs an area frame selection operation on the image 41, and obtains the original partial image area 42 through frame selection. Since part of the extractable text area 43 that intersects with the original partial image area 42 is located outside the original partial image area 42, the terminal automatically expands the original partial image area 42 to the target partial image area 44 based on the range of the extractable text area 43, so that the extractable text area 43 is completely located inside the target partial image area 44.
[0116] Step 104C: Display the target extractable text contained in the expanded partial image area in the text extraction panel.
[0117] Further, the terminal determines the extractable text area located within the expanded partial image area, determines the extractable text in the extractable text area as the target extractable text, and displays the same in the text extraction panel.
[0118] Illustratively, as shown in FIG. 4, if automatic expansion is not performed, the extractable text of the content of the original partial image area 42 displayed in the text extraction panel will miss the character "field"; after the automatic expansion is performed, the extractable text in the target partial image area 44 displayed in the text extraction panel includes the character "field".
[0119] In this embodiment, the terminal automatically identifies whether there is a frame selection missing in the partial image area indicated by the area frame selection operation, and automatically expands the partial image area when there is a frame selection missing. On the one hand, this ensures the completeness of the selected text displayed in the subsequent text extraction panel, and on the other hand, eliminates the need for the user to manually adjust the frame selection area continuously (especially for areas with a small screen size or high text density), which improves the efficiency and accuracy of text extraction in the partial image area.
[0120] In some scenarios, the same extractable text area may contain multiple text segments. When a user wants to select a specific segment using a region selection operation, the extractable text area may not be completely contained within the local image area indicated by the region selection operation. If automatic expansion is performed in such scenarios, the text extraction panel may display too much distracting text. To further improve the accuracy of automatic expansion timing, in one possible implementation, if the local image area and the extractable text area intersect, and the intersection area is smaller than the extractable text area, the terminal determines the overflow text contained outside the intersection area within the extractable text area.
[0121] Optionally, the terminal determines the overflow area that belongs to the extractable text area but does not belong to the local image area based on the region boundary of the extractable text area and the region boundary of the local image area, and determines the overflow text located in the overflow area.
[0122] As illustrated in Figure 4, the terminal determines the overflow text located outside the original local image area 42 but within the extractable text area 43 as "field".
[0123] For the identified overflow text, the terminal determines whether the overflow text can be discarded.
[0124] Regarding the method for determining whether overflowing text can be discarded, since the text that users expect to extract is usually relevant, the terminal may determine whether the overflowing text can be discarded based on text relevance.
[0125] In one possible implementation, the terminal determines the text relevance between the text within the intersection region and the overflow text. If the text relevance is greater than a relevance threshold, the terminal determines that the overflow text cannot be discarded; if the text relevance is less than the relevance threshold, the terminal determines that the overflow text can be discarded.
[0126] Optionally, the text relevance can be determined using a word frequency-based approach, such as using a Bag of Words (BoW) model or TF-IDF (Term Frequency-Inverse Document Frequency); and / or a semantic-based approach, such as using the vector distance between word embeddings or sentence embeddings; and / or a deep learning-based approach, such as using a BERT model with input relevance scores to determine the relevance; and / or a knowledge graph-based approach, such as using entities and relationships in a knowledge graph to determine the text relevance. This application does not limit the specific method for determining text relevance.
[0127] Optionally, the relevance threshold can be a preset threshold or a user-defined threshold.
[0128] Furthermore, in cases where the overflowing text cannot be discarded, the terminal automatically expands the local image area based on the extractable text area to obtain the expanded local image area.
[0129] As illustrated in Figure 4, since the textual relevance between the overflow text "field" and the text "A City Sports" in the communication area is greater than the relevance threshold, the terminal performs automatic expansion of the original local image area 42.
[0130] When the overflowing text can be discarded, the terminal does not automatically expand the local image area, thus displaying the target extractable text contained within the local image area in the text extraction panel.
[0131] Of course, in addition to determining whether overflowing text can be discarded based on text relevance, the terminal can also determine whether overflowing text can be discarded based on the punctuation marks between the overflowing text and the text in the intersection area (for example, when there are terminating marks (such as question marks, periods, etc.) between the overflowing text and the text in the intersection area), or based on the user's feedback on the expansion prompt (when overflowing text is detected, prompting the user whether automatic expansion is needed), which will not be elaborated in this embodiment.
[0132] In this embodiment, the terminal determines whether to automatically fill the local image area by identifying whether the overflow text in the overflow area can be discarded, thus avoiding interference from the text added after meaningless expansion that would otherwise interfere with subsequent text extraction.
[0133] In some embodiments, in order to improve the efficiency of comparing images with the text to be extracted during the text extraction process and to eliminate the tedious operation of repeatedly closing the text extraction panel to view the image, in one possible implementation, the text extraction panel is not displayed in full screen, but only occupies a portion of the display area.
[0134] Correspondingly, the terminal displays the local image corresponding to the local image area in the image display area outside the text extraction panel.
[0135] The display area occupied by the text extraction panel can be a fixed size, or a size that dynamically changes according to the amount of text that can be extracted (with a lower and upper limit). This embodiment does not limit this.
[0136] Furthermore, to facilitate users' viewing of text within a partial image, in some embodiments, the partial image is magnified to the maximum scale of the image display area.
[0137] To illustrate, when the size of the local image is w1×h1 and the size of the image display area is w2×h2, the size of the magnified local image is: w1×max(w2 / w1, h2 / h1), h1×max(w2 / w1, h2 / h1).
[0138] As illustrated in Figure 5, the terminal displays the segmented extractable text in the text prompt panel 52, and displays a magnified partial image 51 in the upper half of the image display area.
[0139] In some embodiments, to facilitate text comparison by the user (i.e., comparing the text in the image with the text in the text extraction panel to ensure recognition accuracy), when a selection operation is received for extractable text in the text extraction panel, the terminal highlights the area of the selected text indicated by the selection operation in the local image. For example, the terminal highlights the area corresponding to the text selected by the user in the text extraction area in the local image; and when the user cancels the text selection, the area corresponding to the canceled text will be de-highlighted.
[0140] In one possible design, the partial image displayed in the image display area supports further region selection. In response to a region selection operation on the partial image, the terminal updates the partial image displayed in the image display area, as well as the target extractable text displayed in the text extraction panel.
[0141] The updated local image in the image display area is the portion of the image selected through the region selection operation; the updated target extractable text in the text extraction panel is the extractable text contained in the updated local image.
[0142] It should be noted that during the process of further selecting regions in a local image, the terminal can also automatically expand the selected region, which will not be elaborated here.
[0143] In this embodiment, by displaying the local image and the extractable text within the local image on the same screen, the user can simultaneously compare the image and text in the local image when selecting text in the text extraction panel. This eliminates the need for the user to repeatedly close the text extraction panel to view the image, thus improving the efficiency of subsequent text selection.
[0144] In this embodiment of the application, the extractable text contained in the extractable text region may include at least one of the following:
[0145] 1. Image text within an image.
[0146] Image text can be understood as embedded text within an image, typically requiring OCR recognition. Correspondingly, the extractable text region containing image text is a portion of the image's pixel area.
[0147] 2. The translated text corresponding to the image text in the picture.
[0148] The translated text is obtained by translating the text in the image. In one possible implementation, the terminal first performs OCR recognition on the image to obtain the image text; when the image text contains text in a non-target language and a translation operation is received, the terminal translates the non-target language text into text in the target language, and this target language text is the translated text.
[0149] For example, when the image contains English text, the translated text is the Chinese equivalent of the English text.
[0150] In some embodiments, the translated text is displayed on top of the corresponding image text. Accordingly, the extractable text region containing the translated text is a portion of the pixel region of the image's upper layer.
[0151] 3. The text represented by the graphic code in the image.
[0152] Common graphic codes can include barcodes and QR codes (circular, radial, square, data matrix, etc.). Graphic codes typically carry specific text, which can include links, strings, etc.
[0153] In one possible implementation, the text represented by the graphic code needs to be parsed using a specific graphic code parsing algorithm. Accordingly, the extractable text region containing the text represented by the graphic code is the graphic code region in the image.
[0154] In this embodiment of the application, the terminal can simultaneously highlight any one or more of the above three types of extractable text areas.
[0155] In one possible application scenario, in response to a text recognition operation on an image, the terminal highlights a first type of extractable text region and / or a second type of extractable text region in the image. The extractable text contained in the first type of extractable text region is image text, and the extractable text contained in the second type of extractable text region is text represented by a graphic code.
[0156] The text recognition operation is used to trigger the recognition of extractable text in an image. Unlike previous methods that only highlighted the area containing explicit extractable text (i.e., image text) in an image, in this embodiment, when implicit extractable text (i.e., text represented by a graphic code) exists in an image, the terminal also highlights the area containing the implicit extractable text to inform the user that the area supports text extraction.
[0157] In some embodiments, the size of the first type of extractable text region is greater than or equal to the size of the image text, and the size of the second type of extractable text region is greater than or equal to the size of the graphic code.
[0158] As illustrated in Figure 6, when a text recognition operation is received for image 61, since image 61 contains both image text and QR code, the terminal highlights the first type of extractable text region 62 and the second type of extractable text region 63 in image 61.
[0159] Optionally, in response to a subsequent region selection operation on the image, if the local image region indicated by the region selection operation includes a second type of extractable text region, the target extractable text displayed by the terminal in the text extraction panel includes the text parsed from the graphic code.
[0160] As illustrated in Figure 7, when the selected partial image 71 contains image text and a QR code, the terminal displays the first extractable text 73 corresponding to the image text and the second extractable text 74 (link) obtained by parsing the QR code in the text extraction panel 72.
[0161] In another possible application scenario, in response to a text translation operation on an image, the terminal highlights a third type of extractable text region in the image, where the extractable text contained in the third type of extractable text region is the translated text.
[0162] The text translation operation triggers the translation of text in the image that is not in the target language into text in the target language. In some embodiments, the translated text is displayed in the upper layer of the image, and correspondingly, the third type of extractable text region is located in this upper layer.
[0163] Furthermore, the size of the third type of extractable text region is related to the size of the translated text, and the size of the translated text is usually consistent with the size of the corresponding image text.
[0164] As illustrated in Figure 8, when a text translation operation is received for image 81, the terminal highlights the third type of extractable text region 82 in image 81, which displays the translated text in the target language.
[0165] Optionally, in response to a subsequent region selection operation on the image, if the local image region indicated by the region selection operation includes a third type of extractable text region, the target extractable text displayed by the terminal in the text extraction panel includes the translated text.
[0166] It should be noted that when a text translation operation is received and the image contains image text in the target language, the terminal can also highlight the first type of extractable text area, which displays the image text recognized by OCR (without translation).
[0167] Of course, in other possible application scenarios, the terminal can simultaneously highlight the first type of extractable text region, the second type of extractable text region, and the third type of extractable text region in the image, which will not be elaborated here.
[0168] In this embodiment, the terminal highlights the graphic code area in the image, and displays the parsing result of the graphic code in the text extraction panel when the graphic code area is selected in the subsequent process. This enables the text extraction function to support the extraction of implicit text in the graphic code, thus expanding the applicable scenarios of the text extraction function.
[0169] For example, the terminal supports extracting text from images and graphic codes simultaneously, improving ease of use and meeting the user's need to extract graphic codes and forward the parsed links. This can improve the operational efficiency of the information recipient in this scenario, meaning that the information recipient can directly open the link without having to decode the graphic code.
[0170] In addition, by highlighting the area where the translated text is located in the image, the terminal enables the text extraction function to extract the translated text corresponding to the image text in the image, further expanding the applicable scenarios of the text extraction function.
[0171] In some embodiments, the text extraction panel is displayed upon receiving a text extraction operation, and the execution sequence of the text extraction operation and the region selection operation can be:
[0172] 1. First, perform the region selection operation, then perform the text extraction operation;
[0173] 2. First perform the text extraction operation, then perform the region selection operation.
[0174] The following illustrative examples illustrate the text extraction process under the two execution sequences described above.
[0175] Please refer to Figure 9, which shows a flowchart of a method for extracting text from an image according to another exemplary embodiment of this application. This embodiment uses the method applied to a terminal as an example for illustration. The method may include the following steps.
[0176] Step 901: Highlight the extractable text area in the image. The extractable text area is the area containing the extractable text.
[0177] Step 902: In response to the region selection operation on the image, highlight the local image region indicated by the region selection operation.
[0178] In this embodiment, when the extractable text area is highlighted, the terminal supports receiving user selection operations of regions in the image. Furthermore, to allow the user to intuitively understand the currently selected area and subsequently manually adjust the selection range, the terminal highlights the local image area indicated by the region selection operation.
[0179] The highlighting methods include highlighting local image areas, reducing brightness, and thickening the edges of areas, etc. This embodiment does not limit these methods.
[0180] It should be noted that after receiving the region selection operation, the terminal can use the region selection optimization scheme provided in the above embodiment to automatically expand the local image region; and the automatically expanded local image region is also highlighted, which will not be described in detail here.
[0181] As illustrated in Figure 10, when a text recognition operation is received for image 1001, the terminal selects and displays the extractable text region 1002 within image 1001. When a region selection operation is received for image 1001, the terminal highlights the original local image region 1003 indicated by the region selection operation. When it is detected that the original local image region 1003 has a missing selection area, the terminal automatically expands the original local image region 1003 into a target local image region 1004 and highlights the target local image region 1004.
[0182] Step 903: Once the local image area has been selected, in response to the text extraction operation, the target extractable text is displayed in the text extraction panel.
[0183] Optionally, if no further region selection operation is received within a preset time period, the terminal determines that the local image region selection is complete; or, if a confirmation operation to confirm the selection is complete is received, the terminal determines that the local image region selection is complete.
[0184] Optionally, the text extraction operation can be a trigger operation of the text extraction control in the interface (such as a click operation of the word segmentation control), a voice command operation, or a gesture operation, etc. This embodiment does not limit this.
[0185] When a text extraction request is received, the terminal displays a text extraction panel on the interface, showing the target extractable text within a localized area of the image. This text extraction panel can be displayed in full screen, or it can be displayed alongside the image display area.
[0186] In some embodiments, to enable users to more clearly see the text within the selected area, after the partial image area is selected, the terminal displays a magnified partial image corresponding to the selected area. Furthermore, in response to the text extraction operation, the terminal displays the target extractable text in the text extraction panel.
[0187] Optionally, the local image is magnified to its maximum display scale to increase the display size of the text in the local image.
[0188] Optionally, when the image is displayed in full screen, the user can perform a region selection operation on the full-screen image. Once the region selection is complete, the terminal stops displaying the image in full screen and displays a magnified partial image.
[0189] Optionally, when displaying a magnified partial image, the terminal supports undoing the previous selection operation. In response to the selection undo operation, the terminal stops displaying the partial image and redisplays the original image.
[0190] The undo operation can be a gesture (such as swiping left to undo), a trigger operation on the undo control, or a voice command operation, etc.
[0191] In some embodiments, after the image is redisplayed due to the selection cancellation operation, the user can re-select the area of the image. The process of re-selecting the area can be referred to the above embodiments, and will not be repeated here.
[0192] As illustrated in Figure 10, when the region selection is completed, the terminal magnifies and displays the local image 1005 corresponding to the target local image region 1004. When a text extraction operation is received, the terminal displays the local image 1005 in the upper half of the image display area and displays the word segmentation results of the extractable text within the local image 1005 in the lower half of the text extraction panel 1006.
[0193] Please refer to Figure 11, which shows a flowchart of a method for extracting text from an image according to another exemplary embodiment of this application. This embodiment uses the method applied to a terminal as an example for illustration. The method may include the following steps.
[0194] Step 1101: Highlight the extractable text area in the image. The extractable text area is the area containing the extractable text.
[0195] Step 1102: In response to the text extraction operation, display all extractable text contained in the image in the text extraction panel, and display the image in the image display area outside the text extraction panel.
[0196] Unlike related technologies where the terminal displays a full-screen text extraction panel when a text extraction operation is received from an image, and displays all extractable text in the image for the user to select, in this embodiment, to facilitate the user's selection of extractable text within a local image area, the terminal displays the text extraction panel and the image display area in parallel.
[0197] The text extraction panel displays the full extractable text contained in the image (with vertical scrolling support), while the image display area shows a scaled-down version of the image.
[0198] As illustrated in Figure 12, when a text recognition operation is received for image 1201, the terminal selects and displays the extractable text region 1202 in image 1201. When a text extraction operation is received, the terminal displays all extractable text contained in image 1201 in the text extraction panel 1204, and displays a scaled-down version of image 1201 in the image display area above.
[0199] Step 1103: In response to the region selection operation on the image, highlight the local image region indicated by the region selection operation in the image display area.
[0200] In this embodiment, during text extraction, the image displayed within the image display area supports region selection.
[0201] Furthermore, to allow users to intuitively understand the currently selected area so that they can continue to manually adjust the selection range later, the terminal highlights the local image area indicated by the area selection operation.
[0202] The highlighting methods include highlighting local image areas, reducing brightness, and thickening the edges of areas, etc. This embodiment does not limit these methods.
[0203] It should be noted that after receiving the region selection operation, the terminal can use the region selection optimization scheme provided in the above embodiment to automatically expand the local image region; and the automatically expanded local image region is also highlighted, which will not be described in detail here.
[0204] Compared to the original full-screen image display, the text within the image is also smaller because the image size displayed within the image display area is relatively small. To make it clearer to the user which text has been selected by the current area selection operation, in one possible implementation, in response to the area selection operation on the image, the terminal highlights the extractable text contained within a local image area in the text extraction panel.
[0205] In some embodiments, when the amount of extractable text is too large to be fully displayed on a single screen in the text extraction panel, the terminal scrolls the text in the text extraction panel until the extractable text contained in the local image area is highlighted.
[0206] As illustrated in Figure 12, when a region selection operation is received for the scaled-down image 1201, the terminal highlights the local image region 1205 indicated by the region selection operation and highlights the extractable text 1206 within the local image region 1205 in the text extraction panel 1204.
[0207] Step 1104: After selecting the local image area, display the target extractable text in the text extraction panel.
[0208] Optionally, if no further region selection operation is received within a preset time period, the terminal determines that the local image region selection is complete; or, if a confirmation operation to confirm the selection is complete is received, the terminal determines that the local image region selection is complete.
[0209] After selecting a specific area of the image, the text extraction panel will no longer display all the extractable text contained in the image, but only the extractable text contained within that specific area of the image.
[0210] Optionally, to facilitate text-image comparison for users during subsequent text selection, the terminal displays a partial image corresponding to a localized image area within the image display area. This partial image is magnified to the maximum scale of the localized image area.
[0211] As illustrated in Figure 12, when the area selection is completed, the terminal displays the local image 1207 corresponding to the local image area 1205 in the upper half of the image display area, and displays the word segmentation results of the text that can be extracted from the local image 1207 in the text extraction panel 1204 in the lower half.
[0212] The above embodiments are illustrated using the extraction of image text from an image as an example. In another illustrative example, when an image contains both image text and graphic codes, as shown in Figure 13, when a text recognition operation is received on image 1301, the terminal selects and displays the first type of extractable text region 1302 containing image text and the second type of extractable text region 1303 containing graphic codes in image 1301.
[0213] When a text extraction operation is received, the terminal displays the full extractable text 1305 contained in image 1301 in the text extraction panel 1304, and displays a scaled-down image 1301 in the image display area above. The full extractable text 1305 includes the extractable text corresponding to the image text and the text obtained by parsing the graphic code (i.e., the text represented by the graphic code, taking a link as an example in the figure).
[0214] When a region selection operation is received for the scaled-down image 1301, the terminal highlights the local image region 1306 indicated by the region selection operation and highlights the extractable text within the local image region 1306 in the text extraction panel 1304.
[0215] It should be noted that, in the case of determining whether there is a missing selection area, if the local image area indicated by the region selection operation intersects with the second type of extractable text area, and the intersection area is smaller than the second type of extractable text area, that is, the region selection operation only selects a part of the graphic code, the terminal automatically expands the local image area so that the intersection area is equal to the second type of extractable text area, thereby completely selecting the graphic code within the local image area.
[0216] Alternatively, if the local image region indicated by the region selection operation intersects with the second type of extractable text region, and the intersecting region is smaller than the second type of extractable text region, and the proportion of the intersecting region to the second type of extractable text region is greater than the proportion threshold (e.g., 50%), the terminal will automatically expand the local image region.
[0217] When the region selection is completed, the terminal displays the local image 1307 corresponding to the local image region 1306 in the upper half of the image display area, and displays the word segmentation results of the extractable text within the local image 1307 in the text extraction panel 1304 in the lower half.
[0218] It should be noted that this embodiment only illustrates the example of performing text extraction operation first and then region selection operation. For the extraction process of extractable text in graphic codes and image text, the region selection operation can also be performed first and then the text extraction operation can be performed. This embodiment will not elaborate on this.
[0219] In another illustrative example, when the image contains image text in a language other than the target language, as shown in Figure 14, when a text translation operation is received for image 1401, the terminal translates the non-target language text in image 1401 into target language text, displays the translated text on top of the corresponding image text, and selects and displays the third type of extractable text region 1402 corresponding to the translated text.
[0220] When a text extraction operation is received, the terminal displays the full extractable text 1404 contained in image 1401 in the text extraction panel 1403, and displays a scaled-down version of image 1401 in the image display area above. The full extractable text 1404 includes the translated text.
[0221] When a region selection operation is received for the scaled-down image 1401, the terminal highlights the local image region indicated by the region selection operation and highlights the extractable text within the local image region in the text extraction panel 1403.
[0222] It should be noted that the terminal can automatically expand the local image area by identifying whether there is a missing selection box, which will not be elaborated here.
[0223] When the region selection is completed, the terminal displays the local image 1405 corresponding to the local image region in the upper half of the image display area, and displays the word segmentation results of the text that can be extracted from the local image 1405 in the text extraction panel 1403 in the lower half.
[0224] It should be noted that this embodiment only illustrates the example of performing text extraction operation first and then region selection operation. For the process of extracting translated text from an image, the region selection operation can also be performed first and then the text extraction operation can be performed. This embodiment will not elaborate on this.
[0225] Please refer to Figure 15, which shows a structural block diagram of an image text extraction apparatus provided in an exemplary embodiment of this application. The apparatus includes:
[0226] Display module 1501 is used to highlight the extractable text area in the image, wherein the extractable text area is the area containing extractable text;
[0227] The extraction module 1502 is used to display target extractable text in the text extraction panel in response to a region selection operation on the image. The target extractable text is the extractable text contained within the local image region indicated by the region selection operation.
[0228] Optionally, the extraction module 1502 is used for:
[0229] Receive the operation of selecting the region of the image;
[0230] If there is a missing selection area within the local image area indicated by the area selection operation, the local image area is automatically expanded to obtain the expanded local image area.
[0231] The text extraction panel displays the target extractable text contained within the expanded local image area.
[0232] Optionally, the extraction module 1502 is used for:
[0233] Determine the intersection between the local image region indicated by the region selection operation and each of the extractable text regions;
[0234] If there is an intersection with the extractable text region, and the intersection region is smaller than the extractable text region, the local image region is automatically expanded based on the extractable text region to obtain the expanded local image region. The intersection region between the expanded local image region and the extractable text region is the extractable text region.
[0235] Optionally, the extraction module 1502 is used for:
[0236] If there is an intersection with the extractable text region, and the intersection region is smaller than the extractable text region, then the overflow text contained outside the intersection region in the extractable text region is determined.
[0237] In the case where the overflowing text cannot be discarded, the local image region is automatically expanded based on the extractable text region to obtain the expanded local image region.
[0238] If the overflowing text is discardable, the target extractable text contained within the local image area is displayed in the text extraction panel.
[0239] Optionally, the extraction module 1502 is used for:
[0240] Determine the text relevance between the text within the intersection area and the overflow text;
[0241] If the text relevance is greater than the relevance threshold, it is determined that the overflowing text cannot be discarded;
[0242] If the text relevance is less than the relevance threshold, the overflowing text is determined to be discardable.
[0243] Optionally, display module 1501 is also used for:
[0244] A partial image corresponding to the local image area is displayed in the image display area outside the text extraction panel, and the partial image is magnified to the maximum scale of the image display area.
[0245] Optionally, the extraction module 1502 is also used for:
[0246] In response to the region selection operation on the local image, the local image displayed in the image display area and the target extractable text displayed in the text extraction panel are updated.
[0247] Optionally, the extractable text region includes at least one of the following extractable texts:
[0248] The image text in the image, the translated text corresponding to the image text in the image, and the text represented by the graphic code in the image;
[0249] The translated text is displayed on top of the corresponding image text.
[0250] Optionally, display module 1501 is used for:
[0251] In response to the text recognition operation on the image, a first type of extractable text region and / or a second type of extractable text region in the image are highlighted. The extractable text contained in the first type of extractable text region is the image text, and the extractable text contained in the second type of extractable text region is the text represented by the graphic code.
[0252] In response to a text translation operation on the image, a third type of extractable text region in the image is highlighted, wherein the extractable text contained in the third type of extractable text region is the translated text.
[0253] Optionally, the extraction module 1502 is used for:
[0254] In response to the region selection operation on the image, the local image region indicated by the region selection operation is highlighted;
[0255] Once the selected area of the local image is bounded, in response to the text extraction operation, the target extractable text is displayed in the text extraction panel.
[0256] Optionally, the extraction module 1502 is used for:
[0257] Once the selected local image area is defined, a magnified local image corresponding to that local image area is displayed.
[0258] In response to the text extraction operation, the target extractable text is displayed in the text extraction panel.
[0259] Optionally, display module 1501 is also used for:
[0260] In response to the undo operation of the selection box, the display of the partial image is stopped, and the image is displayed again.
[0261] Optionally, the extraction module 1502 is used for:
[0262] In response to the text extraction operation, the full extractable text contained in the image is displayed in the text extraction panel, and the image is displayed in the image display area outside the text extraction panel;
[0263] In response to the region selection operation on the image, the local image region indicated by the region selection operation is highlighted in the image display area;
[0264] Once the selected area of the local image is defined, the target extractable text is displayed in the text extraction panel.
[0265] Optionally, the extraction module 1502 is used for:
[0266] In response to the region selection operation of the image, the extractable text contained within the local image region in the text extraction panel is highlighted.
[0267] In summary, in this embodiment of the application, when it is necessary to extract extractable text from a local image region, the user can select the desired local image region by performing a region selection operation. The terminal then displays the extractable text located within that local image region in the text extraction panel, facilitating text selection. Compared to locating the desired text from the full extractable text displayed in the text extraction panel, or first cropping the image and then performing text recognition and extraction on the cropped image, directly selecting a region on the image is more efficient and more intuitive for the user.
[0268] It should be noted that the apparatus provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the apparatus can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and their implementation process can be found in the method embodiments, which will not be repeated here.
[0269] Referring to Figure 16, which is a schematic diagram of the structure of a terminal provided in an exemplary embodiment of this application, the terminal may include one or more of the following components: a processor 1610 and a memory 1620.
[0270] Optionally, the processor 1610 connects various parts within the electronic device using various interfaces and lines. It executes various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 1620, and by calling data stored in the memory 1620. Optionally, the processor 1610 can be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 1610 can integrate one or more of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), Neural-network Processing Unit (NPU), and baseband chip. Specifically, the CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for the touchscreen display; the NPU implements artificial intelligence (AI) functions; and the baseband chip handles wireless communication. It is understandable that the aforementioned baseband chip may not be integrated into the processor 1610, but may be implemented as a separate chip.
[0271] The memory 1620 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 1620 may include a non-transitory computer-readable storage medium. The memory 1620 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 1620 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function, instructions for implementing the various method embodiments described below, etc.; the data storage area may store data created based on the use of the electronic device, etc.
[0272] In addition, those skilled in the art will understand that the structure of the terminal shown in the above figures does not constitute a limitation on the terminal. The terminal may include more (e.g., screen, communication components, sensor components) or fewer components than shown, or combine certain components, or have different component arrangements.
[0273] This application provides a computer-readable storage medium storing at least one computer instruction, which is executed by a processor to implement the image text extraction method as described in the above embodiments.
[0274] On the other hand, embodiments of this application provide a computer program product, which includes computer instructions that, when executed by a processor, implement the method for extracting text from images as described in the above embodiments.
[0275] Those skilled in the art will recognize that the functions described in the embodiments of this application in one or more of the above examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transfer of a computer program from one place to another. Storage media can be any available medium that can be accessed by a general-purpose or special-purpose computer.
[0276] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.
Claims
1. A method for extracting text from an image, the method being used on a terminal, the method comprising: Highlight the extractable text region in the image, where the extractable text region is the region containing extractable text; In response to a region selection operation on the image, the target extractable text is displayed in the text extraction panel. The target extractable text is the extractable text contained within the local image region indicated by the region selection operation.
2. The method according to claim 1, wherein, The step of displaying the target extractable text in the text extraction panel in response to a region selection operation on the image includes: Receive the operation of selecting the region of the image; If there is a missing selection area within the local image area indicated by the area selection operation, the local image area is automatically expanded to obtain the expanded local image area. The text extraction panel displays the target extractable text contained within the expanded local image area.
3. The method according to claim 2, wherein, When there is a missing selection area within the local image region indicated by the region selection operation, the local image region is automatically expanded to obtain the expanded local image region, including: Determine the intersection between the local image region indicated by the region selection operation and each of the extractable text regions; If there is an intersection with the extractable text region, and the intersection region is smaller than the extractable text region, the local image region is automatically expanded based on the extractable text region to obtain the expanded local image region. The intersection region between the expanded local image region and the extractable text region is the extractable text region.
4. The method according to claim 3, wherein, When there is an intersection with the extractable text region, and the intersection region is smaller than the extractable text region, automatically expanding the local image region based on the extractable text region to obtain the expanded local image region includes: If there is an intersection with the extractable text region, and the intersection region is smaller than the extractable text region, then the overflow text contained outside the intersection region in the extractable text region is determined. In the case where the overflowing text cannot be discarded, the local image region is automatically expanded based on the extractable text region to obtain the expanded local image region. The method further includes: If the overflowing text is discardable, the target extractable text contained within the local image area is displayed in the text extraction panel.
5. The method according to claim 4, wherein, The method includes: Determine the text relevance between the text within the intersection area and the overflow text; If the text relevance is greater than the relevance threshold, it is determined that the overflowing text cannot be discarded; If the text relevance is less than the relevance threshold, the overflowing text is determined to be discardable.
6. The method according to any one of claims 1 to 5, wherein, The method further includes: A partial image corresponding to the local image area is displayed in the image display area outside the text extraction panel, and the partial image is magnified to the maximum scale of the image display area.
7. The method according to claim 6, wherein, The method further includes: In response to the region selection operation on the local image, the local image displayed in the image display area and the target extractable text displayed in the text extraction panel are updated.
8. The method according to any one of claims 1 to 7, wherein, The extractable text region contains at least one of the following extractable texts: The image text in the image, the translated text corresponding to the image text in the image, and the text represented by the graphic code in the image; The translated text is displayed on top of the corresponding image text.
9. The method according to claim 8, wherein, Highlighting the extractable text region in the image includes at least one of the following: In response to the text recognition operation on the image, a first type of extractable text region and / or a second type of extractable text region in the image are highlighted. The extractable text contained in the first type of extractable text region is the image text, and the extractable text contained in the second type of extractable text region is the text represented by the graphic code. In response to a text translation operation on the image, a third type of extractable text region in the image is highlighted, wherein the extractable text contained in the third type of extractable text region is the translated text.
10. The method according to any one of claims 1 to 9, wherein, The step of displaying the target extractable text in the text extraction panel in response to a region selection operation on the image includes: In response to the region selection operation on the image, the local image region indicated by the region selection operation is highlighted; Once the selected area of the local image is bounded, in response to the text extraction operation, the target extractable text is displayed in the text extraction panel.
11. The method according to claim 10, wherein, When the selected area of the local image is selected, in response to the text extraction operation, the target extractable text is displayed in the text extraction panel, including: Once the selected local image area is defined, a magnified local image corresponding to that local image area is displayed. In response to the text extraction operation, the target extractable text is displayed in the text extraction panel.
12. The method according to claim 11, wherein, After displaying the magnified local image corresponding to the local image region, the method further includes: In response to the undo operation of the selection box, the display of the partial image is stopped, and the image is displayed again.
13. The method according to any one of claims 1 to 9, wherein, The method further includes: In response to the text extraction operation, the full extractable text contained in the image is displayed in the text extraction panel, and the image is displayed in the image display area outside the text extraction panel; In response to a region selection operation on the image, the target extractable text is displayed in the text extraction panel, including: In response to the region selection operation on the image, the local image region indicated by the region selection operation is highlighted in the image display area; Once the selected area of the local image is defined, the target extractable text is displayed in the text extraction panel.
14. The method according to claim 13, wherein, The method further includes: In response to the region selection operation of the image, the extractable text contained within the local image region in the text extraction panel is highlighted.
15. A device for extracting text from an image, wherein, The device includes: The display module is used to highlight the extractable text area in the image, wherein the extractable text area is the area containing extractable text; The extraction module is used to display target extractable text in the text extraction panel in response to a region selection operation on the image. The target extractable text is the extractable text contained within the local image region indicated by the region selection operation.
16. A terminal comprising a processor and a memory, the memory storing at least one computer instruction, the at least one computer instruction being loaded and executed by the processor to implement the method for extracting text from an image as described in any one of claims 1 to 14.
17. A computer-readable storage medium storing at least one computer instruction, the at least one computer instruction being executed by a processor to implement the method for extracting text from an image as described in any one of claims 1 to 14.
18. A computer program product comprising computer instructions that, when executed by a processor, implement the method for extracting text from an image as described in any one of claims 1 to 14.
