A text reading method and apparatus

Abstract

The application discloses a text reading method and device, and relates to the technical field of computers. A specific embodiment of the method comprises the following steps: acquiring an image, performing optical character recognition on the image to obtain first text information; screening a plurality of characters in the first text information whose confidence is lower than a preset threshold value and replacing the plurality of characters with a preset first string; replacing target characters in the first text information with a preset second string, taking the second string at a specified position as a grouping identifier, and then performing grouping processing on the first text information; and inputting the processed first text information into a trained deep learning model in groups, so as to replace the first string with a first output for each group of text information, and delete the second string or the second string and the text information after the second string according to a second output, to obtain second text information. Thus, the embodiment of the application can solve the technical problem of poor watermark separation effect when reading an image text.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a text reading method and apparatus. Background Technology

[0002] Currently, text reading software is widely used, providing users with convenient and fast file scanning and information retrieval services. It can be applied to various text format conversion scenarios, greatly improving the efficiency of users in collecting information.

[0003] In the process of realizing this invention, the inventors discovered at least the following problems in the prior art:

[0004] While users enjoy the convenience and efficiency of text reading functions, issues such as missing text characters and reading errors are common, especially when the image being read contains a watermark. If the area of ​​the watermark text overlaps with the area of ​​the target text, the error rate for characters and sentences in the target text reading is even more severe. Furthermore, although existing text reading software provides the function of concatenating text obtained from optical character recognition (OCR), since watermark text and target text are usually not in the same direction, OCR is very likely to recognize the watermark text as a separate text block and concatenate it into other text blocks, resulting in a chaotic final text. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a text reading method and apparatus that can solve the technical problem of poor watermark separation effect when reading image text in existing methods.

[0006] To achieve the above objectives, according to one aspect of the present invention, a text reading method is provided, comprising: acquiring an image; performing optical character recognition on the image to obtain first text information; filtering out multiple characters in the first text information whose confidence level is lower than a preset threshold and replacing them with a preset first string; identifying target characters in the first text information and replacing them with a preset second string; sorting all the second strings; using the second string with a specified sequence number as a group identifier; and then grouping the first text information according to the group identifier; inputting the processed first text information group by group into a trained deep learning model, so as to replace the first string with a first output for each group of text information, and deleting the second string or the text information following the second string according to the second output to obtain second text information.

[0007] Optionally, based on the second output, delete the second string or the text information following the second string, including:

[0008] Obtain a second output to get a processing identifier corresponding to the second string in each group of text information; in response to recognizing that the type of the processing identifier is a preset first type, delete the corresponding second string and the text information following it; in response to recognizing that the type of the processing identifier is a preset second type, delete the corresponding second string; in response to recognizing that the type of the processing identifier is a preset third type, delete the corresponding second string and add a newline to the text information following the second string.

[0009] Optionally, the first text information is then grouped according to the group identifier, including:

[0010] Group the two group identifiers and the text information between them into one group, and replace the group identifier that comes first in the order of each group with the preset starting identifier.

[0011] Optionally, for each set of text information, before replacing the first string with the first output, the following steps are included:

[0012] The processed first text information is input into the trained masked language model and the trained next sentence prediction model in groups, respectively, to obtain the first output of the masked language model and the second output of the next sentence prediction model for each group of text information.

[0013] Optionally, training a deep learning model includes:

[0014] Initial text information is collected and preprocessed to obtain a training set including a watermark string; a deep learning model is constructed, wherein the deep learning model includes a masked language model and a next sentence prediction model; the training set is input into the deep learning model with the goal of making the output corresponding to the training set approximate the corresponding part of the initial text information.

[0015] Optionally, the initial text information is preprocessed, including:

[0016] Obtain the preset watermark text, and generate a watermark string by combining the watermark text and the second string; randomly insert the watermark string into the initial text information multiple times to generate text information with the watermark added corresponding to the initial text information.

[0017] Optionally, it also includes:

[0018] The deep learning model is trained using a deep learning network to minimize the corresponding loss function, thus obtaining a well-trained deep learning model.

[0019] In addition, the present invention also provides a text reading device, including an acquisition module for performing optical character recognition on the image to obtain first text information; a processing module for filtering out multiple characters in the first text information with a confidence level lower than a preset threshold and replacing them with a preset first string; a module for identifying target characters in the first text information and replacing them with a preset second string, sorting all the second strings, using the second string with a specified sequence number as a group identifier, and then grouping the first text information according to the group identifier; and an application module for inputting the processed first text information group by group into a trained deep learning model, so as to replace the first string with a first output for each group of text information, and delete the second string or the text information following the second string according to the second output to obtain second text information.

[0020] Optionally, it further includes: a training module for collecting initial text information, preprocessing the initial text information to obtain a training set including a watermark string; constructing a deep learning model, wherein the deep learning model includes a masking language model and a next-sentence prediction model; and inputting the training set into the deep learning model with the goal of making the output corresponding to the training set approximate the corresponding part of the initial text information.

[0021] One embodiment of the above invention has the following advantages or beneficial effects: The present invention trains a deep learning model, including a masking language model and a next-sentence prediction model, using a training set including a watermark string to obtain a trained deep learning model. This step realizes the construction and training of a model with watermark separation functionality. Furthermore, by acquiring an image and performing optical character recognition on the image to obtain first text information, the format conversion from image to information text is completed. This achieves the approximate recognition of the text information included in the image using existing optical character recognition technology, laying the groundwork for subsequent refined processing of the text using the trained model. Simultaneously, by filtering out multiple characters in the first text information with confidence levels below a preset threshold and replacing them with a preset first string, the filtering of characters suspected of overlapping with the watermark in the approximate identified text information is completed, and the watermark separation function is realized based on the masking language... The model's input format standardizes and replaces the characters obtained from the above filtering process, avoiding text format chaos caused by the processing. Furthermore, by replacing the target characters in the first text information with a preset second string, and grouping the first text information according to the text position of the second string, the model further standardizes and replaces the processed text information based on the input format of the next sentence prediction model. This ensures that the input information contains the two model-preset identifiable identifiers and meets the length requirements, avoiding an increase in the final text reading error rate due to input format issues. Finally, by inputting the processed first text information group by group into the trained deep learning model, the model's output is used to process the first text information to obtain the second text information. This realizes the application of the trained model and completes the optimization of the roughly identified text information.

[0022] The further effects of the aforementioned unconventional alternative methods will be explained below in conjunction with specific implementation methods. Attached Figure Description

[0023] The accompanying drawings are provided to better understand the invention and are not intended to unduly limit the scope of the invention. Wherein:

[0024] Figure 1 This is a schematic diagram of the main flow of the text reading method according to the first embodiment of the present invention;

[0025] Figure 2 This is a schematic diagram of text information replacement processing according to an embodiment of the present invention;

[0026] Figure 3 This is a schematic diagram of the target output of a deep learning model according to an embodiment of the present invention;

[0027] Figure 4This is a schematic diagram of the second output of a deep learning model according to an embodiment of the present invention;

[0028] Figure 5 This is a schematic diagram of the training set in the preprocessing of a deep learning model according to an embodiment of the present invention;

[0029] Figure 6 This is a schematic diagram of the main flow of the text reading method according to the second embodiment of the present invention;

[0030] Figure 7 This is a schematic diagram of the main modules of a text reading device according to a first embodiment of the present invention;

[0031] Figure 8 This is a schematic diagram of the main modules of a text reading device according to a second embodiment of the present invention;

[0032] Figure 9 This is an exemplary system architecture diagram in which embodiments of the present invention can be applied;

[0033] Figure 10 This is a schematic diagram of the structure of a computer system suitable for implementing terminal devices or servers of the present invention. Detailed Implementation

[0034] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0035] Figure 1 This is a schematic diagram of the main flow of the text reading method according to the first embodiment of the present invention, as shown below. Figure 1 As shown, the text reading method includes:

[0036] Step S101: Acquire an image, perform optical character recognition on the image, and obtain first text information.

[0037] In this embodiment, after acquiring the image of the text to be read, optical character recognition (OCR) technology is first used to roughly read the text information included in the image. This step achieves the format conversion of the image into text information. However, since the accuracy of existing OCR technology still needs to be improved, the first text information may have reading errors due to the following reasons: reading errors caused by the overlap of object text and watermark text; and text layout errors caused by the direction of the watermark text.

[0038] Step S102: Filter out multiple characters in the first text information whose confidence level is lower than a preset threshold, and replace them with a preset first string.

[0039] In this embodiment, characters with a confidence level below a preset threshold in the first text information are filtered out. This means filtering out characters in the first text information that are suspected of overlapping with the watermark. Based on the input specifications of a preset masking language model, these characters are uniformly replaced with a preset first string to ensure that the model can identify the location of the text to be processed in the input and perform corresponding processing. Figure 2 As shown, the first string after replacing some characters is "[Mask]" in the figure.

[0040] In some embodiments, the average confidence level calculated after obtaining the text of the optical character recognition image can also be obtained, and the characters can be sorted according to the confidence level of the characters in the first text information to filter out characters in the first text information whose confidence level is lower than the average level and whose ranking is in the last preset value, for example, the last 15%, so as to replace the characters as suspected characters that overlap with the watermark.

[0041] Step S103: Identify the target character in the first text information and replace it with a preset second string. Sort all the second strings and use the second string with the specified sequence number as a group identifier. Then, group the first text information according to the group identifier.

[0042] In this embodiment, the specified sequence number of the second string used as the group identifier can be determined based on the configuration of the trained model. For example, it can be set that each statement of the first text information is processed by the model only once, in which case it is sufficient to determine that the sequence number corresponding to the group identifier is unique; alternatively, it can be set that each statement of the first text information is processed by the model multiple times, such as specifying the sequence numbers as {1,4},{2,5},{3,6}, which indicates that the text information including four consecutive second strings is input into the model as a group, and each statement in the input information is processed by the model at most multiple times.

[0043] In some embodiments, to enable the trained model to recognize the first grouped text information more efficiently, the two grouping identifiers and the text information between them can be grouped together, and the first grouping identifier in each group can be replaced with a preset starting identifier. After this step, each group of text information input to the trained model includes a preset starting identifier and an ending identifier determined based on the model's parameters.

[0044] For example, such as Figure 2As shown, three "statements" from the first text information can be grouped together and input into a trained model for prediction. Here, the text information between two adjacent second strings is considered a single "statement." Therefore, each group of input text information must include four adjacent second strings, with the first second string replaced by a preset start identifier. For the model, the start identifier is used to determine the beginning of the input information, and the third second string after the start identifier is used to determine the end of the input information. Figure 2 In this context, the default starting identifier is "[cls]", and the default second string is "[sep]".

[0045] Step S104: Input the processed first text information into the trained deep learning model group by group, so that the first output replaces the first string for each group of text information, and delete the second string or the text information after the second string according to the second output to obtain the second text information.

[0046] In some embodiments, in order to simultaneously realize the functions of restoring the replaced characters of the input text information and separating the watermarked text and segmenting the text, the processed first text information can be input into the trained masking language model and the trained next sentence prediction model in groups, respectively, to obtain the first output of the masking language model and the second output of the next sentence prediction model for each group of text information.

[0047] In other embodiments, the text information following the second string may be: the text information located between the second string and the second string that follows it.

[0048] For example, such as Figure 3 As shown in the figure, the MLM target answer is the first target output corresponding to the masked character model mentioned above, and the NSP target answer is the second target output corresponding to the next sentence prediction model mentioned above. The working principle of the masked character model is as follows: it identifies each first string after the third string in the input information, reads the context corresponding to each first string, and thus restores the character at the text position corresponding to the first string to obtain the first output. The first output of the masked language model is the probability value predicted for the position "[Mask]", which is a 21128-dimensional vector, corresponding to the probability of each of the 21128 commonly used characters. The character with the highest probability in the predicted answer is taken as the output. The working principle of the next sentence prediction model is as follows: it identifies the starting identifier in the input information, i.e., a specified number of consecutive second strings after "[cls]", which is set to 3 in the figure. It reads the context corresponding to each second string one by one to determine whether the text information before and after each second string is continuous, and whether the text information after it is watermark text, to obtain the second output.

[0049] In some embodiments, to make the second output corresponding to the next sentence prediction model more concise and clear, different processing identifiers can be defined according to different situations. The specific operations include: obtaining the second output to obtain the processing identifier corresponding to the second string in each group of text information; in response to identifying that the type of the processing identifier is a preset first type, deleting the corresponding second string and the text information following it; in response to identifying that the type of the processing identifier is a preset second type, deleting the corresponding second string; in response to identifying that the type of the processing identifier is a preset third type, deleting the corresponding second string and adding a newline to the text information following the second string.

[0050] For example, in Figure 3 The target answer in NSP is the second output mentioned above. The specific output is identified by two numbers. A leading 0 indicates the text preceding the first "[sep]" in the input; a leading 1 indicates the text preceding the second "[sep]"; and a leading 2 indicates the text preceding the third "[sep]". A trailing 1 corresponds to the second type of identifier, where the text before and after the second string should be continuous, requiring only the deletion of the second string. A trailing 2 corresponds to the third type of identifier, indicating that the text before and after the second string is not continuous, but the text between the second string and the following string must be retained; therefore, the second string is deleted, and a newline is created at the same position. A trailing 0 corresponds to the first type of identifier, indicating that the text following the second string is determined to be watermark text and must be discarded.

[0051] In some embodiments, by determining the corresponding sequence number of the group identifier, the model can process the sentences included in the input information multiple times. For the masked language model, the corresponding output processing can be adjusted to sum and average the three probability values ​​of each "[Mask]" position, or other methods can be used to make the vector sum equal to one, and then the position with the highest probability is selected to decode into a specific word. Similarly, for the next sentence prediction model, the prediction results for the same "[sep]" position can be combined and the corresponding identifier can be output. For example, such as Figure 4 As shown, "C" was judged twice, so the probability results obtained from the two judgments were summed, and the identifier corresponding to the result with the larger probability sum was selected.

[0052] In some embodiments, to obtain a trained deep learning model, initial text information can be collected, preprocessed to obtain a training set including the watermark string, and then a deep learning model can be constructed, comprising a masked language model and a next-sentence prediction model. The training set is then input into the deep learning model, with the goal of making the output corresponding to the training set approximate the corresponding part of the initial text information. This step achieves the construction and training of a model capable of recognizing and separating watermarked text from text information.

[0053] In some embodiments, to obtain the training set of the deep learning model more quickly and efficiently, a preset watermark text can be obtained, and a watermark string can be generated by combining the watermark text and the second string. The watermark string is then randomly inserted into the initial text information multiple times to generate watermarked text information corresponding to the initial text information. In a further embodiment, the preset watermark can be generated according to business needs. For example, when reading text from a screenshot of a fixed website, the website watermark may be relatively fixed, such as "name + mobile phone number" or "name + employee number". Based on business patterns, a simulated watermark can be generated, resulting in a more ideal model training effect. In a further embodiment, one or more consecutive characters in the watermark text can be selected, and a second string can be inserted before and after the selected character or consecutive characters to obtain the watermark string. Furthermore, the second string and the watermark string can be randomly inserted multiple times at a specified ratio. For example, as shown... Figure 5 As shown, the watermark string corresponds to "[sep]test watermark[sep]" in the figure.

[0054] In a further embodiment, the preprocessing of the initial text information, in addition to randomly inserting watermark strings, also includes: randomly selecting a preset number of characters in the initial text information and replacing them with a preset first string to simulate the situation where the selected characters overlap with the watermark and cannot be recognized, so as to train the masked language model for character prediction based on the corresponding target answer in the initial text information; identifying target characters in the initial text information and replacing them with a preset second string, thereby uniformly replacing newline characters, so as to train the masked language model for text segmentation based on the corresponding target answer in the initial text information; sorting the second strings included in the initial text information, using the second strings with specified serial numbers as group identifiers, and then grouping the third text information according to the group identifiers to obtain input information that conforms to the model.

[0055] In other embodiments, to improve the processing accuracy of the deep learning model, a deep learning network can be used to train the deep learning model, minimizing the corresponding loss function to obtain a trained deep learning model. For example, the loss function corresponding to the deep learning model can be the kernel function of the loss function of the masked language model and the loss function of the next sentence prediction model.

[0056] Figure 6 This is a schematic diagram of the main flow of a text reading method according to a second embodiment of the present invention, the text reading method including:

[0057] Step S601: Collect initial text information, preprocess the initial text information to obtain a training set including the watermark string.

[0058] Step S602: Construct a deep learning model, wherein the deep learning model includes a masked language model and a next sentence prediction model.

[0059] Step S603: Input the training set into the deep learning model, use the deep learning network to train the deep learning model, so that the corresponding loss function is minimized, and obtain the trained deep learning model.

[0060] Step S604: Acquire an image, perform optical character recognition on the image, and obtain first text information.

[0061] Step S605: Filter out multiple characters in the first text information whose confidence level is lower than a preset threshold, and replace them with a preset first string.

[0062] Step S606: Identify the target character in the first text information and replace it with a preset second string. Sort all the second strings and use the second string with the specified sequence number as a group identifier.

[0063] Step S607: Divide the two group identifiers and the text information between them into one group, and replace the group identifier that comes first in the order of each group with a preset starting identifier.

[0064] Step S608: Input the processed first text information into the trained masked language model and the trained next sentence prediction model in groups, respectively, to obtain the first output of the masked language model and the second output of the next sentence prediction model for each group of text information.

[0065] Step S609: For each group of text information, replace the first string with the first output, and delete the second string or the text information following the second string according to the second output.

[0066] Preferably, the processing identifier included in the second output is identified; in response to identifying the type of the processing identifier as a preset first type, the corresponding second string and the text information following it are deleted; in response to identifying the type of the processing identifier as a preset second type, the corresponding second string is deleted; in response to identifying the type of the processing identifier as a preset third type, the corresponding second string is deleted, and the text information following the second string is replaced with a newline.

[0067] Step S610: Obtain the second text information.

[0068] Figure 7 This is a schematic diagram of the main modules of a text reading device according to an embodiment of the present invention, such as... Figure 4 As shown, the text reading device 700 includes an acquisition module 701, a processing module 702, and an application module 703. The acquisition module 701 acquires an image and performs optical character recognition on the image to obtain first text information. The processing module 702 filters out multiple characters in the first text information whose confidence level is lower than a preset threshold and replaces them with a preset first string. It identifies target characters in the first text information and replaces them with a preset second string. All second strings are sorted, and the second strings with specified serial numbers are used as group identifiers. The first text information is then grouped according to these group identifiers. The application module 703 inputs the processed first text information group by group into a trained deep learning model. For each group of text information, the first output replaces the first string, and the second output deletes the second string or the text information following the second string, thus obtaining second text information.

[0069] In some embodiments, the processing module 702 further performs grouping processing on the first text information according to the grouping identifier, and is also used to: group the two grouping identifiers and the text information between them into one group, and replace the grouping identifier that comes first in each group with a preset starting identifier.

[0070] In some embodiments, before replacing the first string with the first output for each group of text information, the application module 703 is further configured to: input the processed first text information into the trained masked language model and the trained next sentence prediction model in turn, to obtain the first output corresponding to the masked language model and the second output corresponding to the next sentence prediction model for each group of text information.

[0071] In some embodiments, the application module 703 deletes the second string or the text information following the second string based on the second output, and is further configured to: obtain the second output to obtain a processing identifier corresponding to the second string in each group of text information; in response to identifying that the type of the processing identifier is a preset first type, delete the corresponding second string and the text information following it; in response to identifying that the type of the processing identifier is a preset second type, delete the corresponding second string; in response to identifying that the type of the processing identifier is a preset third type, delete the corresponding second string and add a newline to the text information following the second string.

[0072] Figure 8 This is a schematic diagram of the main modules of a text reading device according to an embodiment of the present invention, such as... Figure 8 As shown, the text reading device 800 includes an acquisition module 801, a watermark loading module 802, an application module 803, and a training module 804. The training module 804 collects initial text information, preprocesses the initial text information to obtain a training set including a watermark string; constructs a deep learning model, wherein the deep learning model includes a masked language model and a next sentence prediction model; inputs the training set into the deep learning model with the goal of making the output of the training set approximate the corresponding part of the initial text information; the acquisition module 701 acquires an image, performs optical character recognition on the image to obtain first text information; the processing module 702 filters out multiple characters in the first text information with a confidence level lower than a preset threshold and replaces them with a preset first string; identifies target characters in the first text information and replaces them with a preset second string, sorts all the second strings, uses the second string with a specified sequence number as a group identifier, and then groups the first text information according to the group identifier; the application module 703 inputs the processed first text information into the trained deep learning model group by group, so as to replace the first string with the first output for each group of text information, and delete the second string or the text information after the second string according to the second output to obtain the second text information.

[0073] In some embodiments, the training module 804 is further configured to: obtain a preset watermark text, generate a watermark string by combining the watermark text and the second string; and randomly insert the watermark string into the initial text information multiple times to generate text information with the watermark added corresponding to the initial text information.

[0074] In some embodiments, the training module 804 is further configured to: train the deep learning model using a deep learning network to minimize the corresponding loss function, thereby obtaining a trained deep learning model.

[0075] It should be noted that the text reading method and the text reading device described in this invention are related in their specific implementation, so repeated content will not be described again.

[0076] Figure 9 An exemplary system architecture 900 is shown for which the text reading method or text reading device of embodiments of the present invention can be applied.

[0077] like Figure 9 As shown, system architecture 900 may include terminal devices 901, 902, and 903, network 904, and server 905. Network 904 is used as a medium to provide a communication link between terminal devices 901, 902, and 903 and server 905. Network 904 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0078] Users can use terminal devices 901, 902, and 903 to interact with server 905 via network 904 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 901, 902, and 903.

[0079] Terminal devices 901, 902, and 903 can be various electronic devices with text reading screens and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0080] Server 905 can be a server that provides various services, such as a backend management server that supports users using terminal devices 901, 902, and 903 (for example only). The backend management server can analyze and process data such as received product information query requests, and feed back the processing results (such as target push information, product information - for example only) to the terminal devices.

[0081] It should be noted that the text reading method provided in this embodiment of the invention is generally executed by server 905, and correspondingly, the computing device is generally located in server 905.

[0082] It should be understood that Figure 9 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0083] The following is for reference. Figure 10 It shows a schematic diagram of the structure of a computer system 1000 suitable for implementing a terminal device of the present invention. Figure 10 The terminal device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0084] like Figure 10 As shown, the computer system 1000 includes a central processing unit (CPU) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage section 1008 into a random access memory (RAM) 1003. The RAM 1003 also stores various programs and data required for the operation of the computer system 1000. The CPU 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0085] The following components are connected to the I / O interface 1005: an input section 1005 including a keyboard, mouse, etc.; an output section 1006 including a cathode ray tube (CRT), liquid crystal text reader (LCD), etc., and a speaker, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card, modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 1010 as needed so that computer programs read from it can be installed into the storage section 1008 as needed.

[0086] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1009, and / or installed from removable medium 1011. When the computer program is executed by central processing unit (CPU) 1001, it performs the functions defined in the system of this invention.

[0087] It should be noted that the computer-readable medium shown in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0088] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0089] The modules described in the embodiments of the present invention can be implemented in software or hardware. The described modules can also be housed in a processor; for example, a processor can be described as including an acquisition module, a processing module, and an application module. The names of these modules do not necessarily limit the functionality of the module itself.

[0090] In another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently without being assembled into the device. The computer-readable medium carries one or more programs, which, when executed by the device, cause the device to include: acquiring an image; performing optical character recognition on the image to obtain first text information; filtering out multiple characters in the first text information whose confidence level is lower than a preset threshold and replacing them with a preset first string; identifying target characters in the first text information and replacing them with a preset second string; sorting all the second strings; using the second string with a specified sequence number as a group identifier; and grouping the first text information according to the group identifier; inputting the processed first text information group by group into a trained deep learning model, so as to replace the first string with a first output for each group of text information, and deleting the second string or the text information following the second string according to the second output to obtain second text information.

[0091] According to the technical solution of the present invention, the problem of poor watermark separation effect when reading image text can be solved.

[0092] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A text reading method, characterized in that, include: Acquire an image, perform optical character recognition on the image, and obtain first text information; Filter out multiple characters in the first text information whose confidence level is lower than a preset threshold, and replace them with a preset first string; The system identifies target characters in the first text information and replaces them with preset second strings. All second strings are sorted, and the second string with a specified sequence number is used as a group identifier. The first text information is then grouped according to the group identifier, including: grouping the two group identifiers and the text information between them into one group, and replacing the first group identifier in each group with a preset starting identifier. Alternatively, if each statement in the first text information is to be processed by the model only once, it is sufficient to ensure that the sequence number corresponding to the group identifier is unique; or if each statement in the first text information is to be processed by the model multiple times, it indicates that multiple consecutive second strings containing text information are input into the model as a group, and each statement in the input information is processed by the model at most multiple times. The processed first text information is input into the trained deep learning model in groups. For each group of text information, the first output replaces the first string, and the second string or the text information following the second string is deleted according to the second output to obtain the second text information. This includes: obtaining the second output to obtain the processing identifier corresponding to the second string in each group of text information; in response to identifying the type of the processing identifier as a preset first type, deleting the corresponding second string and the text information following it; in response to identifying the type of the processing identifier as a preset second type, deleting the corresponding second string; in response to identifying the type of the processing identifier as a preset third type, deleting the corresponding second string and adding a newline to the text information following the second string. The initial text information is preprocessed during the training of the deep learning model, including: randomly selecting a preset number of characters from the initial text information and replacing them with a preset first string; simulating the situation where the selected characters overlap with the watermark and cannot be recognized; training the deep learning model for character prediction based on the corresponding target answer in the initial text information; recognizing the target characters in the initial text information and replacing them with a preset second string; training the deep learning model for text segmentation based on the corresponding target answer in the initial text information; sorting the second strings included in the initial text information; using the second strings with specified numbers as group identifiers; and then grouping the third text information according to the group identifiers to obtain input information that conforms to the deep learning model. Before replacing the first string with the first output for each group of text information, the process includes: inputting the processed first text information into the trained masked language model and the trained next sentence prediction model one by one to obtain the first output of the masked language model and the second output of the next sentence prediction model for each group of text information.

2. The method according to claim 1, characterized in that, Training deep learning models includes: Collect initial text information, preprocess the initial text information to obtain a training set including the watermark string; Construct a deep learning model, wherein the deep learning model includes a masked language model and a next sentence prediction model; The training set is input into the deep learning model with the goal of making the output corresponding to the training set approximate the corresponding part of the initial text information.

3. The method according to claim 1, characterized in that, Preprocessing the initial text information includes: Obtain the preset watermark text, and generate a watermark string by combining the watermark text and the second string; The watermark string is randomly inserted multiple times into the initial text information to generate text information with the watermark added corresponding to the initial text information.

4. The method according to any one of claims 1-3, characterized in that, Also includes: The deep learning model is trained using a deep learning network to minimize the corresponding loss function, thus obtaining a well-trained deep learning model.

5. A text reading device, characterized in that, include: The acquisition module is used to perform optical character recognition on the image to obtain the first text information; The processing module is used to filter out multiple characters in the first text information whose confidence level is lower than a preset threshold, and replace them with a preset first string; The system is used to identify target characters in the first text information and replace them with preset second strings. All second strings are sorted, and the second string with a specified sequence number is used as a group identifier. The first text information is then grouped according to the group identifier, including: grouping the two group identifiers and the text information between them into one group, and replacing the first group identifier in each group with a preset starting identifier. Specifically, if each statement in the first text information is processed by the model only once, it is sufficient to ensure that the sequence number corresponding to the group identifier is not repeated; or if each statement in the first text information is processed by the model multiple times, it indicates that the text information included in multiple consecutive second strings is input into the model as a group, and each statement in the input information is processed by the model at most multiple times. An application module is used to input the processed first text information into a trained deep learning model in groups, so as to replace the first string with the first output for each group of text information, and delete the second string or the text information following the second string according to the second output to obtain the second text information. The module includes: obtaining the second output to obtain a processing identifier corresponding to the second string in each group of text information; deleting the corresponding second string and the text information following it in response to identifying the type of the processing identifier as a preset first type; deleting the corresponding second string in response to identifying the type of the processing identifier as a preset second type; and deleting the corresponding second string and adding a newline to the text information following the second string in response to identifying the type of the processing identifier as a preset third type. The training module is used to: preprocess the initial text information during the training of the deep learning model, including: randomly selecting a preset number of characters in the initial text information and replacing them with a preset first string; simulating the situation where the selected characters overlap with the watermark and cannot be recognized; training the deep learning model for character prediction based on the corresponding target answer in the initial text information; recognizing the target character in the initial text information and replacing it with a preset second string; training the deep learning model for text segmentation based on the corresponding target answer in the initial text information; sorting the second strings included in the initial text information; using the second string with a specified sequence number as a group identifier; and then grouping the third text information according to the group identifier to obtain input information that conforms to the deep learning model. The application module is further configured to input the processed first text information into the trained masked language model and the trained next sentence prediction model in groups, respectively, to obtain the first output of the masked language model and the second output of the next sentence prediction model for each group of text information.

6. The apparatus according to claim 5, characterized in that, Also includes: The training module is used to collect initial text information, preprocess the initial text information, and obtain a training set including the watermark string. A deep learning model is constructed, wherein the deep learning model includes a masked language model and a next sentence prediction model; the training set is input into the deep learning model with the goal of making the output corresponding to the training set approximate the corresponding part of the initial text information.

7. An electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-4.

8. A computer-readable medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-4.

Images