Text recognition method, device, readable medium and electronic device

Abstract

The present disclosure relates to a text recognition method, device, readable medium and electronic device, the method comprising: obtaining a heat map corresponding to each character in a text image to be recognized; performing a first downsampling process on the text image according to a plurality of the heat maps to obtain a feature image corresponding to the text image; and obtaining text content corresponding to the text image according to the feature image. That is, the present disclosure can perform a first downsampling process on the text image according to the heat map corresponding to each character in the text image to be recognized, so that excessive compression of the width or height of the text image can be avoided, so that the text content obtained according to the feature image obtained after the first downsampling process is more accurate, thereby improving the accuracy of text image recognition.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular, to a text recognition method, device, readable medium and electronic device. Background Art

[0002] With the widespread application of text image recognition technology, people have higher and higher requirements for the accuracy of text image recognition, and need to be able to accurately recognize each character in the text image. In related technologies, the text content in the text image is recognized through a neural network model based on deep learning.

[0003] However, for some more complex text images, such as advertisements, movie posters, etc., the titles in the text images often use special layouts. Therefore, when the text image is recognized by the same neural network model, some characters cannot be recognized normally, resulting in a relatively low accuracy rate in text image recognition. Summary of the invention

[0004] This summary is provided to introduce concepts in a brief form that will be described in detail in the detailed description below. This summary is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0005] In a first aspect, the present disclosure provides a text recognition method, the method comprising:

[0006] Obtain the heat map corresponding to each character in the text image to be recognized;

[0007] According to the plurality of heat maps, performing a first downsampling process on the text image to obtain a feature image corresponding to the text image;

[0008] According to the feature image, the text content corresponding to the text image is acquired.

[0009] In a second aspect, the present disclosure provides a text recognition device, the device comprising:

[0010] A heat map acquisition module is used to obtain a heat map corresponding to each character in the text image to be recognized;

[0011] A feature image acquisition module, configured to perform a first downsampling process on the text image according to the plurality of heat maps, to obtain a feature image corresponding to the text image;

[0012] The text content acquisition module is used to acquire the text content corresponding to the text image according to the feature image.

[0013] In a third aspect, the present disclosure provides a computer-readable medium having a computer program stored thereon, which, when executed by a processing device, implements the steps of the method described in the first aspect of the present disclosure.

[0014] In a fourth aspect, the present disclosure provides an electronic device, including:

[0015] a storage device having a computer program stored thereon;

[0016] A processing device is used to execute the computer program in the storage device to implement the steps of the method described in the first aspect of the present disclosure.

[0017] Through the above technical solution, by obtaining the heat map corresponding to each character in the text image to be recognized; performing a first downsampling process on the text image according to the multiple heat maps, obtaining a feature image corresponding to the text image; and obtaining the text content corresponding to the text image according to the feature image. That is to say, the present disclosure can perform a first downsampling process on the text image according to the heat map corresponding to each character in the text image to be recognized, so that excessive compression of the width or height of the text image can be avoided, so that the text content obtained according to the feature image obtained after the first downsampling process is more accurate, thereby improving the accuracy of text image recognition.

[0018] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale. In the drawings:

[0020] Figure 1 is a flowchart of a text recognition method provided according to an exemplary embodiment;

[0021] Figure 2 is a schematic diagram of a text image provided according to an exemplary embodiment;

[0022] Figure 3 is a heat map provided according to an exemplary embodiment;

[0023] Figure 4 is a flowchart of another text recognition method provided according to an exemplary embodiment;

[0024] Figure 5 is a block diagram of a text recognition device provided according to an exemplary embodiment;

[0025] Figure 6 is a block diagram of another text recognition device provided according to an exemplary embodiment;

[0026] Figure 7 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0027] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0028] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0029] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0030] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0031] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0032] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0033] First, the application scenario of the present disclosure is described. Traditional OCR technology is divided into two steps: text detection and text recognition. The text image is segmented by text detection to obtain the text area in the text image, and the text content in the text area is obtained by text recognition. In the related art, text recognition based on deep learning mainly includes CTC (Connectionist Temporal Classification) methods represented by CRNN (Convolutional Recurrent Neural Network) and Attention methods represented by Transformer. The CTC method and the Attention method have strong recognition capabilities for horizontal text images, but for some special text images, such as advertisements, movie posters and other text images, since the titles of text images often use special layouts, including text lines of different heights, all text images are downsampled according to one-dimensional text lines, which will cause the height of two-dimensional text to be over-compressed, resulting in information loss, and thus unable to be recognized normally; all text images are downsampled according to the height of two-dimensional text, which makes the sequence length to be processed longer, the amount of calculation is greatly increased, and the efficiency of text recognition is too low.

[0034] In order to solve the above-mentioned technical problems, the present disclosure provides a text recognition method, device, readable medium and electronic device, which can perform a first down-sampling process on the text image according to the heat map corresponding to each character in the text image to be recognized. In this way, excessive compression of the width or height of the text image can be avoided, so that the text content obtained according to the feature image obtained after the first down-sampling process is more accurate, thereby improving the accuracy of text image recognition.

[0035] The present disclosure is described below in conjunction with specific embodiments.

[0036] Figure 1 is a flowchart of a text recognition method provided according to an exemplary embodiment. Figure 1 As shown, the method may include:

[0037] S101, obtaining a heat map corresponding to each character in a text image to be recognized.

[0038] In this step, after obtaining the text image to be recognized, the text image to be recognized can be input into a pre-trained heat map acquisition model to output the heat map corresponding to each character in the text image. For example, Figure 2 is a schematic diagram of a text image provided according to an exemplary embodiment, such as Figure 2As shown, the text image includes three text lines of different heights: "1", "5", "%", and "EXTRA DE DTO", among which "1" is the highest and "EXTRA DE DTO" is the lowest. Figure 2 The text image shown, Figure 3 is a thermal diagram provided according to an exemplary embodiment, such as Figure 3 As shown, the highest heat map on the left is the heat map corresponding to "1", the two heat maps above are the heat maps corresponding to "5" and "%", and the multiple heat maps below are the heat maps corresponding to each character in "EXTRA DE DTO" (only 4 heat maps are shown in the figure, and other heat maps are not shown).

[0039] Optionally, before obtaining the heat map corresponding to each character in the text image to be recognized, the text image to be recognized can be subjected to a second downsampling process to obtain a downsampled image corresponding to the text image, and then the heat map corresponding to each character in the downsampled image is obtained. The parameters of the second downsampling process and the first downsampling process are different.

[0040] For example, the text image can be subjected to a second downsampling process by a pre-trained feature image acquisition model to obtain a downsampled image corresponding to the text image. The feature image acquisition model can be based on a CNN (Convolutional Neural Networks) framework and trained by a model training method of the prior art, which will not be described in detail here. The CNN framework includes the first two blocks of ResNet18, and the downsampling ratio of the feature image acquisition model for the second downsampling process can be obtained in advance through experimental testing. For example, the downsampling ratio can be 4. If the size corresponding to the text image is (H, W) and the downsampling ratio is N, the size of the feature image obtained after the second downsampling process on the text image is (H / N, W / N).

[0041] S102: Perform a first downsampling process on the text image according to the multiple heat maps to obtain a feature image corresponding to the text image.

[0042] In this step, after obtaining the heat map corresponding to each character in the text image, for each character, a first downsampling process can be performed on the character according to the heat map corresponding to the character through the downsampling method of the prior art to obtain a feature image corresponding to the text image.

[0043] S103: Acquire text content corresponding to the text image according to the feature image.

[0044] In this step, after obtaining the feature image corresponding to the text image, the feature image can be identified to obtain the text content corresponding to the text image. The feature image can be identified by a decoding module, and the implementation of the decoding module can refer to the Transformer model of the prior art. For example, the decoding module can be composed of multiple encoders and multiple decoders.

[0045] By adopting the above method, the text image can be subjected to a first downsampling process according to the heat map corresponding to each character in the text image to be recognized. In this way, excessive compression of the width or height of the text image can be avoided, so that the text content obtained according to the feature image obtained after the first downsampling process is more accurate, thereby improving the accuracy of text image recognition.

[0046] Figure 4 is a flowchart of another text recognition method provided according to an exemplary embodiment. Figure 4 As shown, the method may include:

[0047] S401, input the text image to be recognized into a pre-trained heat map acquisition model, and output the heat map corresponding to each character in the text image.

[0048] In this step, before the text image is input into the heat map acquisition model, the text image can be subjected to a second downsampling process to obtain a downsampled image corresponding to the text image. After that, the downsampled image is input into the heat map acquisition model to output the heat map corresponding to each character in the text image.

[0049] Among them, the heat map acquisition model can be trained in the following way:

[0050] S1. Obtain a sample set.

[0051] The sample set includes a plurality of sample images and a character area corresponding to each character in the sample image.

[0052] For each sample image, the character area corresponding to each character in the sample image can be annotated. For example, the four vertices corresponding to each character in the sample image can be detected by the bounding box method, and the character area corresponding to the character can be determined by the four vertices. It should be noted that the above method for determining the character area is only an example, and the character area can also be determined by other methods in the prior art, which is not limited in the present disclosure.

[0053] S2. Train the target neural network model using the sample set to obtain the thermal map acquisition model.

[0054] After obtaining the sample set, the model training step can be executed repeatedly until the trained target neural network meets the preset stop iteration condition according to the character area and the predicted heat map, and the trained target neural network model is used as the heat map acquisition model, and the predicted heat map is the heat map output after the sample image is input into the trained target neural network model. The model training step may include:

[0055] Multiple sample images are input into the target neural network model, and a prediction heat map corresponding to each sample image is output. When it is determined according to the character area and the prediction heat map that the target neural network model does not satisfy the preset stop iteration condition, a target loss value is determined according to the character area and the prediction heat map, and the parameters of the target neural network model are updated according to the target loss value to obtain a trained target neural network model, and the trained target neural network model is used as a new target neural network model.

[0056] The preset stop iteration condition may be a condition for stopping iteration commonly used in the prior art, and the present disclosure does not limit the preset stop iteration condition.

[0057] S402: Determine the complexity corresponding to the text image according to a plurality of heat maps.

[0058] Among them, the complexity can be used to characterize the two-dimensional degree of the text image. For example, the larger the value of the complexity, the higher the two-dimensional degree of the text image, and the smaller the value of the complexity, the lower the two-dimensional degree of the text image.

[0059] In this step, after obtaining the heat map corresponding to each character in the downsampled image, the number of heat maps in the vertical direction can be obtained, and the maximum number is used as the complexity corresponding to the text image. Figure 3 Taking the heat map shown as an example, the number of heat maps in the vertical direction includes 1 and 2, so it can be determined that the complexity corresponding to the text image is 2.

[0060] S403: Perform the first down-sampling process on the text image according to the complexity of the text image to obtain a feature image corresponding to the text image.

[0061] In this step, in a possible implementation, after determining the complexity of the text image, the difference between the complexity of the text image and a preset complexity threshold can be determined; based on the difference, a target downsampling model is determined from a plurality of pre-trained downsampling models, and different downsampling models correspond to different downsampling ratios; a first downsampling process is performed on the text image through the target downsampling model to obtain a feature image corresponding to the text image.

[0062] Wherein, the downsampling ratio may include a height downsampling ratio and a width downsampling ratio. After obtaining the difference between the complexity of the text image and a preset complexity threshold, the target height downsampling ratio and the target width downsampling ratio corresponding to the text image may be determined according to the difference and the preset sampling ratio threshold. Thereafter, the target downsampling model may be determined from a plurality of downsampling models according to the target height downsampling ratio and the target width downsampling ratio. The preset sampling ratio threshold may be the product of the height downsampling ratio and the width downsampling ratio. After the preset sampling ratio threshold is determined, the height downsampling ratio and the width downsampling ratio may be determined according to the preset sampling ratio threshold. The preset sampling ratio threshold may be obtained according to a pre-tested experiment. For example, the preset sampling ratio threshold may be 8.

[0063] For example, when the preset sampling rate threshold is 8, the multiple downsampling models may include a first downsampling model, a second downsampling model and a third downsampling model. The downsampling rate corresponding to the first downsampling model may be (8,1), the downsampling rate corresponding to the second downsampling model may be (4,2), and the downsampling rate corresponding to the third downsampling model may be (2,4), wherein the first number represents the height downsampling rate and the second number represents the width downsampling rate. If the difference between the complexity of the text image and the preset complexity threshold is 0, indicating that the complexity of the text image is very low and the horizontal degree of the text lines in the text image is very high, then it can be determined that the height downsampling rate corresponding to the text image is very high and the width downsampling rate is very low. For example, the height downsampling rate may be 8 and the width downsampling rate may be 1, that is, the first downsampling model is used as the target downsampling model; if the difference between the complexity of the text image and the preset complexity threshold is 1, indicating that the complexity of the text image is relatively low and the horizontal degree of the text lines in the text image is relatively high, then it can be determined that the height downsampling rate corresponding to the text image is very high and the width downsampling rate is very low. The downsampling ratio is relatively high, and the width downsampling ratio is relatively low. For example, the height downsampling ratio can be 4, and the width downsampling ratio can be 2, that is, the second downsampling model is used as the target downsampling model; if the difference between the complexity of the text image and the preset complexity threshold is 2, it means that the complexity of the text image is relatively high, the horizontal degree of the text line in the text image is relatively low, and the text line is more two-dimensional, then it can be determined that the height downsampling ratio corresponding to the text image is very low, and the width downsampling ratio is very high. For example, the height downsampling ratio can be 2, and the width downsampling ratio can be 4, that is, the third downsampling model is used as the target downsampling model.

[0064] It should be noted that the downsampling ratios corresponding to the above downsampling models are only examples, and the present disclosure does not limit this. Multiple downsampling models can be trained by the model training method of the prior art, which will not be repeated here.

[0065] In another possible implementation, after determining the complexity of the text image, the downsampling ratio corresponding to the complexity can be determined according to a pre-established sampling ratio association relationship, and the sampling ratio association relationship can include downsampling ratios corresponding to different complexities. For example, if the complexity of the text image is 1, the downsampling ratio can be (8,1); if the complexity of the text image is 2, the downsampling ratio can be (4,2); if the complexity of the text image is 3, the downsampling ratio can be (2,4). The above downsampling ratios are for illustrative purposes only and are not limited in the present disclosure.

[0066] Furthermore, after determining the downsampling ratio corresponding to the text image, the text image may be subjected to a first downsampling process using a downsampling process method in the prior art to obtain a feature image corresponding to the text image.

[0067] S404: Acquire text content corresponding to the text image according to the feature image.

[0068] By adopting the above method, a heat map corresponding to each character in the text image to be recognized can be obtained, the complexity of the text image can be determined based on multiple heat maps, and the downsampling ratio for the first downsampling processing of the text image can be determined based on the complexity. In this way, excessive compression of the width or height of the text image can be avoided, so that the text content obtained according to the feature image obtained after the first downsampling processing is more accurate, thereby improving the accuracy of text image recognition; and the downsampling ratio for the first downsampling processing of the text image determined according to the complexity is more accurate, thereby further improving the accuracy of text image recognition.

[0069] Figure 5 is a block diagram of a text recognition device provided according to an exemplary embodiment. Figure 5 As shown, the device may include:

[0070] The heat map acquisition module 501 is used to obtain the heat map corresponding to each character in the text image to be recognized;

[0071] A feature image acquisition module 502 is used to perform a first downsampling process on the text image according to the plurality of heat maps to obtain a feature image corresponding to the text image;

[0072] The text content acquisition module 503 is used to acquire the text content corresponding to the text image according to the feature image.

[0073] Optionally, the heat map acquisition module 501 is further used for:

[0074] The text image to be recognized is input into a pre-trained heat map acquisition model, and the heat map corresponding to each character in the text image is output.

[0075] Optionally, the heat map acquisition module 501 is further used for:

[0076] Acquire a sample set, the sample set comprising a plurality of sample images and a character region corresponding to each character in the sample image;

[0077] The target neural network model is trained using the sample set to obtain the heat map acquisition model.

[0078] Optionally, the feature image acquisition module 502 is further used to:

[0079] Determine the complexity corresponding to the text image according to the multiple heat maps;

[0080] According to the complexity of the text image, the first downsampling process is performed on the text image to obtain a feature image corresponding to the text image.

[0081] Optionally, the feature image acquisition module 502 is further used to:

[0082] Determine the difference between the complexity of the text image and a preset complexity threshold;

[0083] According to the difference, a target downsampling model is determined from a plurality of pre-trained downsampling models, and different downsampling models correspond to different downsampling ratios;

[0084] The first downsampling process is performed on the text image through the target downsampling model to obtain a feature image corresponding to the text image.

[0085] Optionally, the downsampling ratio includes a height downsampling ratio and a width downsampling ratio; the feature image acquisition module 502 is further used to:

[0086] According to the difference and a preset sampling rate threshold, a target height downsampling ratio and a target width downsampling ratio corresponding to the text image are determined;

[0087] The target downsampling model is determined from a plurality of downsampling models according to the target height downsampling ratio and the target width downsampling ratio.

[0088] Optionally, Figure 6 is a block diagram of another text recognition device provided according to an exemplary embodiment. Figure 6 As shown, the device also includes:

[0089] A down-sampled image acquisition module 504 is used to perform a second down-sampling process on the text image to be recognized to obtain a down-sampled image corresponding to the text image;

[0090] The heat map acquisition module 501 is also used for:

[0091] Get the heat map corresponding to each character in the downsampled image.

[0092] Through the above-mentioned device, the text image can be subjected to a first downsampling process according to the heat map corresponding to each character in the text image to be recognized. In this way, excessive compression of the width or height of the text image can be avoided, so that the text content obtained according to the feature image obtained after the first downsampling process is more accurate, thereby improving the accuracy of text image recognition.

[0093] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0094] Reference below Figure 7 , which shows a schematic diagram of the structure of an electronic device 700 (such as a terminal device or a server) suitable for implementing the embodiment of the present disclosure. The terminal device in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0095] like Figure 7 As shown, the electronic device 700 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the electronic device 700 are also stored. The processing device 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0096] Typically, the following devices may be connected to the I / O interface 705: an input device 706 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 708 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 709. The communication device 709 may allow the electronic device 700 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 7 The electronic device 700 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0097] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 709, or installed from a storage device 708, or installed from a ROM 702. When the computer program is executed by the processing device 701, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0098] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0099] In some embodiments, the electronic device can communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0100] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0101] The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: obtains a heat map corresponding to each character in the text image to be recognized; performs a first down-sampling process on the text image according to multiple heat maps to obtain a feature image corresponding to the text image; and obtains text content corresponding to the text image according to the feature image.

[0102] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0103] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0104] The modules involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a module does not limit the module itself in some cases. For example, a heat map acquisition module may also be described as a "module for acquiring a heat map corresponding to each character in a text image to be recognized."

[0105] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0106] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0107] According to one or more embodiments of the present disclosure, Example 1 provides a text recognition method, including: obtaining a heat map corresponding to each character in a text image to be recognized; performing a first downsampling process on the text image based on multiple heat maps to obtain a feature image corresponding to the text image; and obtaining text content corresponding to the text image based on the feature image.

[0108] According to one or more embodiments of the present disclosure, Example 2 provides the method of Example 1, wherein obtaining a heat map corresponding to each character in a text image to be recognized includes: inputting the text image to be recognized into a pre-trained heat map acquisition model, and outputting a heat map corresponding to each character in the text image.

[0109] According to one or more embodiments of the present disclosure, Example 3 provides the method of Example 2, and the heat map acquisition model is trained in the following manner: obtaining a sample set, the sample set including multiple sample images, and a character area corresponding to each character in the sample image; training a target neural network model with the sample set to obtain the heat map acquisition model.

[0110] According to one or more embodiments of the present disclosure, Example 4 provides the method of Example 1, wherein the first downsampling process is performed on the text image according to the multiple heat maps to obtain a feature image corresponding to the text image, including: determining the complexity corresponding to the text image according to the multiple heat maps; and performing the first downsampling process on the text image according to the complexity of the text image to obtain a feature image corresponding to the text image.

[0111] According to one or more embodiments of the present disclosure, Example 5 provides the method of Example 4, wherein the first downsampling processing is performed on the text image according to the complexity of the text image to obtain a feature image corresponding to the text image, including: determining a difference between the complexity of the text image and a preset complexity threshold; determining a target downsampling model from a plurality of pre-trained downsampling models according to the difference, different downsampling models corresponding to different downsampling ratios; and performing the first downsampling processing on the text image through the target downsampling model to obtain a feature image corresponding to the text image.

[0112] According to one or more embodiments of the present disclosure, Example 6 provides the method of Example 5, wherein the downsampling ratio includes a height downsampling ratio and a width downsampling ratio; determining the target downsampling model from multiple pre-trained downsampling models based on the difference includes: determining the target height downsampling ratio and the target width downsampling ratio corresponding to the text image based on the difference and a preset sampling rate threshold; determining the target downsampling model from the multiple downsampling models based on the target height downsampling ratio and the target width downsampling ratio.

[0113] According to one or more embodiments of the present disclosure, Example 7 provides a method of any one of Examples 1 to 6, and before obtaining the heat map corresponding to each character in the text image to be recognized, the method also includes: performing a second downsampling process on the text image to be recognized to obtain a downsampled image corresponding to the text image; obtaining the heat map corresponding to each character in the text image to be recognized includes: obtaining the heat map corresponding to each character in the downsampled image.

[0114] According to one or more embodiments of the present disclosure, Example 8 provides a text recognition device, including: a heat map acquisition module, used to obtain a heat map corresponding to each character in a text image to be recognized; a feature image acquisition module, used to perform a first down-sampling process on the text image according to a plurality of the heat maps to obtain a feature image corresponding to the text image; and a text content acquisition module, used to obtain text content corresponding to the text image according to the feature image.

[0115] According to one or more embodiments of the present disclosure, Example 9 provides a computer-readable medium having a computer program stored thereon, which implements the steps of the method described in any one of Examples 1 to 7 when executed by a processing device.

[0116] According to one or more embodiments of the present disclosure, Example 10 provides an electronic device, comprising: a storage device on which a computer program is stored; and a processing device for executing the computer program in the storage device to implement the steps of the method described in any one of Examples 1 to 7.

[0117] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0118] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0119] Although the subject matter has been described in language specific to structural features and / or method logic actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims. Regarding the device in the above embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

Claims

1. A text recognition method, characterized in that: The method comprises: Obtain the heat map corresponding to each character in the text image to be recognized; According to the plurality of heat maps, performing a first downsampling process on the text image to obtain a feature image corresponding to the text image; According to the feature image, obtaining text content corresponding to the text image; The step of performing a first downsampling process on the text image according to the plurality of heat maps to obtain a feature image corresponding to the text image comprises: Determining the complexity corresponding to the text image according to the plurality of heat maps; According to the complexity of the text image, performing the first downsampling process on the text image to obtain a feature image corresponding to the text image; The step of performing the first downsampling process on the text image according to the complexity of the text image to obtain a feature image corresponding to the text image includes: Determining a difference between the complexity of the text image and a preset complexity threshold; According to the difference, a target downsampling model is determined from a plurality of pre-trained downsampling models, where different downsampling models correspond to different downsampling ratios; The first downsampling process is performed on the text image through the target downsampling model to obtain a feature image corresponding to the text image.

2. The method according to claim 1, characterized in that The step of obtaining a heat map corresponding to each character in the text image to be recognized includes: The text image to be recognized is input into a pre-trained heat map acquisition model, and a heat map corresponding to each character in the text image is output.

3. The method according to claim 2, characterized in that The heat map acquisition model is trained in the following way: Acquire a sample set, the sample set comprising a plurality of sample images and a character area corresponding to each character in the sample images; The target neural network model is trained using the sample set to obtain the thermal map acquisition model.

4. The method according to claim 1, characterized in that: The downsampling ratio includes a height downsampling ratio and a width downsampling ratio; and determining a target downsampling model from a plurality of pre-trained downsampling models according to the difference includes: Determine a target height downsampling ratio and a target width downsampling ratio corresponding to the text image according to the difference and a preset sampling rate threshold; The target downsampling model is determined from a plurality of the downsampling models according to the target height downsampling ratio and the target width downsampling ratio.

5. The method according to any one of claims 1 to 4, characterized in that: Before obtaining the heat map corresponding to each character in the text image to be recognized, the method further includes: Performing a second downsampling process on the text image to be recognized to obtain a downsampled image corresponding to the text image; The step of obtaining a heat map corresponding to each character in the text image to be recognized includes: Obtain a heat map corresponding to each character in the downsampled image.

6. A text recognition device, characterized in that: The device comprises: A heat map acquisition module is used to obtain a heat map corresponding to each character in the text image to be recognized; A feature image acquisition module, configured to perform a first downsampling process on the text image according to the plurality of heat maps, to obtain a feature image corresponding to the text image; A text content acquisition module, used to acquire text content corresponding to the text image according to the feature image; The feature image acquisition module is also used for: Determining the complexity corresponding to the text image according to the plurality of heat maps; According to the complexity of the text image, performing the first downsampling process on the text image to obtain a feature image corresponding to the text image; The feature image acquisition module is also used for: Determining a difference between the complexity of the text image and a preset complexity threshold; According to the difference, a target downsampling model is determined from a plurality of pre-trained downsampling models, where different downsampling models correspond to different downsampling ratios; The first downsampling process is performed on the text image through the target downsampling model to obtain a feature image corresponding to the text image.

7. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processing device, the steps of the method described in any one of claims 1 to 5 are implemented.

8. An electronic device, characterized in that: include: a storage device having a computer program stored thereon; A processing device, configured to execute the computer program in the storage device to implement the steps of the method according to any one of claims 1 to 5.

Images