Image processing method and device, readable storage medium and electronic equipment

Abstract

The present disclosure relates to an image processing method, device, readable storage medium and electronic equipment. The method comprises: extracting a text line region from a target image; for each text line region, performing segmentation on the text line region to obtain at least one single character image corresponding to the text line region; and inputting the at least one single character image corresponding to the text line region into a pre-trained character recognition model to obtain recognized text corresponding to the text line region. Since text recognition is performed according to each single character image corresponding to the text line region, the influence of character gaps in the text line region on the text recognition result can be avoided, thereby improving the accuracy of text recognition. In addition, text recognition is simultaneously performed according to each single character image corresponding to the text line region, rather than performing recognition on each single character image respectively, thereby being able to capture the correlation features between adjacent characters in the text line region, thereby further improving the accuracy of text recognition.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, in particular, to an image processing method and device, readable storage medium and electronic equipment. BACKGROUND

[0002] With the continuous development of technology, text recognition technology has been more and more widely used, especially optical character recognition (OCR) and the like. OCR obtains an image by scanning, photographing or sensing an electronic device such as a user writing on a printed body, and obtains the text content in the image by detecting the shape of the text on the image. However, due to the influence of character gaps in the image, the text recognition result is not accurate enough. Therefore, how to improve the accuracy of text recognition is a problem to be solved in the related field. SUMMARY

[0003] This summary is provided to introduce a selection of concepts that are further described below in the detailed description. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it used to limit the scope of the claimed subject matter's scope.

[0004] In a first aspect, the present disclosure provides an image processing method, comprising:

[0005] extracting a text line region from a target image;

[0006] for each of the text line regions, performing segmentation on the text line region to obtain at least one single character image corresponding to the text line region; and inputting the at least one single character image corresponding to the text line region into a pre-trained character recognition model to obtain recognized text corresponding to the text line region.

[0007] In a second aspect, the present disclosure provides an image processing device, comprising:

[0008] an extraction module configured to extract a text line region from a target image;

[0009] a segmentation module configured to, for each of the text line regions extracted by the extraction module, perform segmentation on the text line region to obtain at least one single character image corresponding to the text line region; and a recognition module configured to input the at least one single character image corresponding to the text line region segmented by the segmentation module into a pre-trained character recognition model to obtain recognized text corresponding to the text line region.

[0010] In a third aspect, the present disclosure provides a computer readable medium, having stored thereon a computer program, which, when executed by a processing apparatus, implements the steps of the method provided in the first aspect of the present disclosure.

[0011] In a fourth aspect, the present disclosure provides an electronic device, comprising:

[0012] a storage device having stored thereon a computer program;

[0013] a processing apparatus configured to execute the computer program in the storage device to implement the steps of the method provided in the first aspect of the present disclosure.

[0014] In the above technical solution, first, a text line region is extracted from a target image; then, for each text line region, the text line region is segmented to obtain at least one single character image corresponding to the text line region; and the at least one single character image corresponding to the text line region is input into a pre-trained character recognition model to obtain a recognized text corresponding to the text line region. Since the text recognition is performed according to each single character image corresponding to the text line region, the influence of character gaps in the text line region on the text recognition result can be avoided, thereby improving the accuracy of text recognition. In addition, the text recognition is performed according to each single character image corresponding to the text line region at the same time, rather than performing recognition on each single character image respectively, so that the correlation features between adjacent characters in the text line region can be captured, thereby further improving the accuracy of text recognition.

[0015] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF DRAWINGS

[0016] The above and other features and advantages of the embodiments of the present disclosure will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings in which:

[0017] Figure 1 is a flowchart of an image processing method according to an exemplary embodiment.

[0018] Figure 2 is a structural schematic diagram of a feature extraction model according to an exemplary embodiment.

[0019] Figure 3 is a flowchart of a feature extraction model training method according to an exemplary embodiment.

[0020] Figure 4 is a structural schematic diagram of a character recognition model according to an exemplary embodiment.

[0021] Figure 5 is a block diagram of an image processing apparatus according to an exemplary embodiment.

[0022] Figure 6 is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0023] Embodiments of the present disclosure will be described in more detail with reference to the drawings. While certain embodiments of the present disclosure will be shown in the drawings, it should be understood that the present disclosure can be embodied in various forms and should not be construed as being limited to the embodiments set forth herein, but should be construed as including all modifications, equivalents, and / or alternatives falling within the spirit and scope of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for illustrative purposes and should not be construed as limiting the scope of protection of the present disclosure.

[0024] It should be understood that each step recited in the method embodiments of the present disclosure can be executed in different orders and / or in parallel. In addition, the method embodiments can include additional steps and / or omit the execution of the steps shown. The scope of the present disclosure is not limited in this respect.

[0025] The term "comprising" and variations thereof as used herein are used inclusively, i.e., "comprising, but not limited to." The term "based on" is "based, at least in part, on." The term "one embodiment" means "at least one embodiment." The term "another embodiment" means "at least one additional embodiment." The term "some embodiments" means "at least some embodiments." Related definitions are given below in the description of the application.

[0026] It should be noted that the terms "first", "second", and the like in the present disclosure are used only to distinguish different devices, modules, or units, and do not imply the order or interdependence of the functions performed by these devices, modules, or units.

[0027] It should be noted that the terms "one", "multiple" in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that "one" or "multiple" should be understood as "one or more" unless otherwise explicitly stated in the context.

[0028] The names of the messages or information exchanged between the devices in the embodiments of the present disclosure are only for illustrative purposes, and are not intended to limit the scope of the messages or information.

[0029] Figure 1 is a flowchart of an image processing method according to an exemplary embodiment. As shown in Figure 1 , the method includes S101-S103.

[0030] In S101, a text line region is extracted from the target image.

[0031] In the present disclosure, the target image contains text information, which can be Chinese, English, numbers, or the like. The present disclosure does not specifically limit the language of the text information contained in the target image. Moreover, the text information can be located at any position or multiple positions in the target image.

[0032] In addition, a text line detection algorithm can be used to perform text line detection on the target image to obtain at least one text line region corresponding to the target image. The text line detection algorithm detects text in the target image in units of "lines" and can detect the boundaries of each text line in the target image. The boundaries can be marked using a rectangular frame, and the region defined by the rectangular frame is the detected text line region. The above text line detection algorithm can use a commonly used text line detection algorithm, such as an edge-based detection algorithm, a connected domain-based detection algorithm, a texture-based detection algorithm, a deep learning-based detection algorithm, etc.

[0033] In S102, for each text line region, the text line region is segmented to obtain at least one single character image corresponding to the text line region.

[0034] In the present disclosure, a single character image is an image containing a single character.

[0035] For example, the text line region A includes 8 characters, and by segmenting the text line region A, 8 single character images corresponding to the text line region A can be obtained.

[0036] In S103, the at least one single character image corresponding to the text line region is input into a pre-trained character recognition model to obtain a recognized text corresponding to the text line region.

[0037] In the above technical solution, first, a text line region is extracted from a target image. Then, for each text line region, the text line region is segmented to obtain at least one single character image corresponding to the text line region. The at least one single character image corresponding to the text line region is input into a pre-trained character recognition model to obtain a recognized text corresponding to the text line region. Since the text recognition is performed according to the single character images corresponding to the text line region, the influence of character gaps in the text line region on the text recognition result can be avoided, thereby improving the accuracy of text recognition. In addition, the text recognition is performed simultaneously according to the single character images corresponding to the text line region, rather than performing recognition on each single character image separately, thereby capturing the correlation features between adjacent characters in the text line region, and further improving the accuracy of text recognition.

[0038] The following is a detailed description of the specific implementation method for segmenting the text line region in step S102 above to obtain at least one single character image corresponding to the text line region. Specifically, it can be achieved through the following steps (1) and (2):

[0039] (1) Input the text line region into a pre-trained feature extraction model to obtain a target sequence that characterizes whether each pixel column in the text line region contains characters.

[0040] In this disclosure, the length of the target sequence is equal to the number of pixel columns contained in the text line region, that is, the width of the text line region.

[0041] For example, if the size of the text line area is 32*48, where the width is 32 pixels and the height is 48 pixels, then the width of the text line area is 32 pixels, that is, the text line area includes 32 pixel columns, where each pixel column includes 48 pixels, and the length of the target sequence is 32.

[0042] (2) Based on the target sequence, the text line region is segmented to obtain at least one single character image corresponding to the text line region.

[0043] like Figure 2 As shown, the above feature extraction model may include a feature extraction module, a first encoding module, a first fully connected layer, and a post-processing module connected in sequence. The feature extraction module is used to extract features from the text line region to obtain a first feature sequence corresponding to the text line region; the first encoding module is used to encode the first feature sequence to obtain a first encoded sequence; the first fully connected layer is used to generate a second feature sequence representing the positional information of each character in the text line region based on the first encoded sequence; the post-processing module is used to normalize the second feature sequence and adjust the length of the normalized second feature sequence to a target length to obtain a target sequence representing whether each pixel column in the text line region contains a character, wherein the target length is equal to the number of pixel columns contained in the text line region.

[0044] In this disclosure, the feature extraction module described above can be composed of multiple CNNs (Convolutional Neural Networks).

[0045] The first encoding module can consist of a single encoding network or multiple sequentially connected encoding networks. Preferably, the first encoding module consists of multiple sequentially connected encoding networks, which can improve the richness and accuracy of feature extraction, thereby improving the accuracy of text recognition.

[0046] In one embodiment, the length of the second feature sequence is a preset fixed value M, and the second feature sequence can include the probability of each position containing a character in M positions. At this time, the post-processing module can normalize the second feature sequence in the following manner: first, normalize each value in the second feature sequence to a value in the range of 0-1; then, normalize the value greater than or equal to a preset threshold (for example, 0.5) in the second feature sequence obtained after normalization to 1, and normalize the value less than the preset threshold in the second feature sequence obtained after normalization to 0, thereby obtaining a 0-1 sequence, i.e., the target sequence, wherein 1 in the target sequence indicates that the pixel column corresponding to it in the character line region contains a character, and 0 indicates that the pixel column corresponding to it in the character line region does not contain a character.

[0047] In another embodiment, the length of the second feature sequence is a preset fixed value M, and the second feature sequence can include the probability of each position not containing a character (for example, containing a character gap) in M positions. At this time, the post-processing module can normalize the second feature sequence in the following manner: first, normalize each value in the second feature sequence to a value in the range of 0-1; then, normalize the value greater than or equal to a preset threshold (for example, 0.5) in the second feature sequence obtained after normalization to 1, and normalize the value less than the preset threshold in the second feature sequence obtained after normalization to 0, thereby obtaining a 0-1 sequence, i.e., the target sequence, wherein 0 in the target sequence indicates that the pixel column corresponding to it in the character line region contains a character, and 1 indicates that the pixel column corresponding to it in the character line region does not contain a character.

[0048] In addition, the length of the second feature sequence can be a preset fixed value, which can be greater than, less than, or equal to the number of pixel columns contained in the character line region.

[0049] When the length of the second feature sequence is greater than the number of pixel columns contained in the character line region, i.e., the length of the second feature sequence is greater than the target length, at this time, the second feature sequence obtained after normalization can be sampled to obtain a sequence of the target length, i.e., the target sequence.

[0050] When the length of the second feature sequence is less than the number of pixel columns contained in the character line region, i.e., the length of the second feature sequence is less than the target length, at this time, the second feature sequence obtained after normalization can be interpolated to adjust the length of the second feature sequence obtained after normalization to the target length, thereby obtaining the target sequence.

[0051] When the length of the second feature sequence is equal to the number of pixel columns contained in the text line region, i.e., the length of the second feature sequence is equal to the target length, the length of the second feature sequence obtained after normalization can not be adjusted, and the second feature sequence obtained after normalization can be directly determined as the target sequence.

[0052] The training manner of the feature extraction model is described in detail below. Specifically, the feature extraction model can be trained by S301-S304 shown in FIG. 3. Figure 3

[0053] In S301, a sample text line region and a label sequence used to represent whether each pixel column in the sample text line region contains a character are obtained.

[0054] In S302, the sample text line region is input into the feature extraction model to obtain a training sequence used to represent whether each pixel column in the sample text line region contains a character.

[0055] In S303, a target loss is determined according to the training sequence and the label sequence.

[0056] In an embodiment, the target loss can be determined based on a mean square error loss function according to the training sequence and the label sequence.

[0057] In another embodiment, the target loss can be determined based on a CTC loss function according to the training sequence and the label sequence, where CTC is Connectionist Temporal Classification, i.e., a neural network-based time sequence classification.

[0058] When determining the loss of the feature extraction model, the CTC loss function can be used in the training process of serialized data without alignment. The CTC loss function can automatically align data without alignment, and therefore, when the CTC loss function is used for calculation when determining the loss of the feature extraction model, the accuracy of the determined target loss can be effectively guaranteed, thereby guaranteeing the accuracy of subsequent parameter updating of the feature extraction model.

[0059] In S304, the feature extraction model is parameter updated according to the target loss, and the training is ended when the target loss is less than or equal to a preset threshold to obtain the feature extraction model.

[0060] In the present disclosure, the preset threshold can be set according to the actual use scenario. The higher the accuracy of the feature extraction model is required, the smaller the value of the preset threshold is.

[0061] ​In this step, when it is determined that the target loss of the feature extraction model is greater than the preset threshold, the steps of obtaining the sample text line region and the label sequence used to represent whether each pixel column in the sample text line region contains a character, to updating the parameters of the feature extraction model according to the target loss can be re-executed. During the re-execution of the above steps, different training samples can be selected for training, so that the comprehensiveness and universality of the features that can be learned in the feature extraction model can be improved, and the learning efficiency of the feature extraction model can be improved. When the target loss is less than or equal to the preset threshold, the training is ended, indicating that the feature extraction accuracy of the feature extraction model at this time is high, and meets the use requirements of the user, and at this time, the trained feature extraction model is obtained.

[0062] The following will be described in detail with respect to the specific implementation of the step (2) of splitting the text line region according to the target sequence to obtain at least one single character image corresponding to the text line region.

[0063] Specifically, in the case that the target sequence is a 0-1 sequence, and 1 represents that the pixel column corresponding to the text line region contains a character, and 0 represents that the pixel column corresponding to the text line region does not contain a character, at least one single character image corresponding to the text line region can be obtained by splitting the text line region in the following manner: first, the pixel column corresponding to the continuous 1 in the target sequence in the text line region is determined as a single character region; and then, each single character region is split from the text line region to obtain at least one single character image corresponding to the text line region.

[0064] For example, the target sequence is 000111001111011000011110001011000, where 1 represents that the pixel column corresponding thereto in the text line region contains a character, and 0 represents that the pixel column corresponding thereto in the text line region does not contain a character. The pixel column in the text line region corresponding to the continuous 1s in the fourth to sixth bits of the target sequence can be determined as the single character region A, i.e., the fourth, fifth and sixth pixel columns in the text line region (from left to right) are determined as the single character region A; the pixel column in the text line region corresponding to the continuous 1s in the ninth to twelfth bits of the target sequence is determined as the single character region B, i.e., the ninth, tenth, eleventh and twelfth pixel columns in the text line region are determined as the single character region B; the pixel column in the text line region corresponding to the continuous 1s in the fourteenth and fifteenth bits of the target sequence is determined as the single character region C, i.e., the fourteenth and fifteenth pixel columns in the text line region are determined as the single character region C; the pixel column in the text line region corresponding to the continuous 1s in the twentieth to twenty-third bits of the target sequence is determined as the single character region D, i.e., the twentieth, twenty-first, twenty-second and twenty-third pixel columns in the text line region are determined as the single character region D; the pixel column in the text line region corresponding to the continuous 1s in the twenty-ninth and thirtieth bits of the target sequence is determined as the single character region E, i.e., the twenty-ninth and thirtieth pixel columns in the text line region are determined as the single character region E; then, the single character region A, single character region B, single character region C, single character region D and single character region E are cut out from the text line region respectively to obtain five single character images corresponding to the text line region. The twenty-seventh bit in the target sequence is 1, and the bits before and after the twenty-seventh bit are 0, which are not continuous 1s. Therefore, it can be considered that the pixel column in the text line region corresponding to the twenty-seventh bit in the target sequence does not contain a character.

[0065] In the case where the target sequence is a 0-1 sequence, and 0 represents that the pixel column corresponding thereto in the text line region contains a character, and 1 represents that the pixel column corresponding thereto in the text line region does not contain a character, at least one single character image corresponding to the text line region can be obtained by cutting in the following manner: first, the pixel column in the text line region corresponding to the continuous 0s in the target sequence is determined as a single character region; then, each single character region is cut out from the text line region to obtain at least one single character image corresponding to the text line region.

[0066] As Figure 4As shown, the character recognition model includes a preprocessing module, a second encoding module, a decoding module, and a second full connection layer connected in sequence. The preprocessing module is configured to normalize each single character image in at least one single character image corresponding to the text line region into an image of a preset size (for example, 32*32), and straighten the normalized single character image into a one-dimensional row vector of a preset length (for example, 1024), and then form a multi-dimensional matrix by using the one-dimensional vector corresponding to each single character image corresponding to the text line region. The second encoding module is configured to encode the multi-dimensional matrix to obtain a second encoding sequence. The decoding module is configured to capture the correlation features between adjacent characters in the text line region according to the second encoding sequence to generate a feature vector corresponding to each single character image. The second full connection layer is configured to generate a recognized text corresponding to the text line region according to each feature vector.

[0067] The second encoding module can be composed of one encoding network or multiple encoding networks connected in sequence. Preferably, the second encoding module is composed of multiple encoding networks connected in sequence, so as to improve the richness and accuracy of feature extraction, and further improve the accuracy of text recognition.

[0068] The decoding module can be composed of one decoding network or multiple decoding networks connected in sequence. Preferably, the decoding module is composed of multiple decoding networks connected in sequence, so as to improve the richness and accuracy of feature extraction, and further improve the accuracy of text recognition.

[0069] Figure 5 is a block diagram of an image processing device according to another exemplary embodiment. As shown, Figure 5 The device 500 includes:

[0070] The extraction module 501 is configured to extract a text line region from a target image.

[0071] The segmentation module 502 is configured to segment each text line region extracted by the extraction module 501 to obtain at least one single character image corresponding to the text line region. The recognition module 503 is configured to input the at least one single character image corresponding to the text line region segmented by the segmentation module 502 into a pre-trained character recognition model to obtain a recognized text corresponding to the text line region.

[0072] In the technical solution, first, a text line region is extracted from a target image; then, for each text line region, the text line region is segmented to obtain at least one single character image corresponding to the text line region; and the at least one single character image corresponding to the text line region is input into a pre-trained character recognition model to obtain recognized text corresponding to the text line region. Since text recognition is performed according to each single character image corresponding to the text line region, the influence of character gaps in the text line region on the text recognition result can be avoided, and thus the accuracy of text recognition can be improved. In addition, text recognition is performed according to each single character image corresponding to the text line region at the same time, rather than performing recognition on each single character image respectively, so that the correlation features between adjacent characters in the text line region can be captured, and thus the accuracy of text recognition can be further improved.

[0073] Optionally, the segmentation module 502 comprises:

[0074] an input sub-module, configured to input the text line region into a pre-trained feature extraction model to obtain a target sequence used for representing whether each pixel column in the text line region contains a character, wherein a length of the target sequence is equal to a number of pixel columns contained in the text line region;

[0075] a first segmentation sub-module, configured to segment the text line region according to the target sequence to obtain at least one single character image corresponding to the text line region.

[0076] Optionally, the feature extraction model comprises a feature extraction module, a first encoding module, a first full connection layer and a post-processing module connected in sequence.

[0077] The feature extraction module is configured to perform feature extraction on the text line region to obtain a first feature sequence corresponding to the text line region.

[0078] The first encoding module is configured to encode the first feature sequence to obtain a first encoding sequence.

[0079] The first full connection layer is configured to generate, according to the first encoding sequence, a second feature sequence used for representing position information of each character in the text line region.

[0080] The post-processing module is configured to normalize the second feature sequence and adjust a length of the second feature sequence obtained after normalization to a target length to obtain the target sequence used for representing whether each pixel column in the text line region contains a character, wherein the target length is equal to the number of pixel columns contained in the text line region.

[0081] Optionally, the feature extraction model is obtained by training a training device, wherein the training device comprises:

[0082] an acquisition module configured to acquire a sample text line region and a label sequence used to represent whether each pixel column in the sample text line region contains a character;

[0083] an input module configured to input the sample text line region into the feature extraction model to obtain a training sequence used to represent whether each pixel column in the sample text line region contains a character;

[0084] a determination module configured to determine a target loss according to the training sequence and the label sequence;

[0085] an update module configured to perform parameter update on the feature extraction model according to the target loss, and end the training when the target loss is less than or equal to a preset threshold, so as to obtain the feature extraction model.

[0086] Optionally, the determination module is configured to determine the target loss based on a CTC loss function according to the training sequence and the label sequence.

[0087] Optionally, the target sequence is a 0-1 sequence, wherein 1 represents that a pixel column corresponding to the text line region contains a character, and 0 represents that a pixel column corresponding to the text line region does not contain a character.

[0088] The first segmentation sub-module comprises:

[0089] a determination sub-module configured to determine, as a single character region, a pixel column in the text line region corresponding to a continuous 1 in the target sequence;

[0090] a second segmentation sub-module configured to segment each single character region from the text line region to obtain at least one single character image corresponding to the text line region.

[0091] Optionally, the character recognition model comprises a preprocessing module, a second encoding module, a decoding module and a second full connection layer connected in sequence.

[0092] The preprocessing module is configured to, for each single character image in the at least one single character image corresponding to the text line region, normalize the single character image into an image with a preset size, straighten the single character image obtained after normalization into a one-dimensional row vector with a preset length, and then group the one-dimensional vectors corresponding to each single character image into a multi-dimensional matrix.

[0093] The second encoding module is configured to encode the multi-dimensional matrix to obtain a second encoding sequence.

[0094] The decoding module is configured to generate a feature vector corresponding to each single character image according to the second encoding sequence.

[0095] the second full connection layer is configured to generate, according to each feature vector, recognized text corresponding to the text line region.

[0096] In addition, it should be noted that the training device can be independent of the image processing device 500, or can be integrated into the image processing device 500, and the present disclosure does not make specific limitations thereto.

[0097] The present disclosure also provides a computer readable medium having stored thereon a computer program, which, when executed by a processing device, implements the steps of the image processing method provided by the present disclosure.

[0098] Reference will now be made to the drawings, and specific examples relating to the present disclosure will be illustrated with reference to the drawings. Figure 6 which shows a structural schematic diagram of an electronic device (terminal device or server) 600 suitable for implementing embodiments of the present disclosure. The terminal device in the embodiments of the present disclosure can include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Personal Computers), PMPs (Portable Multimedia Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and the like, as well as fixed terminals such as digital TVs, desktop computers, and the like. Figure 6 The electronic device shown is merely an example, and should not impose any limitation on the functions and use range of the embodiments of the present disclosure.

[0099] As shown in Figure 6 The electronic device 600 can include a processing device (such as a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 602 or programs loaded from a storage device 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0100] Generally, the following devices can be connected to the I / O interface 605: input devices 606 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, and the like; output devices 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, and the like; storage devices 608 including, for example, a magnetic tape, a hard disk, and the like; and communication devices 609. The communication devices 609 can allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 6The electronic device 600 is shown with various elements, but it is understood that not all of these elements are required for implementation or possession. More or less elements can alternatively be implemented or possessed.

[0101] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for executing the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are performed.

[0102] It should be noted that the computer-readable medium described above in the present disclosure can be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. The computer-readable storage medium, for example, can be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination thereof. More specific examples of the computer-readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program used or used in conjunction with an instruction execution system, apparatus, or device. In the present disclosure, the computer-readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take on various forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained in the computer-readable medium can be transmitted using any suitable medium, including but not limited to a wire, an optical fiber, an RF (radio frequency) or the like, or any suitable combination thereof.

[0103] In some embodiments, the client, server, or both can communicate using any known or later developed network protocols, such as the HyperText Transfer Protocol (HTTP), and can be interconnected with any form or medium of digital data communication (for example, a communication network) and any combination of one or more local area networks (LANs), wide area networks (WANs), the Internet, and peer-to-peer networks (for example, ad hoc peer-to-peer networks), as well as any currently known or later developed networks.

[0104] The computer-readable medium described above can be included in the electronic device described above; alternatively, it can exist separately from the electronic device and be not incorporated into the electronic device.

[0105] The computer-readable medium described above carries one or more programs, which, when executed by the electronic device, cause the electronic device to: extract a text line region from a target image; for each text line region, perform segmentation on the text line region to obtain at least one single character image corresponding to the text line region; and input the at least one single character image corresponding to the text line region into a pre-trained character recognition model to obtain recognized text corresponding to the text line region.

[0106] Computer program code for carrying out operations of the present disclosure can be written in any one or combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0107] The computer program product of the first aspect can include one or more non-transitory computer-readable media storing instructions that, when executed, cause one or more processors to perform the operations of the first aspect. The computer program product of the first aspect can include a non-transitory computer-readable medium storing code that, when executed, causes a computer to perform operations for the first aspect.

[0108] The modules involved in the embodiments of the present disclosure can be implemented in the form of software or in the form of hardware. In some cases, the name of a module does not constitute a limitation on the module itself. For example, an extraction module can also be described as a "module for extracting a text line region from a target image".

[0109] The functions described above in the present disclosure can be performed at least in part by one or more hardware logic components. For example, non-limiting, exemplary types of hardware logic components that can be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system on a chip (SOCs), complex programmable logic devices (CPLDs), etc.

[0110] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable storage media can include, without limitation, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media can include one or more lines of electrical connections, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0111] According to one or more embodiments of the present disclosure, example 1 provides an image processing method, comprising: extracting a text line region from a target image; for each text line region, segmenting the text line region to obtain at least one single character image corresponding to the text line region; inputting the at least one single character image corresponding to the text line region into a pre-trained character recognition model to obtain recognized text corresponding to the text line region.

[0112] According to one or more embodiments of the present disclosure, example 2 provides the method of example 1, wherein the segmenting the text line region to obtain at least one single character image corresponding to the text line region comprises: inputting the text line region into a pre-trained feature extraction model to obtain a target sequence for representing whether each pixel column in the text line region contains a character, wherein the length of the target sequence is equal to the number of pixel columns contained in the text line region; and segmenting the text line region according to the target sequence to obtain at least one single character image corresponding to the text line region.

[0113] According to one or more embodiments of the present disclosure, example 3 provides the method of example 2, wherein the feature extraction model comprises a feature extraction module, a first encoding module, a first full connection layer and a post-processing module connected in sequence; wherein the feature extraction module is configured to perform feature extraction on the text line region to obtain a first feature sequence corresponding to the text line region; the first encoding module is configured to encode the first feature sequence to obtain a first encoding sequence; the first full connection layer is configured to generate a second feature sequence for representing position information of each character in the text line region according to the first encoding sequence; and the post-processing module is configured to normalize the second feature sequence and adjust the length of the normalized second feature sequence to a target length to obtain a target sequence for representing whether each pixel column in the text line region contains a character, wherein the target length is equal to the number of pixel columns contained in the text line region.

[0114] According to one or more embodiments of the present disclosure, example 4 provides the method of example 2, wherein the feature extraction model is trained in the following manner: obtaining a sample text line region and a labeled sequence for representing whether each pixel column in the sample text line region contains a character; inputting the sample text line region into the feature extraction model to obtain a training sequence for representing whether each pixel column in the sample text line region contains a character; determining a target loss according to the training sequence and the labeled sequence; performing parameter update on the feature extraction model according to the target loss, and ending the training when the target loss is less than or equal to a preset threshold to obtain the feature extraction model.

[0115] According to one or more embodiments of the present disclosure, example 5 provides the method of example 4, wherein determining the target loss according to the training sequence and the labeled sequence comprises: determining the target loss according to the training sequence and the labeled sequence based on a CTC loss function.

[0116] According to one or more embodiments of the present disclosure, example 6 provides the method of example 2, wherein the target sequence is a 0-1 sequence, wherein 1 represents that the pixel column corresponding to it in the text line region contains a character, and 0 represents that the pixel column corresponding to it in the text line region does not contain a character; and wherein the splitting the text line region according to the target sequence to obtain at least one single character image corresponding to the text line region comprises: determining the pixel column corresponding to the continuous 1 in the text line region as a single character region in the text line region; and splitting each single character region from the text line region to obtain at least one single character image corresponding to the text line region.

[0117] According to one or more embodiments of the present disclosure, example 7 provides the method of any one of examples 1-6, wherein the character recognition model comprises a preprocessing module, a second encoding module, a decoding module and a second full connection layer connected in sequence; wherein the preprocessing module is configured to normalize each single character image in the at least one single character image corresponding to the text line region into an image of a preset size, and straighten the single character image obtained after normalization into a one-dimensional row vector of a preset length, and then group the one-dimensional vectors corresponding to each single character image into a multi-dimensional matrix; the second encoding module is configured to encode the multi-dimensional matrix to obtain a second encoding sequence; the decoding module is configured to generate a feature vector corresponding to each single character image according to the second encoding sequence; and the second full connection layer is configured to generate a recognized text corresponding to the text line region according to each feature vector.

[0118] According to one or more embodiments of the present disclosure, example 8 provides an image processing apparatus, comprising: an extraction module configured to extract a text line region from a target image; a splitting module configured to split each text line region extracted by the extraction module to obtain at least one single character image corresponding to the text line region; and an identification module configured to input the at least one single character image corresponding to the text line region split by the splitting module into a pre-trained character recognition model to obtain a recognized text corresponding to the text line region.

[0119] According to one or more embodiments of the present disclosure, example 9 provides a computer readable medium having stored thereon a computer program, which, when executed by a processing apparatus, implements the steps of the method of any one of examples 1-7.

[0120] According to one or more embodiments of the present disclosure, example 10 provides an electronic device, comprising: a storage device having stored thereon a computer program; and a processing device configured to execute the computer program in the storage device to implement the steps of the method according to any one of examples 1-7.

[0121] The above description is only preferred embodiments of the present disclosure and the explanation of the applied technical principles. It should be understood by those skilled in the art that the disclosed scope of the present disclosure is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also cover other technical solutions formed by the combinations of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the technical solutions formed by the mutual replacement of the above features and the technical features disclosed in the present disclosure (but not limited to) with similar functions.

[0122] In addition, although each operation is described in a particular order, this should not be understood as requiring the operations to be performed in the specific order shown or in a sequential order. In certain circumstances, multitasking and parallel processing can be advantageous. Similarly, although several implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Certain features described in the context of separate embodiments can also be combined in a single embodiment. Conversely, various features described in the context of a single embodiment can also be separated and implemented in multiple embodiments. The various features described in the context of the above embodiments can be combined in any suitable sub-combination.

[0123] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims. With respect to the devices in the above-described embodiments, the specific manner in which the various modules perform operations has been described in detail in the embodiments related to the method, and will not be described in detail here.

Claims

1. An image processing method, characterized by, The method comprises: extracting a text line region from a target image; for each text line region, inputting the text line region into a pre-trained feature extraction model to obtain a target sequence for representing whether each pixel column in the text line region contains a character, wherein the length of the target sequence is equal to the number of pixel columns contained in the text line region, and the target sequence is a 0-1 sequence, wherein 1 represents that the pixel column corresponding to the text line region contains a character, and 0 represents that the pixel column corresponding to the text line region does not contain a character; determining, as a single character region, a pixel column in the text line region corresponding to a continuous 1 in the target sequence; cutting out each single character region from the text line region to obtain at least one single character image corresponding to the text line region; inputting the at least one single character image corresponding to the text line region into a pre-trained character recognition model to obtain recognized text corresponding to the text line region.

2. The method of claim 1, wherein, The feature extraction model comprises a feature extraction module, a first encoding module, a first full connection layer, and a post-processing module connected in sequence; wherein the feature extraction module is configured to perform feature extraction on the text line region to obtain a first feature sequence corresponding to the text line region; the first encoding module is configured to encode the first feature sequence to obtain a first encoding sequence; the first full connection layer is configured to generate, according to the first encoding sequence, a second feature sequence for representing position information of each character in the text line region; the post-processing module is configured to normalize the second feature sequence and adjust the length of the normalized second feature sequence to a target length to obtain a target sequence for representing whether each pixel column in the text line region contains a character, wherein the target length is equal to the number of pixel columns contained in the text line region.

3. The method of claim 1, wherein, The feature extraction model is trained in the following manner: obtain a sample text line region and a label sequence for representing whether each pixel column in the sample text line region contains a character; input the sample text line region into the feature extraction model to obtain a training sequence for representing whether each pixel column in the sample text line region contains a character; determine a target loss according to the training sequence and the label sequence; update parameters of the feature extraction model according to the target loss, and end the training when the target loss is less than or equal to a preset threshold to obtain the feature extraction model.

4. The method of claim 3, wherein, The determination of the target loss according to the training sequence and the label sequence comprises: determining the target loss based on a CTC loss function according to the training sequence and the label sequence.

5. The method according to any one of claims 1-4, characterized in that, The character recognition model comprises a preprocessing module, a second encoding module, a decoding module, and a second full connection layer connected in sequence; The preprocessing module is configured to normalize each single character image in the at least one single character image corresponding to the text line region into an image of a preset size, straighten the normalized single character image into a one-dimensional row vector of a preset length, and then form a multi-dimensional matrix by using the one-dimensional row vector corresponding to each single character image. The second encoding module is configured to encode the multi-dimensional matrix to obtain a second encoding sequence. The decoding module is configured to generate a feature vector corresponding to each single character image according to the second encoding sequence. The second full connection layer is configured to generate recognized text corresponding to the text line region according to each feature vector.

6. An image processing apparatus characterized by comprising: The method comprises the following steps: extracting a text line region from a target image; segmenting each text line region extracted by the extracting module to obtain at least one single character image corresponding to the text line region; inputting the at least one single character image corresponding to the text line region segmented by the segmenting module into a pre-trained character recognition model to obtain recognized text corresponding to the text line region.

7. A computer readable medium having stored thereon a computer program, characterized in that The program is executed by a processing device to implement the steps of the method of any one of claims 1-5.

8. An electronic device, comprising: The method comprises the following steps: a storage device having a computer program stored thereon; a processing device configured to execute the computer program in the storage device to implement the steps of the method of any one of claims 1-5.

Images