Text recognition method, device, electronic device and storage medium

Abstract

The present disclosure provides a text recognition method, apparatus, electronic device, storage medium, and program product, relating to the field of artificial intelligence technologies, particularly to the fields of deep learning, image processing, and computer vision technologies, and can be applied to scenarios such as OCR. The specific implementation solution is as follows: when it is determined that the text in the image to be recognized is vertical text, determine the background complexity of the image to be recognized; determine a target recognition mode that matches the background complexity; determine a target character region in the image to be recognized according to the target recognition mode; and perform text recognition on the image to be recognized based on the target character region to obtain a recognition result.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technologies, and particularly to the fields of deep learning, image processing, and computer vision technologies, and can be applied to scenarios such as OCR. In particular, it relates to a text recognition method, apparatus, electronic device, storage medium, and program product. Background Art

[0002] OCR (Optical Character Recognition) may refer to a technology that uses an electronic device, such as a scanner or a digital camera, to collect a to-be-processed image with text content. And uses a text recognition method to recognize the text content in the to-be-processed image and convert it into a language that can be processed by a computer. OCR is an important aspect in the research and application field of automatic recognition technologies. Summary of the Invention

[0003] The present disclosure provides a text recognition method, apparatus, electronic device, storage medium, and program product.

[0004] According to one aspect of the present disclosure, there is provided a text recognition method, including: when it is determined that the text in the to-be-recognized image is vertical text, determining the background complexity of the to-be-recognized image; determining a target recognition mode that matches the background complexity; according to the target recognition mode, determining a target character region in the to-be-recognized image; and based on the target character region, performing text recognition on the to-be-recognized image to obtain a recognition result.

[0005] According to another aspect of the present disclosure, there is provided a text recognition apparatus, including: a complexity determination module, configured to determine the background complexity of the to-be-recognized image when it is determined that the text in the to-be-recognized image is vertical text; a mode determination module, configured to determine a target recognition mode that matches the background complexity; a processing module, configured to determine a target character region in the to-be-recognized image according to the target recognition mode; and a recognition module, configured to perform text recognition on the to-be-recognized image based on the target character region to obtain a recognition result.

[0006] According to another aspect of the present disclosure, there is provided an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method as in the present disclosure.

[0007] According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the method as in the present disclosure.

[0008] According to another aspect of the present disclosure, there is provided a computer program product including a computer program, and the above computer program, when executed by a processor, implements the method as disclosed in the present disclosure.

[0009] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. Description of the Drawings

[0010] The drawings are used to better understand the solution and do not constitute a limitation to the present disclosure. Among them:

[0011] Figure 1 Schematically shows an exemplary system architecture to which the text recognition method and apparatus according to the embodiments of the present disclosure can be applied;

[0012] Figure 2 Schematically shows a flowchart of the text recognition method according to the embodiments of the present disclosure;

[0013] Figure 3 Schematically shows a schematic diagram of determining a target character region using a decoding recognition mode according to the embodiments of the present disclosure;

[0014] Figure 4 Schematically shows a schematic diagram of a character index sequence according to the embodiments of the present disclosure;

[0015] Figure 5 Schematically shows a schematic diagram of determining a target character region using a binarization recognition mode according to the embodiments of the present disclosure;

[0016] Figure 6 Schematically shows a schematic diagram of a projection view according to another embodiment of the present disclosure;

[0017] Figure 7 Schematically shows a flowchart of the text recognition method according to another embodiment of the present disclosure;

[0018] Figure 8 Schematically shows a schematic diagram of a text detection frame according to the embodiments of the present disclosure;

[0019] Figure 9 Schematically shows a block diagram of the text recognition apparatus according to the embodiments of the present disclosure; and

[0020] Figure 10 Schematically shows a block diagram of an electronic device suitable for implementing the text recognition method according to the embodiments of the present disclosure. Detailed Embodiments

[0021] The following describes exemplary embodiments of the present disclosure with reference to the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, descriptions of well-known functions and structures are omitted below for clarity and conciseness.

[0022] The present disclosure provides a text recognition method, apparatus, electronic device, storage medium, and program product.

[0023] According to one aspect of the present disclosure, there is provided a text recognition method, including: when it is determined that the text in the image to be recognized is vertical text, determining the background complexity of the image to be recognized; determining a target recognition mode that matches the background complexity; determining a target character region in the image to be recognized according to the target recognition mode; and performing text recognition on the image to be recognized based on the target character region to obtain a recognition result.

[0024] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision, disclosure, and application, etc., of the user's personal information involved all comply with the provisions of relevant laws and regulations, necessary confidentiality measures are taken, and public order and good customs are not violated.

[0025] In the technical solution of the present disclosure, before obtaining or collecting the user's personal information, the authorization or consent of the user is obtained.

[0026] Figure 1 An exemplary system architecture to which the text recognition method and apparatus according to the embodiments of the present disclosure can be applied is schematically shown.

[0027] It should be noted that Figure 1 The example shown is only an example of the system architecture to which the embodiments of the present disclosure can be applied, to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments, or scenarios. For example, in another embodiment, the exemplary system architecture to which the text recognition method and apparatus can be applied may include a terminal device, but the terminal device can implement the text recognition method and apparatus provided by the embodiments of the present disclosure without interacting with the server.

[0028] As Figure 1 shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0029] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as knowledge reading applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software, etc. (for example only).

[0030] Terminal devices 101, 102, and 103 can be various electronic devices with a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, and desktop computers, etc.

[0031] Server 105 can be a server providing various services, such as a background management server that supports the content browsed by users using terminal devices 101, 102, and 103 (for example only). The background management server can analyze and process data such as received user requests, and feedback the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0032] It should be noted that the text recognition method provided by the embodiments of the present disclosure can generally be executed by terminal devices 101, 102, or 103. Correspondingly, the text recognition device provided by the embodiments of the present disclosure can also be set in terminal devices 101, 102, or 103.

[0033] Alternatively, the text recognition method provided by the embodiments of the present disclosure can generally also be executed by server 105. Correspondingly, the text recognition device provided by the embodiments of the present disclosure can generally be set in server 105. The text recognition method provided by the embodiments of the present disclosure can also be executed by a server or a server cluster different from server 105 and capable of communicating with terminal devices 101, 102, 103 and / or server 105. Correspondingly, the text recognition device provided by the embodiments of the present disclosure can also be set in a server or a server cluster different from server 105 and capable of communicating with terminal devices 101, 102, 103 and / or server 105.

[0034] For example, when a user uses terminal devices 101, 102, and 103 to capture videos, the server 105 can receive the videos from the terminal devices 101, 102, and 103. The server 105 can split the videos into frames to obtain multiple video frames, and use the video frames as images to be recognized. When it is determined that the text in the image to be recognized is vertical text, the background complexity of the image to be recognized is determined. A target recognition mode that matches the background complexity is determined. According to the target recognition mode, the target character region in the image to be recognized is determined. Based on the target character region, text recognition is performed on the image to be recognized to obtain a recognition result. Alternatively, a server or a server cluster capable of communicating with the terminal devices 101, 102, and 103 and / or the server 105 analyzes the image to be recognized and finally obtains a recognition result.

[0035] It should be understood that Figure 1 the numbers of the terminal devices, networks, and servers in

[0036] are merely illustrative. According to actual needs, there can be any number of terminal devices, networks, and servers.

[0037] Figure 2 A flowchart of a text recognition method according to an embodiment of the present disclosure is schematically shown.

[0038] As Figure 2 shown, the method 200 includes operations S210 to S240.

[0039] In operation S210, when it is determined that the text in the image to be recognized is vertical text, the background complexity of the image to be recognized is determined.

[0040] In operation S220, a target recognition mode that matches the background complexity is determined.

[0041] In operation S230, according to the target recognition mode, the target character region in the image to be recognized is determined.

[0042] In operation S240, based on the target character region, text recognition is performed on the image to be recognized to obtain a recognition result.

[0043] According to an embodiment of the present disclosure, the image to be recognized can be an image or a video frame, and the acquisition method thereof is not limited as long as it is an image including text content.

[0044] According to an embodiment of the present disclosure, vertical text may refer to text arranged in a vertical manner. For example, the text content is the three characters "Story Meeting", arranged from top to bottom to form vertical text.

[0045] The general text recognition model in the related art is trained using images including horizontal text as training samples. Directly applying the general text recognition model to recognize a to-be-recognized image including vertical text will result in problems of low recognition processing efficiency and low recognition accuracy. In the case where it is determined that the text in the to-be-recognized image is vertical text, the text recognition method provided by the embodiment of the present disclosure can be used to recognize the to-be-recognized image including vertical text to improve the recognition accuracy.

[0046] According to an embodiment of the present disclosure, in the case where it is determined that the text in the to-be-recognized image is vertical text, determine the background complexity of the to-be-recognized image. The background complexity may refer to the recognition difficulty. The background complexity can be determined based on the image clarity of the to-be-recognized image, but it is not limited thereto. The background complexity can also be determined based on the noise content of the to-be-recognized image. As long as the background complexity can reflect the text recognition difficulty.

[0047] According to an embodiment of the present disclosure, determining a target recognition mode matching the background complexity may include: performing a level division according to the value of the background complexity from high to low, and presetting multiple background complexity levels. Generating a mapping relationship between the background complexity levels and the recognition modes, and each background complexity level is matched with a recognition mode. Determining the target background complexity level based on the background complexity. Based on the mapping relationship, determining the target recognition mode matching the target background complexity level.

[0048] According to an embodiment of the present disclosure, determining the target character region in the to-be-recognized image according to the target recognition mode may include: detecting the text region in the to-be-recognized image according to the target recognition mode to obtain the target character region in the to-be-recognized image. The target character region may be a character region represented by a detection frame for its boundary.

[0049] According to an embodiment of the present disclosure, performing text recognition on the to-be-recognized image based on the target character region to obtain a recognition result includes: processing the to-be-recognized image based on the target character region to obtain a processed image; performing text recognition on the processed image to obtain a recognition result. The processing may include at least one of the following: cropping, rotating, splicing, and removing noise. Performing text recognition on the processed image may include: inputting the processed image into a general recognition model to obtain a recognition result.

[0050] According to an embodiment of the present disclosure, the general recognition model may include a CNN (convolutional neural network) + RNN (recurrent neural network) + CTC (connectionist temporal classification) model or a CNN + RNN + attention seq2seq model. However, it is not limited to this. As long as it is a model disclosed in the prior art for text detection and recognition, it can be used.

[0051] According to the embodiments of the present disclosure, the target recognition mode is utilized to determine the target character area in the image to be recognized, so that text recognition can be performed on the image to be recognized based on the target character area to obtain a recognition result. The image to be recognized can be processed in a targeted manner to make text recognition accurate and effective.

[0052] According to relevant examples, a recognition method of directly rotating the image to be recognized can be adopted. For example, when it is determined that the text in the image to be recognized is vertical text, the image to be recognized can be rotated so that the vertical text in the image to be recognized is converted into horizontal text to obtain a rotated image. The rotated image is input into the general text recognition model to obtain a recognition result.

[0053] Compared with the recognition method of directly rotating the image to be recognized, the text recognition method provided by the embodiment of the present disclosure has a wide range of applications, can effectively recognize special characters, and avoid the problem of recognition errors caused by the use of rotation to make special characters such as "三" become "川" after rotation.

[0054] According to another related example, a vertical text recognition model recognition method can be adopted. For example, an image including vertical text can be collected as a training sample. A vertical text recognition model is trained using the image including vertical text. In this way, the vertical text recognition model is used to process the image to be recognized including vertical text to obtain a recognition result.

[0055] Compared with the method of vertical recognition model recognition, the text recognition method provided by the embodiment of the present disclosure can quickly and effectively recognize the image to be recognized including vertical text, avoiding the problem of spending a lot of time and cost on operations such as collecting training samples and using training samples to train models.

[0056] According to an exemplary embodiment of the present disclosure, the recognition mode for processing the image to be recognized may include a decoding recognition mode and a binarization recognition mode. The target recognition mode may be determined from the decoding recognition mode and the binarization recognition mode based on background complexity.

[0057] According to an embodiment of the present disclosure, determining a target recognition mode that matches the background complexity may include: when it is determined that the background complexity is greater than or equal to a predetermined background complexity threshold, determining the target recognition mode as a decoding recognition mode. When it is determined that the background complexity is less than the predetermined background complexity threshold, determining the target recognition mode as a binarization recognition mode.

[0058] According to an embodiment of the present disclosure, the predetermined background complexity threshold may be determined according to the actual situation, as long as it can adapt to the target recognition mode of the image to be recognized based on the background complexity of the image to be recognized and the predetermined background complexity threshold.

[0059] According to an embodiment of the present disclosure, the predetermined background complexity is used to distinguish the recognition difficulty of the image to be recognized, so as to use the adapted target recognition mode for recognition, making the recognition targeted while improving the recognition speed and accuracy.

[0060] According to an embodiment of the present disclosure, determining the background complexity of the image to be recognized may include the following operations.

[0061] For example, performing color clustering processing on the image to be recognized to obtain the color category result of the image to be recognized. Based on the color category result, determine the background complexity.

[0062] According to an embodiment of the present disclosure, performing color clustering processing on the image to be recognized to obtain the color category result of the image to be recognized may include: inputting the image to be recognized into a color clustering model to obtain the color category result of the image to be recognized. The color clustering model may include: a K-means model, a GMM model, a Mean shift model, or a Quick shift model. However, it is not limited thereto. As long as it is an open-source model structure for color category recognition.

[0063] According to an embodiment of the present disclosure, based on the color category result, determining the background complexity may include: pre-generating a mapping relationship between the color category result and the background complexity. Based on the color category result of the image to be recognized and the mapping relationship, determine the background complexity of the image to be recognized. For example, the larger the number of color types represented by the color category result, the greater the background complexity, and the higher the corresponding background complexity level. Furthermore, the text recognition difficulty of the image to be recognized is determined through the background complexity. Using the color category result to determine the background complexity is simple and effective.

[0064] According to an embodiment of the present disclosure, when the target recognition mode is a decoding recognition mode, according to the target recognition mode, determining the target character region in the image to be recognized may include the following operations.

[0065] For example, feature extraction is performed on the image to be recognized to obtain a sequence of feature maps. The sequence of feature maps includes a plurality of feature maps, and each feature map corresponds to a sub-image in the image to be recognized. The sub-image is an image pre-divided from the image to be recognized. Based on the sequence of feature maps, a probability matrix regarding character categories is obtained. Based on the probability matrix, a character index sequence is obtained. The character index sequence includes a plurality of character index labels, and the character index label is a label used to characterize the character category. Based on the character index sequence, the target character region in the image to be recognized is determined.

[0066] Figure 3 Schematically shows a schematic diagram of determining a target character region using a decoding recognition mode according to an embodiment of the present disclosure.

[0067] As Figure 3 shown, feature extraction is performed on the image 310 to be recognized to obtain a sequence of feature maps 320. The sequence of feature maps includes a plurality of feature maps 321, and each feature map corresponds to a sub-image 311 in the image to be recognized. The sub-image is an image pre-divided from the image 310 to be recognized. Based on the sequence of feature maps 320, a probability matrix 340 regarding character categories 330 is obtained. There are a plurality of character categories, such as {a, b, c,..., x, y, z, -}, etc. It should be noted that the character category "-" is used to represent a blank, such as no character. Based on the probability matrix 340, a character index sequence 350 is obtained. The character index sequence 350 includes a plurality of character index labels 351. Each character index label corresponds to a sub-image in the image to be recognized. The target character region 312 in the image 310 to be recognized can be determined based on the character index sequence 350.

[0068] According to an embodiment of the present disclosure, performing feature extraction on the image to be recognized to obtain a sequence of feature maps may include: inputting the image to be recognized into a feature extraction module to obtain a sequence of feature maps. The feature extraction module may include at least one of the following: a convolutional neural network, a recurrent neural network.

[0069] According to an embodiment of the present disclosure, the probability matrix may be a posterior probability matrix processed by an activation function Softmax. The probability matrix may include a plurality of column vectors, and each column vector corresponds to a sub-image. Each element in the column vector is used to characterize the prediction probability of the corresponding character category in the sub-image. By performing an ArgMax function operation on the column vector, the character category of the character in the sub-image corresponding to the column vector can be obtained.

[0070] According to embodiments of the present disclosure, the CTC principle can be used to process the probability matrix to obtain the intermediate result of the CTC algorithm, such as a character index sequence. For example, the sequence merging mechanism and the global optimal method in the CTC algorithm are used to process the probability matrix to obtain the character index sequence. The sequence merging mechanism is also called the Blank mechanism, which mainly includes: when a "-" is inserted between repeated characters, consecutive identical characters are not merged, otherwise, consecutive repeated characters in the character sequence are deleted. The global optimal method is also called finding the path with the largest probability value. The Greedy Search or BeamSearch method can be used.

[0071] According to embodiments of the present disclosure, using the decoding recognition mode to determine the target character region in the image to be recognized can perform text recognition on the image to be recognized with a high background complexity and great recognition difficulty, while reducing the time consumption and improving the application scope of the text recognition method provided by the embodiments of the present disclosure.

[0072] According to embodiments of the present disclosure, based on the character index sequence, determining the target character region in the image to be recognized may include the following operations.

[0073] For example, determine the target character index label from the character index sequence. Determine the arrangement position information of the target character index label in the character index sequence. Based on the arrangement position information, determine the central position information of the target character region. Based on the number of target character index labels, the size of the image to be recognized, and the central position information, determine the target character region in the image to be recognized.

[0074] According to embodiments of the present disclosure, the character index sequence includes multiple character index labels. The character index label is a label used to represent the corresponding character category in the sub-image. The character category may include non-blank character categories, such as the A character category, the B character category, etc., and may also include a blank character category, such as no character, which can be identified by the character index label 0. The character index label used to represent the non-blank character category can be used as the target character index label from the character index sequence. The character index label used to represent the non-blank character category can be identified by other numbers.

[0075] Figure 4 Schematically shows a schematic diagram of the character index sequence according to embodiments of the present disclosure.

[0076] As Figure 4As shown, the vertical text in the image 410 to be recognized is "Episode 07". Using the decoding and recognition mode in the text recognition method provided by the embodiments of the present disclosure to process the image 410 to be recognized, a character index sequence 420 is obtained. The character index sequence 420 includes 28 character index tags. It is possible to determine that the image to be recognized is divided into 28 sub-images based on the number of character index tags. The character index sequence 420 includes 3 character index tags for characterizing non-blank character categories, such as "479", "393", and "1098". Take "479", "393", and "1098" as the target character index tags.

[0077] According to the embodiments of the present disclosure, based on the arrangement position information of the target index tag in the character index sequence, for example, "479" is ranked at the 3rd position in the character index sequence. Based on this arrangement position information, determine the position information of the center of the target character region in the image to be recognized, that is, the center position information. Based on the number of target character index tags, the number of target character regions can be determined, such as the number of multiple characters. Based on the center position information, the number of target character index tags, and the size of the image to be recognized, determine the target character region in the image to be recognized.

[0078] It should be noted that the size of the image to be recognized may include the length and width of the image to be recognized. However, it is not limited thereto. The size of the image to be recognized may also include the size of the text detection frame in the image to be recognized, such as the length and width of the text detection frame.

[0079] As Figure 4 shown, based on the center position information, the number of target character index tags, and the size of the image to be recognized, it can be determined that the target character regions in the image to be recognized include the "Di" target character region, the "07" target character region, and the "Ji" target character region.

[0080] According to the embodiments of the present disclosure, in the case where the target recognition mode is determined to be the binarization recognition mode, according to the target recognition mode, determine the target character region in the image to be recognized, including the following operations.

[0081] For example, perform binarization processing on the image to be recognized to obtain a binarized image. Perform horizontal projection on the binarized image to obtain a projection map. Based on the projection map, determine the target character region in the image to be recognized.

[0082] Figure 5 Schematically shows a schematic diagram of determining the target character region using the binarization recognition mode according to the embodiments of the present disclosure.

[0083] As Figure 5As shown, the image to be recognized 510 includes the vertical text "Episode 07". An adaptive binarization algorithm can be used to binarize the image to be recognized 510 to obtain a binarized image 520 with white characters on a black background. The binarized image 520 is horizontally projected to obtain a projection image 530. Based on the projection image 530, a target character area 540 in the image to be recognized 510 is determined.

[0084] According to the relevant example, the binary image can be vertically projected to obtain a projection image, and based on the projection image, the target character area in the image to be recognized is determined.

[0085] Compared with the vertical projection method, horizontal projection of the binary image can meet the scenario of vertical text and avoid projecting multiple lines of text into one character.

[0086] According to the embodiments of the present disclosure, a binary recognition mode is used to recognize an image to be recognized. For an image to be recognized whose background complexity is lower than a predetermined background complexity threshold, the recognition process can be simplified and the recognition efficiency can be improved while ensuring the recognition accuracy.

[0087] According to an embodiment of the present disclosure, based on the projection graph, determining the target character region in the image to be recognized may include: taking each connected domain (also called projection domain) in the projection graph as a target character region. Using an embodiment of the present disclosure, the horizontally arranged "07" in the vertical text "Episode 07" can be recognized as a character, which is helpful for subsequent text recognition.

[0088] It should be noted that the projection image is obtained by horizontally projecting the binary image, so for characters with upper and lower structures and a gap in the middle, such as "吉", after horizontal projection, it will be displayed as two connected domains with a gap in the projection image. In this case, the following operations can be performed to avoid mistakenly recognizing one character as two target character regions.

[0089] For example, based on the projection graph, determining the target character region in the image to be recognized includes: determining multiple connected domains based on the projection graph, determining the spacing distance between two adjacent connected domains to obtain multiple spacing distances, and determining the target character region from the multiple connected domains based on the multiple spacing distances.

[0090] According to an embodiment of the present disclosure, determining a target character region from a plurality of connected domains based on a plurality of spacing distances may include: for each spacing distance in the plurality of spacing distances, determining whether two target connected domains meet a predetermined condition based on the spacing distance and a predetermined connected domain height. If the two target connected domains meet the predetermined condition, the two target connected domains are determined as one target character region. If the two target connected domains do not meet the predetermined condition, the two target connected domains are determined as two target character regions.

[0091] Figure 6 A schematic diagram schematically shows a projection diagram according to another embodiment of the present disclosure.

[0092] As Figure 6 shown, the projection diagram 600 includes three connected regions. For the spacing 610, there are two target connected regions that match the spacing 610, such as the first target connected region 620 and the second target connected region 630. The first target connected region 620 and the second target connected region 630 can be determined whether they meet a predetermined condition based on the spacing distance H of the spacing 610.

[0093] According to an embodiment of the present disclosure, that two target connected regions meet a predetermined condition may include: the spacing distance between the two target connected regions is less than or equal to a predetermined connected region height. That the two target connected regions do not meet the predetermined condition may include: the spacing distance between the two target connected regions is greater than the predetermined connected region height.

[0094] According to an embodiment of the present disclosure, the predetermined connected region height can be set according to the actual situation. For example, the predetermined connected region height can be half of the average height of the two target connected regions, but it is not limited thereto, and it can also be half of the average height of multiple connected regions. As long as it is a predetermined connected region height that can be used to distinguish the association relationship between the two target connected regions.

[0095] According to an embodiment of the present disclosure, in the case where it is determined that the spacing distance between the two target connected regions is greater than the predetermined connected region height, it indicates that the two target connected regions are two characters, and they can be split into two target character regions. In the case where it is determined that the spacing distance between the two target connected regions is less than or equal to the predetermined connected region height, it indicates that the two target connected regions are one character in an up-and-down structure, and the two target connected regions can be merged as one target character region.

[0096] According to an embodiment of the present disclosure, in the case of performing binary pattern recognition on the image to be processed, the spacing distance of the connected regions can be used to determine whether to merge adjacent connected regions, thereby improving the recognition accuracy of the target character regions of special characters such as up-and-down structure characters, and further improving the recognition accuracy of vertical text.

[0097] Figure 7 A flowchart of a text recognition method according to another embodiment of the present disclosure is schematically shown.

[0098] As Figure 7 shown, the method may include operations S710 to S780.

[0099] In operation S710, text detection is performed on the image to be recognized, and the spacing between each character in the image to be recognized and the characters adjacent to the character is determined.

[0100] According to an embodiment of the present disclosure, text detection can be performed on the image to be recognized, and the vertical spacing, for example, the first spacing, between the character A and the adjacent character B, and the horizontal spacing, for example, the second spacing, between the character A and the adjacent character C can be determined.

[0101] According to an embodiment of the present disclosure, a line text detection module can be used to perform text detection on the image to be recognized to determine the spacing between each character and the adjacent character. The text detection module can include at least one of the following: Pixel Link, SegLink (Segment Link), EAST (Efficient and Accuracy Scene Textdetection pipeline). However, it is not limited thereto, as long as it is a model capable of detecting text from an image. In operation S720, based on the first spacing and the second spacing, the arrangement type of the text in the image to be recognized is determined.

[0102] According to an embodiment of the present disclosure, the arrangement type can include a vertical arrangement type and a horizontal arrangement type. The text of the vertical arrangement type is vertical text. The text of the horizontal arrangement type is horizontal text. For multiple characters, it is determined that the first spacing is less than the second spacing. In this case, it is determined that the arrangement type of the text is the vertical arrangement type. For multiple characters, it is determined that the first spacing is greater than or equal to the second spacing. In this case, it is determined that the arrangement type is the horizontal arrangement type.

[0103] In operation S730, when it is determined that the text in the image to be recognized is vertical text, the background complexity of the image to be recognized is determined.

[0104] In operation S740, a target recognition mode matching the background complexity is determined.

[0105] In operation S750, according to the target recognition mode, the target character region in the image to be recognized is determined.

[0106] In operation S760, based on the target character region, the image to be recognized is segmented to obtain character images.

[0107] According to an embodiment of the present disclosure, there are multiple target character regions. Segmenting the image to be recognized based on the target character region to obtain character images can include: segmenting the image to be recognized according to the number of target character regions to obtain multiple character images corresponding one-to-one to the multiple target character regions.

[0108] According to an embodiment of the present disclosure, segmenting the image to be recognized can include: based on the number of target character regions and the size of the text detection frame, the text detection frame is segmented to obtain multiple character images corresponding one-to-one to the multiple target character regions.

[0109] In operation S770, a horizontal text image is generated based on the character image.

[0110] According to an embodiment of the present disclosure, a plurality of character images may be horizontally spliced according to the arrangement order of target characters to generate a horizontal text image.

[0111] According to an exemplary embodiment of the present disclosure, the character image can be segmented according to the frame shape to obtain a frame-shaped character image. The length and width of each character image are determined, and the average length and average width of multiple character images are determined. Character images that do not conform to the average length and average width are scaled to obtain multiple character images that conform to the average length and average width. Multiple character images that conform to the average length and average width are horizontally spliced to obtain a horizontal text image.

[0112] In operation S780, text recognition is performed on the horizontal text image to obtain a recognition result.

[0113] According to an embodiment of the present disclosure, performing text recognition on a horizontal text image to obtain a recognition result may include: inputting the horizontal text image into a universal recognition model to obtain a recognition result.

[0114] According to the embodiments of the present disclosure, the horizontal text image obtained by the above method is beneficial to the text recognition of the general recognition model, and improves the recognition accuracy while avoiding further training of the general recognition model.

[0115] Figure 8 A schematic diagram of a text detection box according to an embodiment of the present disclosure is schematically shown.

[0116] like Figure 8 As shown, text detection is performed on the first image to be identified 810. Taking the character "第" in the first column and the first row as an example, the second spacing between the two adjacent characters "第" and "第", for example, the spacing in the horizontal direction, is determined to be L1, and the first spacing between the two adjacent characters "第" and "07", for example, the spacing in the vertical direction, is determined to be L2. It is determined that L1 is greater than L2. Taking this as an example, the first spacing or the second spacing between each character and the adjacent character is determined to obtain the first spacing and the second spacing corresponding to each character. When it is determined that the first spacing corresponding to each character is less than the second spacing, it is determined that the text in the first image to be identified 810 is vertical text. As shown in FIG. Figure 8As shown, text detection is performed on the second image to be recognized 820. Taking the character "Di" in the first row and first column as an example, the first spacing between two adjacent characters "Di" and "Di" is determined, for example, the spacing in the vertical direction, as L3, and the second spacing between two adjacent characters "Di" and "07" is determined, for example, the spacing in the horizontal direction, as L4. It is determined that L3 is greater than L4. Taking this as an example, the first spacing or the second spacing between each character and its adjacent character is determined, and the first spacing and the second spacing corresponding to each character are obtained. In the case where it is determined that the first spacing corresponding to each character is greater than the second spacing, the text in the second image to be recognized 820 is determined to be horizontal text.

[0117] According to an embodiment of the present disclosure, using the above method to determine the arrangement mode of the text in the image to be recognized is effective and simple.

[0118] Figure 9 A block diagram of a text recognition device according to an embodiment of the present disclosure is schematically shown.

[0119] As Figure 9 shown, the text recognition device 900 includes: a complexity determination module 910, a mode determination module 920, a processing module 930, and a recognition module 940.

[0120] The complexity determination module 910 is configured to determine the background complexity of the image to be recognized in the case where the text in the image to be recognized is determined to be vertical text.

[0121] The mode determination module 920 is configured to determine a target recognition mode that matches the background complexity.

[0122] The processing module 930 is configured to determine a target character region in the image to be recognized according to the target recognition mode.

[0123] The recognition module 940 is configured to perform text recognition on the image to be recognized based on the target character region to obtain a recognition result.

[0124] According to an embodiment of the present disclosure, the target recognition mode includes a decoding recognition mode.

[0125] According to an embodiment of the present disclosure, the processing module includes: a feature extraction sub-module, a matrix processing sub-module, a label processing sub-module, and a first region determination sub-module.

[0126] The feature extraction sub-module is configured to perform feature extraction on the image to be recognized to obtain a sequence of feature maps, where the sequence of feature maps includes a plurality of feature maps, and each feature map corresponds to a sub-image in the image to be recognized, and the sub-image is an image previously divided from the image to be recognized.

[0127] A matrix processing sub-module, configured to obtain a probability matrix regarding character categories based on a sequence of feature maps.

[0128] A label processing sub-module, configured to obtain a character index sequence based on the probability matrix, where the character index sequence includes a plurality of character index labels, and the character index labels are labels used to represent character categories.

[0129] A first region determination sub-module, configured to determine a target character region in the image to be recognized based on the character index sequence.

[0130] According to an embodiment of the present disclosure, the first region determination sub-module includes: a label determination unit, an arrangement determination unit, and a center determination unit.

[0131] The label determination unit is configured to determine a target character index label from the character index sequence.

[0132] The arrangement determination unit is configured to determine arrangement position information of the target character index label in the character index sequence.

[0133] The center determination unit is configured to determine center position information of the target character region based on the arrangement position information.

[0134] A first region determination unit, configured to determine a target character region in the image to be recognized based on the number of target character index labels, the size of the image to be recognized, and the center position information.

[0135] According to an embodiment of the present disclosure, the target recognition mode includes a binary recognition mode.

[0136] According to an embodiment of the present disclosure, the processing module includes: a binary processing sub-module, a projection sub-module, and a second region determination sub-module.

[0137] The binary processing sub-module is configured to perform binary processing on the image to be recognized to obtain a binary image.

[0138] The projection sub-module is configured to perform horizontal projection on the binary image to obtain a projection map.

[0139] The second region determination sub-module is configured to determine a target character region in the image to be recognized based on the projection map.

[0140] According to an embodiment of the present disclosure, the second region determination sub-module includes: a connected component determination unit, a distance determination unit, and a second region determination unit.

[0141] The connected component determination unit is configured to determine a plurality of connected components based on the projection map.

[0142] The distance determination unit is configured to determine the interval distance between two adjacent connected components to obtain a plurality of interval distances.

[0143] A second region determination unit configured to determine a target character region from a plurality of connected components based on a plurality of spacing distances.

[0144] According to an embodiment of the present disclosure, the pattern determination module includes: a first pattern determination sub-module.

[0145] The first pattern determination sub-module is configured to determine that the target recognition pattern is a decoding recognition pattern when it is determined that the background complexity is greater than or equal to a predetermined background complexity threshold.

[0146] According to an embodiment of the present disclosure, the pattern determination module includes: a second pattern determination sub-module.

[0147] The second pattern determination sub-module is configured to determine that the target recognition pattern is a binarization recognition pattern when it is determined that the background complexity is less than a predetermined background complexity threshold.

[0148] According to an embodiment of the present disclosure, the complexity determination module includes: a clustering sub-module and a complexity determination sub-module.

[0149] The clustering sub-module is configured to perform color clustering processing on the image to be recognized to obtain a color category result of the image to be recognized.

[0150] The complexity determination sub-module is configured to determine the background complexity based on the color category result.

[0151] According to an embodiment of the present disclosure, the recognition module includes: a splitting sub-module, a generating sub-module, and a recognition sub-module.

[0152] The splitting sub-module is configured to split the image to be recognized based on the target character region to obtain a character image.

[0153] The generating sub-module is configured to generate a horizontal text image based on the character image.

[0154] The recognition sub-module is configured to perform text recognition on the horizontal text image to obtain a recognition result.

[0155] According to an embodiment of the present disclosure, the text recognition device further includes: a detection module and an arrangement determination module.

[0156] The detection module is configured to perform text detection on the image to be recognized and determine the spacing between each character in the image to be recognized and the characters adjacent to the character, where the spacing includes a first spacing in the vertical direction and a second spacing in the horizontal direction.

[0157] The arrangement determination module is configured to determine the arrangement type of the text in the image to be recognized based on the first spacing and the second spacing, where the arrangement type includes a vertical arrangement type and a horizontal arrangement type.

[0158] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0159] According to an embodiment of the present disclosure, an electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method according to the embodiment of the present disclosure.

[0160] According to an embodiment of the present disclosure, a non-transitory computer-readable storage medium stores computer instructions, wherein the computer instructions are used to cause a computer to execute the method according to the embodiment of the present disclosure.

[0161] According to an embodiment of the present disclosure, a computer program product includes a computer program, and the computer program implements the method according to the embodiment of the present disclosure when executed by a processor.

[0162] Figure 10 A schematic block diagram of an exemplary electronic device 1000 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely exemplary and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0163] As Figure 10 shown, the device 1000 includes a computing unit 1001, which can execute various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1002 or a computer program loaded from a storage unit 1008 into a random access memory (RAM) 1003. In the RAM 1003, various programs and data required for the operation of the device 1000 can also be stored. The computing unit 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0164] Multiple components in device 1000 are connected to I / O interface 1005, including: an input unit 1006, such as a keyboard, a mouse, etc.; an output unit 1007, such as various types of displays, speakers, etc.; a storage unit 1008, such as a disk, an optical disc, etc.; and a communication unit 1009, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1009 allows device 1000 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0165] The computing unit 1001 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1001 include but are not limited to a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1001 executes the various methods and processes described above, such as the text recognition method. For example, in some embodiments, the text recognition method can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as the storage unit 1008. In some embodiments, part or all of the computer program can be loaded and / or installed onto device 1000 via the ROM 1002 and / or the communication unit 1009. When the computer program is loaded into the RAM 1003 and executed by the computing unit 1001, one or more steps of the text recognition method described above can be executed. Alternatively, in other embodiments, the computing unit 1001 can be configured to execute the text recognition method in any other suitable manner (e.g., by means of firmware).

[0166] The various embodiments of the systems and technologies described above in this article can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special or general-purpose programmable processor, receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0167] The program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing devices, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowchart and / or block diagram are implemented. The program codes can be executed entirely on the machine, partially on the machine, as an independent software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0168] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0169] In order to provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and the input received from the user can be in any form (including acoustic input, speech input, or tactile input).

[0170] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend, middleware, or frontend components. The components of the system can be interconnected to each other by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), and the Internet.

[0171] A computer system can include a client and a server. The client and the server are generally far from each other and typically interact through a communication network. The client-server relationship is generated by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, a server of a distributed system, or a server incorporating blockchain.

[0172] It should be understood that various forms of the processes shown above can be used, steps can be reordered, added, or deleted. For example, the steps recited in this disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this is not limited herein.

[0173] The above specific implementation manners do not constitute a limitation on the protection scope of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principle of this disclosure shall be included within the protection scope of this disclosure.

Claims

1. A text recognition method, comprising: When it is determined that the text in the image to be recognized is vertical text, determining the background complexity of the image to be recognized, including: performing color clustering processing on the image to be recognized to obtain a color category result of the image to be recognized; and determining the background complexity based on the color category result; Determining a target recognition mode that matches the background complexity, including: when it is determined that the background complexity is greater than or equal to a predetermined background complexity threshold, determining the target recognition mode as a decoding recognition mode; and when it is determined that the background complexity is less than the predetermined background complexity threshold, determining the target recognition mode as a binarization recognition mode; Determining a target character region in the image to be recognized according to the target recognition mode; and Performing text recognition on the image to be recognized based on the target character region to obtain a recognition result.

2. The method according to claim 1, wherein The target recognition mode includes the decoding recognition mode; The determining a target character region in the image to be recognized according to the target recognition mode includes: Performing feature extraction on the image to be recognized to obtain a sequence of feature maps, where the sequence of feature maps includes a plurality of feature maps, and each feature map corresponds to a sub-image in the image to be recognized, and the sub-image is an image pre-divided from the image to be recognized; Obtaining a probability matrix regarding character categories based on the sequence of feature maps; Obtaining a character index sequence based on the probability matrix, where the character index sequence includes a plurality of character index labels, and the character index label is a label used to represent a character category; and Determining a target character region in the image to be recognized based on the character index sequence.

3. The method according to claim 2, wherein, The determining a target character region in the image to be recognized based on the character index sequence includes: Determining a target character index label from the character index sequence; Determining arrangement position information of the target character index label in the character index sequence; Determining central position information of the target character region based on the arrangement position information; and Determining the target character region in the image to be recognized based on the number of the target character index labels, the size of the image to be recognized, and the central position information.

4. The method according to claim 1, wherein The target recognition mode includes the binarization recognition mode; The determining a target character region in the image to be recognized according to the target recognition mode includes: Performing binarization processing on the image to be recognized to obtain a binarized image; Performing horizontal projection on the binarized image to obtain a projection map; and Determining a target character region in the image to be recognized based on the projection map.

5. The method according to claim 4, wherein, The determining a target character region in the image to be recognized based on the projection map includes: Determining a plurality of connected components based on the projection map; Determining a spacing distance between two adjacent connected components to obtain a plurality of spacing distances; and Determining the target character region from the plurality of connected components based on the plurality of spacing distances.

6. The method according to claim 1, wherein The performing text recognition on the image to be recognized based on the target character region to obtain a recognition result includes: Segment the image to be recognized based on the target character region to obtain character images; Generate a horizontal text image based on the character images; and Perform text recognition on the horizontal text image to obtain a recognition result.

7. The method according to claim 1, further comprising: Perform text detection on the image to be recognized, and determine the spacing between each character in the image to be recognized and the characters adjacent to the character, where the spacing includes a first spacing in the vertical direction and a second spacing in the horizontal direction; and Determine the layout type of the text in the image to be recognized based on the first spacing and the second spacing, where the layout type includes a vertical layout type and a horizontal layout type.

8. A text recognition device, comprising: A complexity determination module, configured to determine the background complexity of the image to be recognized when it is determined that the text in the image to be recognized is vertical text; A mode determination module, configured to determine a target recognition mode that matches the background complexity; A processing module, configured to determine a target character region in the image to be recognized according to the target recognition mode; And A recognition module, configured to perform text recognition on the image to be recognized based on the target character region to obtain a recognition result; Wherein, the mode determination module includes: A first mode determination sub-module, configured to determine that the target recognition mode is a decoding recognition mode when it is determined that the background complexity is greater than or equal to a predetermined background complexity threshold; and A second mode determination sub-module, configured to determine that the target recognition mode is a binarization recognition mode when it is determined that the background complexity is less than the predetermined background complexity threshold; Wherein, the complexity determination module includes: A clustering sub-module, configured to perform color clustering processing on the image to be recognized to obtain a color category result of the image to be recognized; and A complexity determination sub-module, configured to determine the background complexity based on the color category result.

9. The device according to claim 8, wherein, The target recognition mode includes the decoding recognition mode; The processing module includes: A feature extraction sub-module, configured to perform feature extraction on the image to be recognized to obtain a sequence of feature maps, where the sequence of feature maps includes a plurality of feature maps, and each feature map corresponds to a sub-image in the image to be recognized, and the sub-image is an image previously divided from the image to be recognized; A matrix processing sub-module, configured to obtain a probability matrix regarding character categories based on the sequence of feature maps; A label processing sub-module, configured to obtain a character index sequence based on the probability matrix, where the character index sequence includes a plurality of character index labels, and the character index label is a label used to represent a character category; and A first region determination sub-module, configured to determine a target character region in the image to be recognized based on the character index sequence.

10. The apparatus according to claim 9, wherein The first region determination sub-module includes: A label determination unit, configured to determine a target character index label from the character index sequence; An arrangement determination unit, configured to determine the arrangement position information of the target character index label in the character index sequence; A center determination unit for determining the center position information of the target character region based on the arrangement position information; and A first region determination unit for determining the target character region in the image to be recognized based on the number of the target character index labels, the size of the image to be recognized, and the center position information.

11. The apparatus according to claim 8, wherein, The target recognition mode includes the binarization recognition mode; The processing module includes: A binarization processing sub-module for performing binarization processing on the image to be recognized to obtain a binarized image; A projection sub-module for performing horizontal projection on the binarized image to obtain a projection map; and A second region determination sub-module for determining the target character region in the image to be recognized based on the projection map.

12. The apparatus according to claim 11, wherein, The second region determination sub-module includes: A connected component determination unit for determining a plurality of connected components based on the projection map; A distance determination unit for determining the spacing distance between two adjacent connected components to obtain a plurality of spacing distances; and A second region determination unit for determining the target character region from the plurality of connected components based on the plurality of spacing distances.

13. The device according to claim 8, wherein, The recognition module includes: A splitting sub-module for splitting the image to be recognized based on the target character region to obtain character images; A generation sub-module for generating a horizontal text image based on the character images; and A recognition sub-module for performing text recognition on the horizontal text image to obtain a recognition result.

14. The apparatus according to claim 8, further comprising: A detection module for performing text detection on the image to be recognized to determine the spacing between each character in the image to be recognized and the characters adjacent to the character, wherein the spacing includes a first spacing in the vertical direction and a second spacing in the horizontal direction; and An arrangement determination module for determining the arrangement type of the text in the image to be recognized based on the first spacing and the second spacing, wherein the arrangement type includes a vertical arrangement type and a horizontal arrangement type.

15. An electronic device, comprising: At least one processor; And A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method according to any one of claims 1 to 7.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 7.

17. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images