Text recognition method and device, computer equipment, storage medium and program product

Abstract

The invention relates to a text recognition method and device, computer equipment, a storage medium and a program product. The method comprises the following steps: acquiring a to-be-detected image; the to-be-detected image comprises texts in a plurality of writing directions; inputting the to-be-detected image into a text recognition network for text recognition to obtain a recognition result; the text recognition network is obtained by training sample data based on a visual sequence. The text recognition network is a visual model, and the visual model can learn the spatial position information in the image, so that the text recognition network is obtained through training based on the sample data of the visual sequence, feature learning of the text recognition network can be better carried out, and the recognition efficiency of the text recognition network is improved. Therefore, a more accurate text recognition network can be obtained. And furthermore, the to-be-detected image is input into the accurate text recognition network for text recognition, so that a recognition result can be accurately obtained.

Description

Technical Field

[0001] The present application relates to the field of information processing technology, and in particular to a text recognition method, apparatus, computer equipment, storage medium and program product. Background Art

[0002] With the continuous development of information processing technology, text recognition technology has emerged. Text recognition technology has been widely used in document digitization, automated data input and other fields. Text recognition technology can be used to recognize and output text in text images. With the diversification of text image types, many application scenarios currently require text recognition for images containing bidirectional text.

[0003] However, traditional methods for text recognition on images with bidirectional text have low accuracy. Summary of the invention

[0004] Based on this, it is necessary to provide a text recognition method, apparatus, computer equipment, storage medium and program product that can improve the accuracy of text recognition in order to address the above technical problems.

[0005] In a first aspect, the present application provides a text recognition method, comprising:

[0006] Acquire an image to be tested; the image to be tested includes texts in multiple writing directions;

[0007] The image to be tested is input into a text recognition network for text recognition to obtain a recognition result; the text recognition network is trained based on sample data of visual sequence.

[0008] In the above text recognition method, since the text recognition network is a visual model, and the visual model can learn the spatial position information in the image, the embodiment of the present application uses sample data in a visual order to train the text recognition network, which can better learn the features of the text recognition network, thereby obtaining a more accurate text recognition network. Furthermore, the image to be tested is input into an accurate text recognition network for text recognition, and the recognition result can be accurately obtained.

[0009] In one embodiment, the text recognition network includes a text recognition subnetwork and a sequence adjustment subnetwork, and the inputting of the image to be tested into the text recognition network for text recognition to obtain a recognition result includes:

[0010] Inputting the image to be tested into the text recognition subnetwork to perform text recognition to obtain an intermediate text;

[0011] The sequence adjustment subnetwork is used to adjust the text order of the intermediate text into a logical order to obtain the recognition result.

[0012] In this embodiment, the image to be tested can be input into the text recognition subnetwork for text recognition to obtain the intermediate text in the visual order, and then the sequence adjustment subnetwork is used to adjust the text order of the intermediate text to the logical order, so that the recognition result in the logical order can be accurately obtained. In this way, the recognition result in the logical order can be accurately identified based on the trained visual model (i.e., the text recognition network).

[0013] In one embodiment, the text recognition network training method includes:

[0014] Acquire a sample image set; the sample image set includes sample images and label texts; the sample images include text corpora in multiple writing directions, and the text order of the label texts is a visual order;

[0015] The initial recognition network is trained according to the sample image set to obtain the text recognition network.

[0016] In this embodiment, a sample image set can be obtained, wherein the sample image set includes sample images and label texts; the sample images include text corpora in multiple writing directions, and the text order of the label texts is the visual order. Therefore, the sample image data set in which the label text order is the visual order can be used to perform more accurate model training on the initial recognition network in the visual field, and the feature learning of the text recognition network can be better performed, so that a more accurate text recognition network can be obtained.

[0017] In one embodiment, the method for obtaining the label text includes:

[0018] The text order of the text corpus in each of the writing directions is adjusted from a logical order to a visual order to obtain the label text.

[0019] In this embodiment, since the order of words when the user reads and the order in which the text is stored in the memory are in a logical order, but what can be learned based on the visual model is the spatial position information in the image, the text sequence order of the text corpus in each writing direction in the sample image can be converted from a logical order to a visual order to obtain the label text, and the label text in the visual order is used for model training, so that the feature learning of the text recognition network can be better performed.

[0020] In one embodiment, the method further comprises:

[0021] Acquire a background image set and text corpora of the plurality of writing directions; the background image set includes background images with different background patterns;

[0022] Expanding the sample image set according to the background image set and the text corpora of the multiple writing directions to obtain an expanded sample image set;

[0023] The step of training the initial recognition network according to the sample image set to obtain the text recognition network comprises:

[0024] The initial recognition network is trained according to the expanded sample image set to obtain the text recognition network.

[0025] In this embodiment, a background image set and text corpora of multiple writing directions can be obtained, wherein the background image set includes background images of different background patterns. Thus, the sample image set can be expanded according to the background image set and the text corpora of multiple writing directions to obtain an expanded sample image set, which can further enrich the sample image set for model training. Afterwards, the initial recognition network is trained according to the richer expanded sample image set, and a more accurate text recognition network can be trained.

[0026] In one embodiment, the sample image set is expanded according to the background image set and the text corpora of the multiple writing directions to obtain the expanded sample image set, including:

[0027] Randomly writing the text corpora in the multiple writing directions into each of the background images to generate multiple background images with written text;

[0028] The expanded sample image set is generated according to the multiple background images with written texts and the sample image set.

[0029] In this embodiment, text corpora in multiple writing directions can be randomly written into each background image to generate multiple background images with written text. In this way, a large number of bidirectional text image data in real life can be simulated and generated, greatly improving the efficiency of data set preparation. Therefore, based on multiple background images with written text and sample image sets, a richer and expanded sample image set for model training can be obtained.

[0030] In a second aspect, the present application further provides a text recognition device, comprising:

[0031] An acquisition module, used for acquiring an image to be tested; the image to be tested includes texts in multiple writing directions;

[0032] The text recognition module is used to input the image to be tested into a text recognition network for text recognition to obtain a recognition result; the text recognition network is trained based on sample data of visual sequence.

[0033] In a third aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the method in any one of the embodiments of the first aspect are implemented.

[0034] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method in any one of the embodiments of the first aspect are implemented.

[0035] In a fifth aspect, the present application further provides a computer program product, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the method in any one of the embodiments of the first aspect are implemented.

[0036] The above-mentioned text recognition method, device, computer equipment, storage medium and program product obtain an image to be tested; the image to be tested includes texts in multiple writing directions; the image to be tested is input into a text recognition network for text recognition to obtain a recognition result; the text recognition network is trained based on sample data of visual sequence. Since the text recognition network is a visual model, and what the visual model can learn is the spatial position information in the image, therefore, the embodiment of the present application trains the text recognition network based on sample data of visual sequence, which can better perform feature learning of the text recognition network, thereby obtaining a more accurate text recognition network. Furthermore, the image to be tested is input into an accurate text recognition network for text recognition, and an accurate recognition result can be obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0038] Figure 1 A diagram showing an application environment of a text recognition method in an embodiment;

[0039] Figure 2 A flowchart of a text recognition method in one embodiment;

[0040] Figure 3 A schematic diagram of a flow chart of a sequence adjustment step in an embodiment;

[0041] Figure 4 is an example diagram of an image to be tested in an embodiment;

[0042] Figure 5 An example diagram of an intermediate text in an embodiment;

[0043] Figure 6 An example diagram of a recognition result in one embodiment;

[0044] Figure 7 is a flowchart of a text recognition method in another embodiment;

[0045] Figure 8 A schematic diagram of the structure of a CRNN network in one embodiment;

[0046] Fig. 9 is an example diagram of a sample image in an embodiment;

[0047] Fig.10 An example diagram of text prediction in a logical order corresponding to a sample image in one embodiment;

[0048] Fig.11 is an example diagram of label text in visual order in one embodiment;

[0049] Fig.12 is a flow chart of a text recognition method including a sample image set expansion step in another embodiment;

[0050] Fig.13 is a flow chart of a sample image set expansion step in one embodiment;

[0051] Fig.14 is a flowchart of a text recognition method in an optional embodiment;

[0052] Fig.15 FIG. 4 is a structural block diagram of a text recognition device in an embodiment. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0054] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by technicians in the technical field to which this application belongs; the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit this application; the terms "including" and "having" in the specification and claims of this application and the above-mentioned figure descriptions and any variations thereof are intended to cover non-exclusive inclusions.

[0055] In the description of the embodiments of the present application, the technical terms "first", "second", etc. are only used to distinguish different objects, and cannot be understood as indicating or implying relative importance or implicitly indicating the number, specific order or primary and secondary relationship of the indicated technical features. In the description of the embodiments of the present application, the meaning of "multiple" is more than two, unless otherwise clearly and specifically defined.

[0056] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0057] With the continuous development of information processing technology, text recognition technology has emerged. Text recognition technology has been widely used in document digitization, automated data input and other fields. Text recognition technology can be used to recognize and output text in text images. With the diversification of text image types, many application scenarios currently require text recognition for images containing bidirectional text. For example, it is necessary to recognize text written from left to right (such as Latin, Chinese, etc.) and text written from right to left (such as Arabic, Uyghur, etc.) in the same image.

[0058] The existing text recognition technology for bidirectional text images uses a convolutional neural network, an attention mechanism module and a Softmax classifier to recognize the characters of the bidirectional text. However, since bidirectional texts (such as Arabic, Uighur, Persian, etc.) are all multiple characters spliced ​​together, there is no obvious separator between the characters, so the prior art is more difficult to process the bidirectional text sequence in the actual scene, and the efficiency is low. Therefore, the traditional method for text recognition of images of bidirectional text has the problem of low accuracy. After the above-mentioned background technology of the text recognition method provided by the embodiment of the present application is introduced, the implementation environment involved in the text recognition method provided by the embodiment of the present application will be briefly described below.

[0059] The text recognition method provided in the embodiment of the present application can be applied to Figure 1In the computer device shown. The computer device can be a terminal or a server, and the computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a text recognition method is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0060] Those skilled in the art will understand that Figure 1 The structure shown in the figure is only a block diagram of a part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific terminal may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0061] In one embodiment, Figure 2 As shown, a text recognition method is provided, which is applied to Figure 1 The computer device in the example is used to illustrate, including the following steps:

[0062] S201, obtaining an image to be tested; the image to be tested includes texts in multiple writing directions.

[0063] The image to be tested refers to an image that needs to be subjected to text recognition. The image to be tested includes texts in multiple writing directions. The characteristic of the texts in multiple writing directions is that texts in different writing directions are mixed in the same paragraph. Optionally, the multiple writing directions include a text direction written from left to right (such as Latin, Chinese, etc.) and a text direction written from right to left (such as Arabic, Uyghur, etc.).

[0064] In the embodiment of the present application, the computer device can optionally acquire the image to be tested in real time; or the computer device can also acquire the image to be tested at a fixed time; or the computer device can also acquire the image to be tested when receiving a text recognition instruction. Of course, the embodiment of the present application does not limit the specific implementation method of acquiring the image to be tested.

[0065] S202, inputting the image to be tested into a text recognition network for text recognition to obtain a recognition result; the text recognition network is trained based on sample data of visual sequence.

[0066] Among them, the text recognition network may include but is not limited to the text recognition subnetwork, etc. The text recognition subnetwork may include but is not limited to CRNN (Convolutional Recurrent Neural Network) or SVTR (Scene Text Recognition with a Single Visual Model) network, etc. The text recognition subnetwork in the embodiment of the present application is trained based on sample data in visual order. Since the text recognition subnetwork is a visual model, and what the visual model can learn is the spatial position information in the image, therefore, the embodiment of the present application is based on the sample data in visual order to train the text recognition subnetwork, which can be beneficial to the feature learning of the text recognition subnetwork, that is, it is beneficial to the feature learning of the text recognition network. The text recognition process refers to the process of recognizing the text in a picture or scan as editable text. The recognition result refers to the text recognized from the image to be tested.

[0067] In the embodiment of the present application, optionally, the computer device can directly input the image to be tested into the text recognition network for text recognition to obtain a recognition result; or, the computer device can first perform image preprocessing on the image to be tested, and then input the image to be tested after image preprocessing into the text recognition network for text recognition to obtain a recognition result. Of course, the embodiment of the present application does not limit the specific implementation method of text recognition.

[0068] In the above text recognition method, an image to be tested is obtained; the image to be tested includes texts in multiple writing directions; the image to be tested is input into a text recognition network for text recognition to obtain a recognition result; the text recognition network is trained based on sample data in visual order. Since the text recognition network is a visual model, and what the visual model can learn is the spatial position information in the image, therefore, the embodiment of the present application trains the text recognition network based on sample data in visual order, which can better perform feature learning of the text recognition network, thereby obtaining a more accurate text recognition network. Furthermore, the image to be tested is input into an accurate text recognition network for text recognition, and an accurate recognition result can be obtained.

[0069] In one embodiment, the text recognition network includes a text recognition subnetwork and a sequence adjustment subnetwork. Based on this, a method for implementing text recognition is provided, namely, "inputting the image to be tested into the text recognition network for text recognition to obtain a recognition result" in the above S202. Figure 3 As shown, including:

[0070] S301, inputting the image to be tested into the text recognition sub-network to perform text recognition and obtain intermediate text.

[0071] In the embodiment of the present application, since the above-mentioned text recognition network includes a text recognition subnetwork and a sequence adjustment subnetwork, the computer device can input the image to be tested into the text recognition subnetwork for text recognition to obtain an intermediate text. The intermediate text refers to the text directly predicted by the text recognition subnetwork, and the text sequence order of the intermediate text is the visual order. For example, Figure 4 is an example diagram of an image to be tested in an embodiment, Figure 5 An example diagram of intermediate text in an embodiment.

[0072] S302, using a sequence adjustment sub-network to adjust the text order of the intermediate text to a logical order to obtain a recognition result.

[0073] In an embodiment of the present application, the computer device may use a sequence adjustment subnetwork based on a BIDI (bi-directional language, bidirectional text) algorithm to adjust the text order of the intermediate text from a visual order to a logical order to obtain a recognition result. The text sequence order of the recognition result is a logical order. For example, Figure 6 FIG. 1 is an example diagram of a recognition result in an embodiment. The BIDI algorithm is a computer algorithm for processing and correctly displaying mixed texts in different writing directions.

[0074] In this embodiment, the image to be tested can be input into the text recognition subnetwork for text recognition to obtain the intermediate text in the visual order, and then the sequence adjustment subnetwork is used to adjust the text order of the intermediate text to the logical order, so that the recognition result in the logical order can be accurately obtained. In this way, the recognition result in the logical order can be accurately identified based on the trained visual model (i.e., the text recognition network).

[0075] In the above embodiment, a training method for a text recognition network is provided, namely, Figure 2 The text recognition method of the embodiment, such as Figure 7 As shown, it also includes:

[0076] S203, obtaining a sample image set; the sample image set includes sample images and label texts; the sample images include text corpora in multiple writing directions, and the text order of the label texts is the visual order.

[0077] In an embodiment of the present application, optionally, the computer device can directly obtain a sample image set from a database and / or a public platform; or, the computer device can first obtain a sample image of a text corpus including multiple writing directions, and then determine the label text from the sample image, so that the sample image and the label text can be combined to construct a sample image set. Of course, the embodiment of the present application does not limit the specific implementation method for obtaining the sample image set. Among them, the sample image set refers to a data set used to train a text recognition network, the sample image set includes sample images and label text, the sample image includes text corpus in multiple writing directions, the label text refers to the real text corresponding to the sample image, and the text order of the label text is the visual order.

[0078] S204, training the initial recognition network according to the sample image set to obtain a text recognition network.

[0079] In the embodiment of the present application, the computer device can divide the sample image set into a training set, a validation set, and a test set. The training set is used for learning and optimizing model parameters; the validation set is used for selecting and tuning the model to prevent overfitting; and the test set is used for the final evaluation of the performance of the model to ensure the generalization ability of the model. Thus, the computer device can train the initial recognition network based on the training set and the validation set in the sample image set to obtain an intermediate recognition network that performs best on the validation set.

[0080] Since in the process of the above-mentioned model training, the learning, optimization and selection of parameters are based on the data seen by the model, the model performance indicators output by the above-mentioned model training process are not objective, that is, the performance of the intermediate recognition network needs to be further tested. Based on this, the computer device can use the test set in the sample image set to evaluate the final performance of the intermediate recognition network, so as to use the test set data that the model has not seen to evaluate the model performance and obtain the performance evaluation result. If the performance evaluation result indicates that the performance meets the expected requirements, the computer device can determine the intermediate recognition network as the trained text recognition network; if the performance evaluation result indicates that the performance does not meet the expected requirements, the computer device can reconstruct a bidirectional text image data set that is more in line with the real scene, and repeat the above-mentioned model training process until the performance of the model on the test set meets the expected requirements. At this time, the intermediate recognition network that meets the expected requirements can be determined as the trained text recognition subnetwork. It should be noted that the subnetwork that needs to be trained in the text recognition network is only the text recognition subnetwork, so when the trained text recognition subnetwork is obtained, it means that the text recognition network training is completed.

[0081] The initial recognition network refers to a text recognition subnetwork that has not been trained. The initial recognition network may include but is not limited to a CRNN (Convolutional Recurrent Neural Network) or a SVTR (Scene Text Recognition with a Single Visual Model) network. For example, Figure 8It is a structural diagram of a CRNN network in an embodiment, wherein the CRNN network can be used to recognize the text "state" in the input image (Input image), and the CRNN network includes convolutional layers, recurrent layers, and transcription layers. The computer device can input the input image (Input image) into the convolutional layer to obtain the convolutional feature map (Convolutional feature maps) corresponding to the input image, and process the convolutional feature map through the convolutional layer to obtain the convolutional feature map (Convolutional feature maps). Afterwards, the computer device can input the convolutional feature map into the recurrent layer for processing to obtain a feature sequence (Feature sequence), and input the feature sequence into the deep bidirectional long short-term memory network (Deep bidirectional LSTM, Deep bidirectional Long Short Term Memory) in the recurrent layer for processing to obtain a processing result. Then, the computer device can input the processing result into the transcription layer for per-frame prediction or distribution (Per-frame predictions (disbritutions)) to obtain a predicted sequence (Predicted sequence) corresponding to the input image.

[0082] In this embodiment, a sample image set can be obtained, wherein the sample image set includes sample images and label texts; the sample images include text corpora in multiple writing directions, and the text order of the label texts is the visual order. Therefore, the sample image data set in which the label text order is the visual order can be used to perform more accurate model training on the initial recognition network in the visual field, and the feature learning of the text recognition network can be better performed, so that a more accurate text recognition network can be obtained.

[0083] In the above embodiment, a method for obtaining label text is provided, namely Figure 7 The text recognition method of the embodiment further includes:

[0084] The text order of the text corpus in each writing direction is adjusted from the logical order to the visual order to obtain the label text.

[0085] In an embodiment of the present application, a computer device may obtain sample images of texts in multiple writing directions, and adjust the text order of the text corpus in each writing direction from a logical order to a visual order based on the BIDI algorithm to obtain a label text. Fig. 9is an example diagram of a sample image in an embodiment, Fig.10 is an example diagram of text corpus in a logical order corresponding to a sample image in one embodiment, Fig.11 : This is an example diagram of label text in visual order in an embodiment. The visual order refers to the order in which text is displayed on the screen or in the printer, and the logical order refers to the order in which the user reads and enters the text from the keyboard. It should be noted that the text is stored in the memory in a logical order.

[0086] In this embodiment, since the order of text when the user reads and the order in which the text is stored in the memory are in a logical order, but what can be learned based on the visual model is the spatial position information in the image, the text sequence order of the text corpus in each writing direction in the sample image can be converted from a logical order to a visual order to obtain the label text, and the label text in the visual order is used for model training. In this way, the feature learning of the text recognition network can be better performed, and the efficiency and accuracy of text recognition for bidirectional text images can be greatly improved.

[0087] In the above embodiment, a method for expanding a sample image set is provided, namely Figure 7 The text recognition method of the embodiment, such as Fig.12 As shown, it also includes:

[0088] S205, obtaining a background image set and text corpora in multiple writing directions; the background image set includes background images with different background patterns.

[0089] In an embodiment of the present application, a computer device can obtain a background image set and text corpora of multiple writing directions from a database and / or a public platform. The background image set includes background images of different background patterns, and none of the background images contains text. In addition, the computer device can also determine the fonts corresponding to the text corpora of each of the above writing directions, and construct a font library. The font library includes fonts of languages ​​corresponding to the text corpora of multiple writing directions.

[0090] S206, expanding the sample image set according to the background image set and the text corpora of multiple writing directions to obtain an expanded sample image set.

[0091] In the embodiment of the present application, optionally, the computer device can add text corpora of multiple writing directions to multiple sample images in the sample image set, and / or, the background image set can be used to change the background of multiple sample images to obtain multiple new sample images, thereby expanding the sample image set based on the multiple new sample images to obtain the expanded sample image set. Alternatively, the computer device can also construct multiple new images based on the background image set and the text corpora of multiple writing directions, and expand the sample image set based on the multiple new images to obtain the expanded sample image set. In addition, the computer device can also use multiple fonts in the font library to adjust the expanded sample image set to obtain the expanded sample image set after the font is changed, so that the sample image set for model training can be further enriched. Of course, the embodiment of the present application does not limit the specific implementation method of expanding the sample image set.

[0092] S204, namely, “training the initial recognition network according to the sample image set to obtain a text recognition network”, includes:

[0093] S207, training the initial recognition network according to the expanded sample image set to obtain a text recognition network.

[0094] In the embodiment of the present application, the computer device can divide the expanded sample image set into a training set, a validation set, and a test set, and train the initial recognition network according to the training set, the validation set, and the test set in the expanded sample image set to obtain a text recognition network. The specific process of model training can refer to the embodiment corresponding to S204, which will not be described in detail here.

[0095] In this embodiment, a background image set and text corpora of multiple writing directions can be obtained, wherein the background image set includes background images of different background patterns. Thus, the sample image set can be expanded according to the background image set and the text corpora of multiple writing directions to obtain an expanded sample image set, which can further enrich the sample image set for model training. Afterwards, the initial recognition network is trained according to the richer expanded sample image set, and a more accurate text recognition network can be trained.

[0096] In one embodiment, a method for implementing the expansion of the sample image set is provided, namely, the "expanding the sample image set according to the background image set and the text corpora of multiple writing directions to obtain the expanded sample image set" in the above S206. Fig.13 As shown, including:

[0097] S401, randomly writing text corpora in multiple writing directions into each background image to generate multiple background images with written text.

[0098] S402, generating an expanded sample image set according to a plurality of background images with written texts and the sample image set.

[0099] In the embodiment of the present application, the computer device can use the Pillow library of Python to randomly write text corpora in multiple writing directions into each background image based on various fonts in the font library to generate multiple background images with written text. The Pillow library is an image processing library for writing text content into an image. Thus, the computer device can combine multiple background images with written text and a sample image set to construct an expanded sample image set.

[0100] In this embodiment, text corpora in multiple writing directions can be randomly written into each background image to generate multiple background images with written text. In this way, a large number of bidirectional text image data in real life can be simulated and generated, greatly improving the efficiency of data set preparation. Therefore, based on multiple background images with written text and sample image sets, a richer and expanded sample image set for model training can be obtained.

[0101] Based on all the above embodiments, this application also provides a complete text recognition method, such as Fig.14 As shown, the method includes:

[0102] S21, obtaining a sample image set; the sample image set includes sample images and label texts; the sample images include text corpora in multiple writing directions; adjusting the text order of the text corpora in each writing direction from a logical order to a visual order to obtain the label text;

[0103] S22, obtaining a background image set and text corpora in multiple writing directions; the background image set includes background images with different background patterns;

[0104] S23, randomly writing text corpora in multiple writing directions into each background image to generate multiple background images with written text;

[0105] S24, generating an expanded sample image set according to the background images with multiple texts written therein and the sample image set;

[0106] S25, training the initial recognition network according to the expanded sample image set to obtain a text recognition sub-network;

[0107] S26, obtaining an image to be tested; the image to be tested includes texts in multiple writing directions;

[0108] S27, inputting the image to be tested into the text recognition subnetwork to perform text recognition and obtain an intermediate text;

[0109] S28, using the sequence adjustment sub-network to adjust the text order of the intermediate text to a logical order to obtain the recognition result.

[0110] In the above-mentioned text recognition method, an image to be tested is obtained; the image to be tested includes texts in multiple writing directions; the image to be tested is input into a text recognition network for text recognition to obtain a recognition result; the text recognition network is trained based on sample data of visual order. Since the text recognition network is a visual model, and what the visual model can learn is the spatial position information in the image, therefore, the embodiment of the present application trains the text recognition network based on sample data of visual order, which can better perform feature learning of the text recognition network, thereby obtaining a more accurate text recognition network. Furthermore, the image to be tested is input into an accurate text recognition network for text recognition, and the recognition result can be obtained accurately and efficiently.

[0111] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0112] Based on the same inventive concept, the embodiment of the present application also provides a text recognition device for implementing the above-mentioned text recognition method. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above-mentioned method, so the specific limitations in one or more text recognition device embodiments provided below can refer to the limitations of the text recognition method above, and will not be repeated here.

[0113] In an exemplary embodiment, Fig.15 As shown, a text recognition device is provided, including: an acquisition module 31 and a text recognition module 32, wherein:

[0114] The acquisition module 31 is used to acquire the image to be tested; the image to be tested includes texts in multiple writing directions.

[0115] The text recognition module 32 is used to input the image to be tested into a text recognition network for text recognition to obtain a recognition result; the text recognition network is trained based on sample data of visual sequence.

[0116] In one embodiment, the text recognition network includes a text recognition subnetwork and a sequence adjustment subnetwork, and the text recognition module 32 includes:

[0117] A text recognition unit, used for inputting the image to be tested into the text recognition sub-network for text recognition to obtain an intermediate text;

[0118] The sequence adjustment unit is used to adjust the text order of the intermediate text into a logical order by using a sequence adjustment sub-network to obtain a recognition result.

[0119] In one embodiment, the text recognition device further includes:

[0120] The sample image set acquisition module is used to acquire a sample image set; the sample image set includes sample images and label texts; the sample images include text corpora in multiple writing directions, and the text order of the label texts is the visual order;

[0121] The training module is used to train the initial recognition network according to the sample image set to obtain the text recognition network.

[0122] In one embodiment, the text recognition device further includes:

[0123] The label text acquisition module is used to adjust the text order of the text corpus in each writing direction from the logical order to the visual order to obtain the label text.

[0124] In one embodiment, the text recognition device further includes:

[0125] A background image set acquisition module is used to acquire a background image set and text corpora in multiple writing directions; the background image set includes background images with different background patterns;

[0126] An expansion module, used for expanding the sample image set according to the background image set and the text corpus of multiple writing directions to obtain an expanded sample image set;

[0127] The training modules include:

[0128] The training unit is used to train the initial recognition network according to the expanded sample image set to obtain a text recognition network.

[0129] In one embodiment, the expansion module includes:

[0130] A writing unit, used for randomly writing text corpora in multiple writing directions into each background image to generate multiple background images with written texts;

[0131] The expanded sample image set generating unit is used to generate an expanded sample image set according to a plurality of background images with written texts and the sample image set.

[0132] Each module in the above-mentioned text recognition device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.

[0133] In an exemplary embodiment, a computer device is provided. The computer device may be a server, or the computer device may be a terminal. The internal structure diagram thereof may be as follows: Figure 1 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a text recognition method is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0134] Those skilled in the art will understand that Figure 1 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0135] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0136] Acquire an image to be tested; the image to be tested includes texts in multiple writing directions;

[0137] The image to be tested is input into the text recognition network for text recognition to obtain the recognition result; the text recognition network is trained based on sample data of visual sequence.

[0138] In one embodiment, the text recognition network includes a text recognition subnetwork and a sequence adjustment subnetwork. The image to be tested is input into the text recognition network for text recognition to obtain a recognition result. When the processor executes the computer program, the following steps are also implemented:

[0139] The image to be tested is input into the text recognition subnetwork for text recognition to obtain the intermediate text;

[0140] The sequence adjustment subnetwork is used to adjust the text order of the intermediate text to a logical order to obtain the recognition result.

[0141] In one embodiment, in the text recognition network training method, when the processor executes the computer program, the processor further implements the following steps:

[0142] A sample image set is obtained; the sample image set includes sample images and label texts; the sample images include text corpora in multiple writing directions, and the text order of the label texts is the visual order;

[0143] The initial recognition network is trained according to the sample image set to obtain the text recognition network.

[0144] In one embodiment, in the method for obtaining label text, when the processor executes the computer program, the processor further implements the following steps:

[0145] The text order of the text corpus in each writing direction is adjusted from the logical order to the visual order to obtain the label text.

[0146] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0147] Obtaining a background image set and text corpora in multiple writing directions; the background image set includes background images with different background patterns;

[0148] Expanding the sample image set according to the background image set and the text corpora of multiple writing directions to obtain an expanded sample image set;

[0149] The initial recognition network is trained according to the sample image set to obtain a text recognition network, including:

[0150] The initial recognition network is trained according to the expanded sample image set to obtain a text recognition network.

[0151] In one embodiment, the sample image set is expanded according to the background image set and the text corpora of multiple writing directions to obtain an expanded sample image set, and the processor further implements the following steps when executing the computer program:

[0152] Randomly write text corpora in multiple writing directions into each background image to generate multiple background images with written texts;

[0153] An expanded sample image set is generated according to a plurality of background images with written texts and a sample image set.

[0154] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0155] Acquire an image to be tested; the image to be tested includes texts in multiple writing directions;

[0156] The image to be tested is input into the text recognition network for text recognition to obtain the recognition result; the text recognition network is trained based on sample data of visual sequence.

[0157] In one embodiment, the text recognition network includes a text recognition subnetwork and a sequence adjustment subnetwork. The image to be tested is input into the text recognition network for text recognition to obtain a recognition result. When the computer program is executed by the processor, the following steps are also implemented:

[0158] The image to be tested is input into the text recognition subnetwork for text recognition to obtain the intermediate text;

[0159] The sequence adjustment subnetwork is used to adjust the text order of the intermediate text to a logical order to obtain the recognition result.

[0160] In one embodiment, in the text recognition network training method, when the computer program is executed by a processor, the following steps are further implemented:

[0161] A sample image set is obtained; the sample image set includes sample images and label texts; the sample images include text corpora in multiple writing directions, and the text order of the label texts is the visual order;

[0162] The initial recognition network is trained according to the sample image set to obtain the text recognition network.

[0163] In one embodiment, in the method for obtaining label text, when the computer program is executed by a processor, the following steps are further implemented:

[0164] The text order of the text corpus in each writing direction is adjusted from the logical order to the visual order to obtain the label text.

[0165] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0166] Obtaining a background image set and text corpora in multiple writing directions; the background image set includes background images with different background patterns;

[0167] Expanding the sample image set according to the background image set and the text corpora of multiple writing directions to obtain an expanded sample image set;

[0168] The initial recognition network is trained according to the sample image set to obtain a text recognition network, including:

[0169] The initial recognition network is trained according to the expanded sample image set to obtain a text recognition network.

[0170] In one embodiment, the sample image set is expanded according to the background image set and the text corpora of multiple writing directions to obtain an expanded sample image set. When the computer program is executed by the processor, the following steps are also implemented:

[0171] Randomly write text corpora in multiple writing directions into each background image to generate multiple background images with written texts;

[0172] An expanded sample image set is generated according to a plurality of background images with written texts and a sample image set.

[0173] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0174] Acquire an image to be tested; the image to be tested includes texts in multiple writing directions;

[0175] The image to be tested is input into the text recognition network for text recognition to obtain the recognition result; the text recognition network is trained based on sample data of visual sequence.

[0176] In one embodiment, the text recognition network includes a text recognition subnetwork and a sequence adjustment subnetwork. The image to be tested is input into the text recognition network for text recognition to obtain a recognition result. When the computer program is executed by the processor, the following steps are also implemented:

[0177] The image to be tested is input into the text recognition subnetwork for text recognition to obtain the intermediate text;

[0178] The sequence adjustment subnetwork is used to adjust the text order of the intermediate text to a logical order to obtain the recognition result.

[0179] In one embodiment, in the text recognition network training method, when the computer program is executed by a processor, the following steps are further implemented:

[0180] A sample image set is obtained; the sample image set includes sample images and label texts; the sample images include text corpora in multiple writing directions, and the text order of the label texts is the visual order;

[0181] The initial recognition network is trained according to the sample image set to obtain the text recognition network.

[0182] In one embodiment, in the method for obtaining label text, when the computer program is executed by a processor, the following steps are further implemented:

[0183] The text order of the text corpus in each writing direction is adjusted from the logical order to the visual order to obtain the label text.

[0184] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0185] Obtaining a background image set and text corpora in multiple writing directions; the background image set includes background images with different background patterns;

[0186] Expanding the sample image set according to the background image set and the text corpora of multiple writing directions to obtain an expanded sample image set;

[0187] The initial recognition network is trained according to the sample image set to obtain a text recognition network, including:

[0188] The initial recognition network is trained according to the expanded sample image set to obtain a text recognition network.

[0189] In one embodiment, the sample image set is expanded according to the background image set and the text corpora of multiple writing directions to obtain an expanded sample image set. When the computer program is executed by the processor, the following steps are also implemented:

[0190] Randomly write text corpora in multiple writing directions into each background image to generate multiple background images with written texts;

[0191] An expanded sample image set is generated according to a plurality of background images with written texts and a sample image set.

[0192] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0193] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0194] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0195] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A text recognition method, characterized in that: The method comprises: Acquire an image to be tested; the image to be tested includes texts in multiple writing directions; The image to be tested is input into a text recognition network for text recognition to obtain a recognition result; the text recognition network is trained based on sample data of visual sequence.

2. The method according to claim 1, characterized in that The text recognition network includes a text recognition subnetwork and a sequence adjustment subnetwork. The inputting the image to be tested into the text recognition network to perform text recognition and obtain a recognition result includes: Inputting the image to be tested into the text recognition subnetwork to perform text recognition to obtain an intermediate text; The sequence adjustment subnetwork is used to adjust the text order of the intermediate text into a logical order to obtain the recognition result.

3. The method according to claim 1 or 2, characterized in that: The training method of the text recognition network comprises: Acquire a sample image set; the sample image set includes sample images and label texts; the sample images include text corpora in multiple writing directions, and the text order of the label texts is a visual order; The initial recognition network is trained according to the sample image set to obtain the text recognition network.

4. The method according to claim 3, characterized in that The method for obtaining the label text includes: The text order of the text corpus in each of the writing directions is adjusted from a logical order to a visual order to obtain the label text.

5. The method according to claim 3, characterized in that: The method further comprises: Acquire a background image set and text corpora of the plurality of writing directions; the background image set includes background images with different background patterns; Expanding the sample image set according to the background image set and the text corpora of the multiple writing directions to obtain an expanded sample image set; The step of training the initial recognition network according to the sample image set to obtain the text recognition network comprises: The initial recognition network is trained according to the expanded sample image set to obtain the text recognition network.

6. The method according to claim 5, characterized in that The step of expanding the sample image set according to the background image set and the text corpora of the multiple writing directions to obtain an expanded sample image set includes: Randomly writing the text corpora in the multiple writing directions into each of the background images to generate multiple background images with written text; The expanded sample image set is generated according to the multiple background images with written texts and the sample image set.

7. A text recognition device, characterized in that: The device comprises: An acquisition module, used for acquiring an image to be tested; the image to be tested includes texts in multiple writing directions; The text recognition module is used to input the image to be tested into a text recognition network for text recognition to obtain a recognition result; the text recognition network is trained based on sample data of visual sequence.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images