A text recognition method and device based on deep learning

Abstract

The application discloses a text recognition method and device based on deep learning, and relates to the technical field of artificial intelligence images. In the method, a text image to be recognized is acquired; the text image is input into a preset classification model to obtain an image category corresponding to the text image, the image category including a line category and a size category; the text image and the image category are input into a preset feature size selector to obtain a feature image corresponding to the text image; and the feature image is subjected to image segmentation to obtain a first text region of the feature image, so that the first text region of the feature image is recognized to obtain text corresponding to the text image. The technical scheme provided by the application improves the speed of traditional text recognition, reduces the time consumed in the text recognition process, and thus improves the detection speed of the text.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence image technology, specifically to a text recognition method and apparatus based on deep learning. Background Technology

[0002] OCR (Optical Character Recognition) is a crucial branch of computer vision, playing an irreplaceable role in the field. With the continuous development of artificial intelligence, pattern recognition, and computer vision, OCR technology is being applied to numerous scenarios such as document recognition, license plate recognition, invoice recognition, and ID card recognition. Therefore, finding better text recognition methods has become a pressing issue for the field of text recognition and the entire computer vision field.

[0003] Existing OCR text recognition methods can be broadly categorized into two types: traditional OCR text recognition methods and deep learning-based OCR text recognition methods. Traditional OCR text recognition methods primarily rely on digital image processing and traditional machine learning, resulting in performance that sometimes falls short of actual requirements. Deep learning-based OCR text recognition methods, on the other hand, utilize convolutional neural networks to automatically extract image features, and these extracted features are more robust and offer better recognition results. Therefore, most OCR text recognition methods in recent years are based on deep learning. However, most existing deep learning-based OCR text recognition methods complete the text recognition task by combining text detection and text recognition processes. A significant amount of time is consumed in the text recognition process, causing the text detection speed to be insufficient for the needs of certain practical applications.

[0004] Therefore, there is an urgent need for a deep learning-based text recognition method and device that can solve the above-mentioned technical problems. Summary of the Invention

[0005] This application provides a deep learning-based text recognition method and apparatus. This method improves the speed of traditional text recognition by reducing the time consumed in the text recognition process, thereby increasing the text detection speed.

[0006] Firstly, this application provides a deep learning-based text recognition method applied in a server. The method involves acquiring a text image to be recognized, where the text image contains text; inputting the text image into a preset classification model to obtain the image category corresponding to the text image, where the image category includes line category and size category; inputting the text image and image category into a preset feature size selector to obtain the feature image corresponding to the text image; and performing image segmentation on the feature image to obtain the first text region of the feature image, so as to facilitate the recognition of the first text region of the feature image and obtain the text corresponding to the text image.

[0007] By adopting the above technical solution, when recognizing text images, a feature image is obtained by matching the size category and line category in the text image in a preset feature size selector. The feature image is then segmented to obtain the first text region of the feature image, which facilitates text recognition and text acquisition. This improves the speed of traditional text recognition, reduces the time consumed in the text recognition process, and thus improves the text detection speed.

[0008] Optionally, multiple images to be recognized are obtained; the multiple images to be recognized are annotated using a preset text annotation tool, and the images with text are annotated. The multiple images to be recognized include images with text and images without text; the images without text are removed from the multiple images to be recognized to obtain images with text. The images with text are text images.

[0009] By adopting the above technical solution, and using a preset text annotation tool to annotate the text regions in images containing text, it is possible to effectively remove images without text from multiple images to be recognized, and only select images with text for subsequent processing. This allows for better extraction of text regions and further improves the accuracy and efficiency of text recognition.

[0010] Optionally, the row category includes single row, two rows, and multiple rows, and the size category includes first size, second size, and third size. The size category is divided according to the ratio of the area of ​​the second text region in the text image to the area of ​​the entire text image, with the first size being larger than the second size, and the second size being larger than the third size.

[0011] By adopting the above technical solution, text images are divided into different size categories and line categories, thereby better adapting to the characteristics of different text images and improving the accuracy and efficiency of text recognition.

[0012] Optionally, a preset feature size selector is constructed, which includes a mapping relationship between text images and feature images. Specifically, constructing the preset feature size selector includes: constructing a mapping relationship between preset image categories and preset image feature extraction methods; the preset image categories include a first category, a second category, and a third category; the preset image feature extraction methods include a first extraction method and a second extraction method; the preset image category is the image category corresponding to the text image to be extracted; if the preset image category is the first category, the first extraction method is used to extract the text image to be extracted; if the preset image category is the second or third category, the second extraction method is used to extract the text image to be extracted.

[0013] By adopting the above technical solution and establishing a mapping relationship between preset image categories and preset image feature extraction methods, different feature extraction methods can be selected according to the image category corresponding to the text image.

[0014] This allows for better adaptation to different image categories corresponding to text images, improving the accuracy and efficiency of text recognition. By establishing a mapping relationship between preset image categories and preset image feature extraction methods, the most suitable feature extraction method can be automatically selected during the text recognition process.

[0015] Optionally, the first category has a row category of single row or two rows and a size category of first size or second size; the second category has a row category of multiple rows and a size category of first size or second size; and the third category has a size category of the third size.

[0016] By adopting the above technical solution, different image categories are composed of different line categories and size categories. Based on the image category corresponding to different text images, the text images are processed, thereby improving the accuracy of text recognition.

[0017] Optionally, the first extraction method is to use stage4 or stage3 to extract the text image to be extracted; the second extraction method is to use stage2, stage3, stage4 and a preset feature pyramid network model to extract the text image to be extracted.

[0018] By adopting the above technical solutions, different extraction methods can extract different text images to be extracted. Different extraction methods can adapt to the text recognition needs of different scenarios and ensure the accuracy of feature extraction.

[0019] Optionally, the feature image is converted into a binary image, which includes text regions and non-text regions; the non-text regions in the binary image are removed to obtain the first text region of the feature image.

[0020] By employing the above technical solution, binarization is used to segment the feature image, removing non-text regions to obtain the text region, which is then the first text region of the feature image. This avoids scanning the entire image during recognition, improving recognition efficiency.

[0021] In a second aspect of this application, a deep learning-based text recognition device is provided. The device is a server, which includes an acquisition unit, a processing unit, and a recognition unit: the acquisition unit acquires a text image to be recognized, wherein the text image is an image containing text; the processing unit inputs the text image into a preset classification model to obtain the image category corresponding to the text image, wherein the image category includes a line category and a size category; inputs the text image and the image category into a preset feature size selector to obtain a feature image corresponding to the text image; and the recognition unit performs image segmentation on the feature image to obtain a first text region of the feature image, so as to facilitate the recognition of the first text region of the feature image and obtain the text corresponding to the text image.

[0022] Optionally, the acquisition unit is used to acquire multiple images to be recognized; the processing unit is used to annotate the multiple images to be recognized using a preset text annotation tool, annotating the images with text, the multiple images to be recognized including images with text and images without text; and to remove the images without text from the multiple images to be recognized to obtain images with text, the images with text being text images.

[0023] Optionally, the row category includes single row, two rows, and multiple rows, and the size category includes first size, second size, and third size. The size category is divided according to the ratio of the area of ​​the second text region in the text image to the area of ​​the text image, with the first size being larger than the second size, and the second size being larger than the third size.

[0024] Optionally, the preset feature size selector includes a mapping relationship between text images and feature images; the processing unit constructs a mapping relationship between preset image categories and preset image feature extraction methods, the preset image categories include a first category, a second category, and a third category, and the preset image feature extraction methods include a first extraction method and a second extraction method; the preset image category is the image category corresponding to the text image to be extracted; if the preset image category is the first category, the first extraction method is used to extract the text image to be extracted; if the preset image category is the second category or the third category, the second extraction method is used to extract the text image to be extracted.

[0025] Optionally, the processing unit further includes a category subunit, wherein the category subunit is used for the row category of the first category to be a single row or two rows, and the size category to be a first size or a second size; the row category of the second category to be multiple rows, and the size category to be a first size or a second size; and the size category of the third category to be a third size.

[0026] Optionally, the processing unit also includes an extraction subunit, wherein the first extraction method is to extract the text image to be extracted using stage4 or stage3; and the second extraction method is to extract the text image to be extracted using stage2, stage3, stage4 and a preset feature pyramid network model.

[0027] Optionally, the processing unit is used to convert the feature image into a binary image, which includes text regions and non-text regions; and to remove the non-text regions from the binary image to obtain the first text region of the feature image.

[0028] In a third aspect, this application provides an electronic device including a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory, causing the electronic device to perform any of the methods described above in this application.

[0029] In a fourth aspect, this application provides a computer-readable storage medium storing instructions that, when executed, perform any of the methods described above in this application.

[0030] In summary, one or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0031] 1. Improvements to traditional text recognition methods reduce the time consumed in the text recognition process, thereby increasing the text detection speed.

[0032] 2. It can also better extract the first text region, further improving the accuracy and efficiency of text recognition.

[0033] 3. Automatically selecting the most suitable feature extraction method not only improves the processing speed and efficiency of the algorithm, but also improves the accuracy of text recognition.

[0034] 4. It can improve the efficiency of text recognition and reduce the complexity of the algorithm while ensuring the accuracy of feature extraction, thus better adapting to the text recognition needs of different scenarios. Attached Figure Description

[0035] Figure 1 This is a schematic diagram of the first process of a deep learning-based text recognition method provided in an embodiment of this application;

[0036] Figure 2 This is a schematic diagram of the structure of a text recognition device based on deep learning provided in an embodiment of this application;

[0037] Figure 3 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application.

[0038] Explanation of reference numerals in the attached drawings: 201, acquisition unit; 202, processing unit; 203, identification unit; 300, electronic device; 301, processor; 302, communication bus; 303, user interface; 304, network interface; 305, memory. Detailed Implementation

[0039] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0040] In the description of the embodiments of this application, the words "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design that is described as "for example" or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design options. Rather, the use of the words "for example" or "for instance" is intended to present the relevant concepts in a specific manner.

[0041] In the description of the embodiments of this application, the term "multiple" means two or more. For example, multiple systems means two or more systems, and multiple screen terminals means two or more screen terminals. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0042] Optical Character Recognition (OCR) is a technology that converts text information in an image into editable text. OCR technology mainly achieves the detection and recognition of text regions through steps such as image preprocessing, feature extraction, and classification.

[0043] In recent years, with the continuous development of artificial intelligence technology, OCR technology has been widely used. Text recognition is a commonly used image-to-text conversion method in daily office work, and the quality of text detection algorithms plays a crucial role in the image-to-text conversion process. Currently, most text detection methods either use manual input or OCR text recognition methods to extract text from images. Existing OCR text recognition methods can be divided into two main categories: traditional OCR text recognition methods and deep learning-based OCR text recognition methods.

[0044] Traditional OCR text recognition methods primarily rely on digital image processing and traditional machine learning, resulting in performance that sometimes falls short of actual requirements. Deep learning-based OCR methods, however, utilize convolutional neural networks to automatically extract image features, offering stronger robustness and better recognition results. Therefore, most recent OCR text recognition methods are based on deep learning. However, most existing deep learning-based OCR methods combine text detection and recognition processes, consuming significant time in the recognition phase, which can hinder text detection speeds for certain applications.

[0045] Therefore, reducing the time consumed in text recognition to achieve a text detection speed that meets the needs of certain practical applications is a pressing problem. This application provides a deep learning-based text recognition method for fast and accurate text detection in images, applicable to servers. Figure 1 This is a schematic diagram of the first process of a deep learning-based text recognition method provided in an embodiment of this application. (Refer to...) Figure 1 The method includes the following steps S101-S104.

[0046] Step S101: Obtain the text image to be recognized.

[0047] In the above steps, before acquiring the text image to be recognized, the server obtains the image to be recognized from the electronic device. This image is a photograph taken by the user using the electronic device in different scenarios. The photographed images include those with and without text, and can also be downloaded from the internet. There is no limitation on the method of acquiring the image, and the number of images to be recognized can be determined based on the actual situation.

[0048] After the server acquires multiple images to be recognized, it first uses a preset text annotation tool to annotate the images, identifying those with text. These images include both those with and without text. The images without text are then removed, leaving only those with text, which are defined as text images. The preset text annotation tool can be Labelme, an open-source image annotation tool that allows users to annotate specific text regions in images. Annotating any part of the text region in an image facilitates the filtering out of marked images, i.e., text images. It should be noted that other tools can also be used as the preset text annotation tool; no specific limitation is made here.

[0049] For example, if the server receives 20 images to be recognized, and LabelMe is used to label these images, then images containing text are marked. Of these 20 images, 10 contain text, and the remaining 10 do not. The 10 images with text are marked, and the remaining 10 without text are removed, leaving only the 10 images with text. An image to be recognized may contain one or more text regions. By using LabelMe to annotate an image, selecting only one text region indicates that the image contains text. Obviously, LabelMe can also annotate an image to annotate multiple text regions, also indicating that the image contains text. In other words, when labeling an image, it is not necessary to select all text regions; selecting only a portion of the text regions is sufficient.

[0050] Step S102: Input the text image into the preset classification model to obtain the image category corresponding to the text image. The image category includes line category and size category.

[0051] In the above steps, the preset classification model is constructed based on the image's line category and size category. Line categories include single-line, two-line, and multi-line, and are divided according to the number of text lines in the text image. The number of text lines in an image can be identified using either CRNN or Seq2Seq. The CRNN method is based on a Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), and CTC structure, inputting the entire text line as a sequence for recognition. The Seq2Seq+Attention method, on the other hand, segments the text line into multiple words or characters, uses an encoder-decoder structure with an attention mechanism to identify each individual word or character, and finally concatenates them into a complete text line. The line category of the text image is obtained by dividing the text image according to the number of text lines using either of these two methods.

[0052] Size categories include a first size, a second size, and a third size. The size category is determined based on the ratio of the area of ​​the second text region in the text image to the area of ​​the entire text image. The area of ​​the second text region is obtained through a preset classification model. For example, the first size can be set to 75%-100%, the second size to 41%-74%, and the third size to 1%-40%. The size category of the text image is obtained according to the size classification criteria. The first size is larger than the second size, and the second size is larger than the third size.

[0053] In the example above, the server sequentially inputs multiple text images into a preset classification model. The multiple text images include a first text image. After inputting the first text image into the preset classification model, the server obtains the row category and size category of the first text image. When the row category of the first text image is single row and the size category is first size, that is, the image category of the first text image is single row and the size category is first size, the server obtains the first text image and the image category corresponding to the first text image.

[0054] Step S103: Input the text image and image category into the preset feature size selector to obtain the feature image corresponding to the text image.

[0055] In the above steps, before inputting the text image and image category into the preset feature size selector, the preset feature size selector is constructed. The preset feature size selector includes the mapping relationship between the text image and the feature image.

[0056] The process involves obtaining multiple text images, each corresponding to a preset image category. These preset image categories are the image categories corresponding to the text images to be extracted. The preset image categories are then input into a preset feature size selector to establish a mapping relationship between the preset image categories and preset image feature extraction methods. The preset image categories include a first category, a second category, and a third category. The first category has a single or two-row row category and a size category of either the first or second size. The second category has multiple rows and a size category of either the first or second size. The third category has a size category of the third size. The preset image feature extraction methods include a first extraction method and a second extraction method. The first extraction method uses either stage4 or stage3 to extract features from the text image. The second extraction method uses stage2, stage3, stage4, and a preset feature pyramid network model to extract features from the text image. The established mapping relationship is stored in the preset feature size selector for easy matching of the corresponding preset feature extraction method based on the image category of the text image.

[0057] After obtaining the image category corresponding to the text image in the preset classification model, the text image and its corresponding image category are input into a preset feature size selector to determine the preset image category corresponding to the text image. Based on the mapping relationship between the preset image category and the preset image feature extraction method, the preset image feature extraction method corresponding to the text image is determined. Then, features are extracted from the text image according to the preset image feature extraction method to obtain the feature image corresponding to the text image. The preset feature size selector uses a ResNet network, and the mapping relationship between the preset image category and the preset image feature extraction method is shown below.

[0058] The first mapping relationship: If the preset image category is Category 1, then the first extraction method is used to extract the text image to be extracted; that is, if the text image's line category is single line or two lines, and the size category is Size 1 or Size 2, then stage4 or stage3 of the preset feature size selector is used to extract the text image, obtaining the corresponding feature image. For example, when the text image's line category is single line, and the size category is Size 1 or Size 2, stage4 of the preset feature size selector is used to extract features from the text image. When the text image's line category is two lines, and the size category is Size 1 or Size 2, stage3 of the preset feature size selector is used to extract features from the text image, obtaining the corresponding feature image. When the preset image category of the text image is Category 1, either of the two preset feature size selectors in the first extraction method can be used for feature extraction; no limitation is made here.

[0059] The second mapping relationship: If the preset image category is the second or third category, the second extraction method is used to extract the text image to be extracted. When the text image has multiple lines and the size category is the first or second size, stage2, stage3, stage4, and the preset feature pyramid network model are used to extract the text image to obtain the corresponding feature image. When the text image's size category is the third size, stage2, stage3, stage4, and the preset feature pyramid network model are used to extract the text image to obtain the corresponding feature image. Here, the preset feature pyramid network model refers to FPN. FPN is a top-down feature extraction and fusion method, but FPN is a multi-scale object detection algorithm, meaning it has more than one feature prediction layer.

[0060] The initial stage of the text image is in stage 1. The text image is then sequentially input into stages 2, 3, and 4 for feature extraction, and the results of each feature extraction are fused using FPN (Feature Process Network). Specifically: after passing through stage 1, the text image is input into stage 2 to obtain the first feature-extracted image; then the first feature-extracted image is input into stage 3 to obtain the second feature-extracted image; then the second feature-extracted image is input into stage 4 to obtain the third feature-extracted image; finally, the first, second, and third feature-extracted images are fused using FPN to obtain the feature image corresponding to the text image.

[0061] In the preset feature size selector, there are three feature maps of different sizes: stage2, stage3, and stage4. Each of the three sizes corresponds to a different image resolution. The feature maps of different resolutions have different strengths in feature extraction. FPN uses feature maps of different resolutions to perceive objects of different sizes, and through continuous upsampling and cross-layer fusion mechanisms, the output features have both low-level visual information and high-level semantic information.

[0062] Step S104: Perform image segmentation on the feature image to obtain the first text region of the feature image, so as to facilitate the recognition of the first text region of the feature image and obtain the text corresponding to the text image.

[0063] In the above steps, after obtaining the feature image, image segmentation is performed on the feature image to convert the color feature image into a binary image. The binary image includes text regions and non-text regions. Regions with text in the feature image are set as text regions, and regions without text are set as non-text regions. Text regions are set to 1, and non-text regions are set to 0. The non-text regions in the binary image are then removed to obtain the first text region of the feature image.

[0064] Furthermore, removing non-text regions from the binary image involves applying an erosion operation using the SXS kernel. This erosion operation removes edge regions, which in this case refer to non-text regions in the binary image, resulting in multiple first text regions. The reason for these multiple first text regions is that the location and number of text regions differ across different feature images, leading to variations in the number of first text regions obtained.

[0065] Erosion is a common operation in image morphology processing. Its function is to make the edges of the target object smoother and more continuous, and to remove unwanted non-text pixels from the image edges.

[0066] SXS is a storage medium primarily used in cameras and devices for high-definition video and audio acquisition, storage, and transmission. A kernel, as a fundamental concept in image processing, is mainly used for morphological processing. It can perform operations such as erosion, dilation, opening, and closing on binary images to remove noise, connect broken parts, and smooth edges. A kernel is typically a small matrix, and various shapes and sizes can be used, such as square, circular, and cross-shaped kernels. The specific kernel selection should be based on the actual situation.

[0067] After performing an erosion operation on the binary image, multiple first text regions are obtained. Since these first text regions may contain numerous edge areas, a second erosion operation is performed to obtain a second-eroded text region image. Before performing text recognition on the text region image, because the size of the text region image changes after the second erosion operation, the SXS kernel is used to dilate the text region image to restore its size to the size of the feature image. Then, the text region image is recognized to obtain the corresponding text.

[0068] Traditional text recognition methods, after selecting a text region, do not classify the text region's category. They directly input the text region into the feature extraction network for feature extraction, thus regardless of the size of the text region's category, features are extracted sequentially according to the feature maps of the feature extraction network. The text recognition method of this application classifies the text image to be recognized, obtaining the corresponding image category. Different feature extraction methods are used for different image categories, improving the detection speed of text recognition and further enhancing the user experience.

[0069] This application also provides a text recognition device based on deep learning. Figure 2 This is a schematic diagram of the structure of a deep learning-based text recognition device provided in an embodiment of this application. (Refer to...) Figure 2 The server includes an acquisition unit 201, a processing unit 202, and an identification unit 203.

[0070] Acquisition unit 201 acquires the text image to be recognized, which is an image containing text.

[0071] The processing unit 202 inputs the text image into a preset classification model to obtain the image category corresponding to the text image. The image category includes line category and size category. The text image and image category are input into a preset feature size selector to obtain the feature image corresponding to the text image.

[0072] The recognition unit 203 performs image segmentation on the feature image to obtain the first text region of the feature image, so as to recognize the first text region of the feature image and obtain the text corresponding to the text image.

[0073] In one possible implementation, the acquisition unit 201 is used to acquire multiple images to be recognized; the processing unit 202 is used to annotate the multiple images to be recognized using a preset text annotation tool, annotating the images with text, the multiple images to be recognized including images with text and images without text; and to remove the images without text from the multiple images to be recognized to obtain images with text, the images with text being text images.

[0074] In one possible implementation, the line category includes single line, two lines, and multiple lines, and the size category includes a first size, a second size, and a third size. The size category is divided according to the ratio of the area of ​​the second text region in the text image to the area of ​​the text image, with the first size being larger than the second size and the second size being larger than the third size.

[0075] In one possible implementation, the preset feature size selector includes a mapping relationship between text images and feature images; the processing unit 202 is used to construct a mapping relationship between preset image categories and preset image feature extraction methods, wherein the preset image categories include a first category, a second category, and a third category, and the preset image feature extraction methods include a first extraction method and a second extraction method; the preset image category is the image category corresponding to the text image to be extracted; if the preset image category is the first category, the first extraction method is used to extract the text image to be extracted; if the preset image category is the second category or the third category, the second extraction method is used to extract the text image to be extracted.

[0076] In one possible implementation, the processing unit 202 further includes a category subunit, wherein the category subunit is used for the row category of the first category to be single row or two rows and the size category to be a first size or a second size; the row category of the second category to be multiple rows and the size category to be a first size or a second size; and the size category of the third category to be a third size.

[0077] In one possible implementation, the processing unit 202 further includes an extraction subunit, which is used for a first extraction method of extracting the text image to be extracted using stage4 or stage3; and a second extraction method of extracting the text image to be extracted using stage2, stage3, stage4 and a preset feature pyramid network model.

[0078] In one possible implementation, the processing unit 202 is used to convert the feature image into a binary image; the binary image includes text regions and non-text regions; the non-text regions in the binary image are removed to obtain the first text region of the feature image.

[0079] It should be noted that the above embodiments of the apparatus are only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0080] This application also discloses an electronic device. (See reference...) Figure 3, Figure 3 This application provides a schematic diagram of the structure of an electronic device. The electronic device 300 may include: at least one processor 301, at least one network interface 304, a user interface 303, a memory 305, and at least one communication bus 302.

[0081] The communication bus 302 is used to enable communication between these components.

[0082] The user interface 303 may include a display screen and a camera. Optionally, the user interface 303 may also include a standard wired interface and a wireless interface.

[0083] The network interface 304 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0084] The processor 301 may include one or more processing cores. The processor 301 connects to various parts of the server using various interfaces and lines, and performs various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 305, and by calling data stored in the memory 305. Optionally, the processor 301 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 301 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and application requests; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 301 and may be implemented as a separate chip.

[0085] The memory 305 may include random access memory (RAM) or read-only memory. Optionally, the memory 305 may include a non-transitory computer-readable storage medium. The memory 305 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 605 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), instructions for implementing the various method embodiments described above, etc.; the data storage area may store data involved in the various method embodiments described above, etc. Optionally, the memory 305 may also be at least one storage device located remotely from the aforementioned processor 301.

[0086] like Figure 3 As shown, the memory 305, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a text recognition application based on deep learning.

[0087] exist Figure 3 In the electronic device 300 shown, the user interface 303 is mainly used to provide an input interface for the user and to obtain the user input data; while the processor 301 can be used to call the deep learning-based text recognition application stored in the memory 305. When executed by one or more processors, the electronic device performs one or more of the methods described in the above embodiments.

[0088] An electronic device readable storage medium stores instructions that, when executed by one or more processors, cause the electronic device to perform one or more of the methods described in the above embodiments.

[0089] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0090] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0091] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some service interfaces; indirect couplings or communication connections between devices or units may be electrical or other forms.

[0092] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0093] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0094] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0095] The above description is merely an exemplary embodiment of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Those skilled in the art will readily conceive of other embodiments of this disclosure upon considering the specification and the disclosure of practical truths. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described in this disclosure.

Claims

1. A text recognition method based on deep learning, characterized in that, When applied to a server, the method includes: Obtain the text image to be recognized, wherein the text image is an image containing text; The text image is input into a preset classification model to obtain the image category corresponding to the text image. The image category includes a line category and a size category. The line category includes single line, two lines, and multiple lines. The size category includes a first size, a second size, and a third size. The size category is divided according to the ratio of the area of ​​the second text region in the text image to the area of ​​the text image. The first size is larger than the second size, and the second size is larger than the third size. The text image and the image category are input into a preset feature size selector to obtain a feature image corresponding to the text image. Before inputting the text image and the image category into the preset feature size selector to obtain the feature image corresponding to the text image, the preset feature size selector is constructed, which includes a mapping relationship between the text image and the feature image. Constructing the preset feature size selector specifically includes: constructing a mapping relationship between a preset image category and a preset image feature extraction method, wherein the preset image category includes a first category, a second category, and a third category, and the preset image feature extraction method includes a first... Extraction method and second extraction method; the preset image category is the image category corresponding to the text image to be extracted; if the preset image category is the first category, then the first extraction method is used to extract the text image to be extracted; if the preset image category is the second category or the third category, then the second extraction method is used to extract the text image to be extracted; the row category of the first category is a single row or two rows, and the size category is the first size or the second size; the row category of the second category is multiple rows, and the size category is the first size or the second size; the size category of the third category is the third size; The feature image is segmented to obtain a first text region of the feature image, so as to facilitate the recognition of the first text region of the feature image and obtain the text corresponding to the text image.

2. The method according to claim 1, characterized in that, Before acquiring the text image to be recognized, the method further includes: Acquire multiple images to be recognized; The multiple images to be identified are annotated using a preset text annotation tool, and the images with text are annotated. The multiple images to be identified include the images with text and the images without text. Remove the images without text from the multiple images to be identified to obtain the images with text, which are the text images.

3. The method according to claim 1, characterized in that, The first extraction method is to extract the text image to be extracted using stage4 or stage3; the second extraction method is to extract the text image to be extracted using stage2, stage3, stage4 and a preset feature pyramid network model.

4. The method according to claim 1, characterized in that, The step of segmenting the feature image to obtain the first text region of the feature image specifically includes: The feature image is converted into a binary image, which includes text regions and non-text regions. The non-text regions in the binary image are removed to obtain the first text region of the feature image.

5. A text recognition device based on deep learning, characterized in that, The device is a server, which includes an acquisition unit (201), a processing unit (202), and an identification unit (203): The acquisition unit (201) acquires a text image to be recognized, wherein the text image is an image containing text; the processing unit (202) inputs the text image into a preset classification model to obtain an image category corresponding to the text image, wherein the image category includes a line category and a size category, wherein the line category includes single line, two lines, and multiple lines, and the size category includes a first size, a second size, and a third size, wherein the size category is divided according to the ratio of the area of ​​the second text region in the text image to the area of ​​the text image, wherein the first size is larger than the second size, and the second size is larger than the third size; the text image and the image category are input into a preset feature size selector to obtain a feature image corresponding to the text image, wherein the preset feature size selector is constructed before inputting the text image and the image category into the preset feature size selector to obtain the feature image corresponding to the text image, wherein the preset feature size selector includes the text image and the image category. The mapping relationship between this image and the feature image; constructing the preset feature size selector specifically includes: constructing a mapping relationship between preset image categories and preset image feature extraction methods, wherein the preset image categories include a first category, a second category, and a third category, and the preset image feature extraction methods include a first extraction method and a second extraction method; the preset image category is the image category corresponding to the text image to be extracted; if the preset image category is the first category, then the first extraction method is used to extract the text image to be extracted; if the preset image category is the second category or the third category, then the second extraction method is used to extract the text image to be extracted; the row category of the first category is a single row or two rows, and the size category is the first size or the second size; the row category of the second category is multiple rows, and the size category is the first size or the second size; the size category of the third category is the third size; The recognition unit (203) performs image segmentation on the feature image to obtain a first text region of the feature image, so as to recognize the first text region of the feature image and obtain the text corresponding to the text image.

6. An electronic device, characterized in that, The device includes a processor (301), a memory (305), a user interface (303), and a network interface (304). The memory (305) is used to store instructions. The user interface (303) and the network interface (304) are used to communicate with other devices. The processor (301) is used to execute the instructions stored in the memory (305) to cause the electronic device (300) to perform the method as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed, perform the method as described in any one of claims 1-4.

Images