Image text recognition method, device and storage medium

Abstract

The application discloses an image character recognition method and device and a storage medium, relates to the technical field of image recognition, and comprises the following steps: acquiring an image to be processed; decoding the image to be processed by using an nvjpeg library to obtain a decoded image; performing character recognition on the decoded image by using a quantized recognition model to obtain character information; and completing the decoding and character recognition of the image by using an image processor, wherein data is only transmitted between a display memory and the display memory, which is equivalent to the prior art, so that the data transmission time is shortened, the operation speed of character recognition is accelerated, the technical problem of low OCR character recognition efficiency is solved, and efficient image character recognition is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, and in particular to an image character recognition method, device and storage medium. BACKGROUND

[0002] OCR (optical character recognition) character recognition refers to a process in which an electronic device (such as a scanner or digital camera) inspects printed characters on paper and then translates the shapes into computer text using character recognition methods; that is, a process of scanning text materials and then analyzing and processing image files to obtain text and layout information.

[0003] At present, OCR character recognition usually uses a CPU (central processing unit) to decode and / or pre-process an image, and then uses a GPU (graphics processing unit) to calculate the decoded and / or pre-processed image using a neural network model to obtain text information, which has the problem of low image character recognition efficiency. SUMMARY

[0004] The main purpose of the present application is to provide an image character recognition method, device and storage medium, which aims to solve the technical problem of low OCR character recognition efficiency in the prior art.

[0005] To achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0006] In a first aspect, the present application provides an image character recognition method applied to an image processor, the method comprising: obtaining a to-be-processed image, decoding the to-be-processed image using an nvjpeg library to obtain a decoded image; and performing character recognition on the decoded image using a quantized recognition model to obtain text information.

[0007] Optionally, in the image character recognition method, the step of performing character recognition on the decoded image using the quantized recognition model to obtain text information comprises:

[0008] obtaining a rotation angle of the decoded image using a quantized image direction classification model, performing angle correction on the decoded image according to the rotation angle; and performing character recognition on the angle-corrected decoded image using a quantized character recognition model to obtain text information.

[0009] Optionally, in the image character recognition method, after the step of obtaining a rotation angle of the decoded image using a quantized image direction classification model and performing angle correction on the decoded image according to the rotation angle, the method further comprises:

[0010] obtaining a to-be-recognized image by using the quantized position detection model to obtain the text position of the decoded image and cutting the decoded image according to the text position;

[0011] The step of performing text recognition on the angle-corrected decoded image by using the quantized text recognition model to obtain text information comprises:

[0012] performing text recognition on the to-be-recognized image by using the quantized text recognition model to obtain text information.

[0013] Optionally, before the step of performing text recognition on the decoded image by using the quantized recognition model to obtain text information in the image text recognition method, the method further comprises:

[0014] performing preprocessing on the decoded image by using an NPP library to obtain a standard image, wherein the preprocessing comprises scaling processing and normalization processing;

[0015] The step of performing text recognition on the decoded image by using the quantized recognition model to obtain text information comprises:

[0016] performing text recognition on the standard image by using the quantized recognition model to obtain text information.

[0017] Optionally, the image text recognition method further comprises:

[0018] converting the recognition model into a target format and performing int8 quantization on the recognition model by using a deep learning inference optimizer.

[0019] Optionally, the image text recognition method further comprises:

[0020] creating a GPU memory space, wherein the GPU memory space is used to store a first image and / or a second image, the first image is a processed image, and the second image is a to-be-processed image;

[0021] when the GPU memory space receives the second image and the first image is stored in the GPU memory space, covering the first image with the second image.

[0022] In a second aspect, the present application provides an image text recognition device, which comprises a central processor, an image processor and a memory, and the memory stores a recognition program, and the recognition program is executed by the processor to implement the image text recognition method as described above.

[0023] In a third aspect, the present application provides a computer readable storage medium, wherein the storage medium stores a computer program, and the computer program, when executed by one or more image processors, implements the image text recognition method as described above.

[0024] The one or more technical solutions provided by the present application can have the following advantages or at least achieve the following technical effects:

[0025] The image text recognition method, device and storage medium provided by the present application obtain a to-be-processed image through an image processor, decode the to-be-processed image by using an nvjpeg library to obtain a decoded image, and decode the to-be-processed image by using a high-performance GPU acceleration library of the image processor, so that the decoding efficiency is high. The decoded image is subjected to text recognition by using a quantized recognition model to obtain text information. The decoding and text recognition of the image are completed by using the image processor, data is only transmitted between the video memory and the video memory, and compared with the prior art, the data transmission time is shortened, the operation speed of text recognition is accelerated, the technical problem of low OCR text recognition efficiency is solved, and efficient image text recognition is achieved. BRIEF DESCRIPTION OF DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to the provided drawings without creative labor.

[0027] Figure 1 The flowchart of the first embodiment of the image text recognition method of the present application is shown in the figure.

[0028] Figure 2 The hardware structure diagram of the third subject matter related to the present application is shown in the figure. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor belong to the scope of protection of the present application.

[0030] It should be noted that in the embodiments of the present application, in the present application, the terms "comprising", "containing" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such a process, method, article or system. Without more limitations, the elements defined by the statement "comprising" do not exclude the presence of other identical elements in the process, method, article or system including the element. In addition, the meaning of "and / or" appearing throughout the text includes three parallel schemes, for example, "A and / or B" includes A scheme, or B scheme, or A and B scheme.

[0031] In the present application, if there is a description related to "first", "second" and the like, the description of "first", "second" and the like is only for the purpose of description, and cannot be understood as indicating or implying the relative importance of the indicated technical features or implicitly indicating the number of indicated technical features. Therefore, the features limited by "first", "second" can explicitly or implicitly include at least one of the features.

[0032] For those of ordinary skill in the art, the specific meaning of the above terms in the present application can be understood according to specific circumstances. In addition, the technical solutions of various embodiments can be combined with each other, but it is based on the fact that a person of ordinary skill in the art can realize it, when the combination of technical solutions appears contradictory or unachievable, it should be considered that the combination of technical solutions does not exist, nor within the scope of protection required by the present application.

[0033] Embodiment one

[0034] Referring to Figure 1 The flowchart, the first embodiment of the image text recognition method of the present application is proposed, which is applied to an image processor and can include the following steps:

[0035] Step S100: obtaining a to-be-processed image, decoding the to-be-processed image by using an nvjpeg library to obtain a decoded image.

[0036] Specifically, the to-be-processed image is pre-stored in the memory, present or database, and when it is necessary to process the picture, it is sent to the image processor; the nvjpeg library is a high-performance GPU acceleration library, which is used for decoding, encoding and transcoding JPEG format images, and can provide higher throughput and lower delay compared with decoding by CPU only.

[0037] Step S400: performing text recognition on the decoded image by using the quantized recognition model to obtain text information.

[0038] Specifically, model quantization is to convert a floating-point model to a fixed-point model by a certain method, and to establish a mapping relationship between floating-point data and fixed-point data, so that a greater benefit is obtained at the cost of a smaller precision loss. During the training of a convolutional neural network, the data precision of 32-bit single-precision floating-point precision (full precise, abbreviated as fp32) is usually used to represent network weights, biases, activation values and other information. Therefore, when deploying a trained deep network model on an embedded platform with relatively limited computing power and storage space, low-precision quantization operation is needed, that is, by reducing the data precision, such as using 8-bit, 6-bit or even more extreme 2-bit or 3-bit to replace float data (floating-point data type), the demand of the model on storage and computing resources is reduced under the premise of ensuring the network precision, and the network inference delay is reduced. In addition, the recognition model can be one or more models, for example, only a text recognition model is used to recognize the decoded image to obtain text information, and for example, a combined model including an image direction classification model, a position detection model and a text recognition model.

[0039] The image text recognition method provided by the embodiment has the advantages that the image processor is used to obtain a to-be-processed image, the to-be-processed image is decoded by using an nvjpeg library to obtain a decoded image, the to-be-processed image is decoded by using the high-performance GPU acceleration library of the image processor, and the decoding efficiency is high; the decoded image is subjected to text recognition by using the quantized recognition model to obtain text information, the decoding and text recognition of the image are completed by using the image processor, data is only transmitted between the video memory and the video memory, and compared with the prior art, the data transmission time is shortened, the operation speed of text recognition is accelerated, the technical problem of low OCR text recognition efficiency is solved, and efficient image text recognition is implemented.

[0040] Embodiment two

[0041] Based on the same inventive concept, a second embodiment of the image text recognition method of the present application is provided. The image text recognition method of the embodiment can include the following steps:

[0042] Step S100: obtaining a to-be-processed image, decoding the to-be-processed image by using an nvjpeg library to obtain a decoded image.

[0043] Step S200: preprocessing the decoded image by using an NPP library to obtain a standard image, wherein the preprocessing includes scaling processing and normalization processing.

[0044] Specifically, image normalization refers to a process of transforming an image into a fixed standard form through a series of standard processing transformations. The original image can obtain various copy images after experiencing some processing or attacks, and the images can obtain the same standard image after the same parameter image normalization processing. The NPP library is a series of library functions for accelerating processes. The original target of NPP is for image and video processes, but now it is applied to many other fields by developers, including signal processing. By preprocessing the decoded picture, the efficiency and accuracy of subsequent text recognition are further improved.

[0045] Step S300: converting the recognition model into a target format and performing int8 quantization on the recognition model by using a deep learning inference optimizer.

[0046] Specifically, the F32 data is mapped into int8 data, and the model volume after quantization is smaller, the download speed is faster, and the runtime memory and cache are smaller.

[0047] Step S400: performing text recognition on the decoded image by using the quantized recognition model to obtain text information.

[0048] Step S500: creating a video memory space, the video memory space being used to store a first image and / or a second image, wherein the first image is a processed image, and the second image is an unprocessed image; when the video memory space receives the second image and the video memory space stores the first image, covering the first image with the second image.

[0049] Specifically, by creating a video memory space and covering a processed image with an unprocessed image, compared with creating a video memory when receiving an unprocessed image and destroying the video memory after recognition in the prior art, the time consumption of this part of action is reduced, the data copying and moving are reduced as much as possible, and the image text recognition efficiency is further improved.

[0050] Further, step S400 can include:

[0051] Step S401: obtaining a rotation angle of the decoded image by using the quantized image direction classification model, and correcting the decoded image according to the rotation angle.

[0052] Step S402: performing text recognition on the angle-corrected decoded image by using the quantized text recognition model to obtain text information.

[0053] Step S403: obtaining the text position of the decoding image by using the quantized position detection model, and cutting the decoding image according to the text position to obtain a to-be-recognized image

[0054] The image text recognition method provided in the embodiment has the advantages that the to-be-processed image is obtained by using the image processor, the to-be-processed image is decoded by using the nvjpeg library, the to-be-processed image is decoded by using the high-performance GPU acceleration library of the image processor, and the decoding efficiency is high; the decoding image is subjected to text recognition by using the quantized recognition model to obtain text information, the decoding of the image and the text recognition are completed by using the image processor, data is only transmitted between the video memory and the video memory, and compared with the prior art, the data transmission time is shortened, the operation speed of the text recognition is accelerated, the technical problem of low OCR text recognition efficiency is solved, efficient image text recognition is achieved, the F32 data is mapped into int8 data, the model volume after quantization is smaller, the download speed is faster, the runtime memory and cache are smaller, the efficiency and accuracy of subsequent text recognition are further improved by preprocessing the decoding image, the memory space is created, and the unprocessed image is used to cover the processed image, compared with the prior art of creating the memory when receiving the to-be-processed image and destroying the memory after the recognition is completed, the time consumption of the part of actions is reduced, the data copying and moving are reduced as much as possible, and the image text recognition efficiency is further improved.

[0055] Embodiment three

[0056] Based on the same inventive concept, the embodiment provides an image text recognition device, which can include a central processor, an image processor and a memory, and the memory stores an identification program, which, when executed by the processor, implements all or part of the steps of the image text recognition method of the embodiments.

[0057] Specifically, the device can be a terminal device such as a mobile phone, a computer, a tablet computer, a portable computer, etc.

[0058] The memory is used to store various types of data, which can include, for example, instructions of any application program or method in the device, and application-related data. The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), random access memory (RAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. Optionally, the memory can also be a storage device independent of the processor.

[0059] Embodiment Four

[0060] Based on the same inventive concept, the embodiments provide a computer readable storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory (for example, an SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic memory, a magnetic disk, an optical disk, a server, etc., in which a computer program is stored. The computer program can be executed by one or more image processors, and when the computer program is executed by the image processors, all or part of the steps of the embodiments of the image character recognition method of the present application can be implemented.

[0061] It should be noted that the above-mentioned embodiment numbers of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments. The above embodiments are only optional embodiments of the present application, and do not limit the patent scope of the present application. Any equivalent structure or equivalent flow conversion made by using the contents of the present application specification and drawings, or directly or indirectly applied to other related technical fields under the inventive concept of the present application, are included in the patent protection scope of the present application.

Claims

1. An image text recognition method, characterized in that, Applied to an image processor, the method includes: The image to be processed is obtained, and the image to be processed is decoded using the nvjpeg library to obtain the decoded image; The decoded image is then subjected to text recognition using the quantized recognition model to obtain text information. Before the step of performing text recognition on the decoded image using the quantized recognition model to obtain text information, the method further includes: The decoded image is preprocessed using the NPP library to obtain a standard image, wherein the preprocessing includes scaling and normalization. The step of using the quantized recognition model to perform text recognition on the decoded image to obtain text information includes: The quantized recognition model is used to perform text recognition on the standard image to obtain text information.

2. The image text recognition method as described in claim 1, characterized in that, The step of using the quantized recognition model to perform text recognition on the decoded image to obtain text information includes: The rotation angle of the decoded image is obtained using the quantized image orientation classification model, and the angle of the decoded image is corrected based on the rotation angle. The quantized character recognition model is used to perform character recognition on the angle-corrected decoded image in order to obtain text information.

3. The image text recognition method as described in claim 2, characterized in that, After the steps of obtaining the rotation angle of the decoded image using the quantized image orientation classification model and performing angle correction on the decoded image based on the rotation angle, the method further includes: The text positions in the decoded image are obtained using a quantized position detection model, and the decoded image is cropped based on the text positions to obtain the image to be recognized. The step of using the quantized character recognition model to perform character recognition on the angle-corrected decoded image to obtain text information includes: The quantized character recognition model is used to perform character recognition on the image to be recognized in order to obtain text information.

4. The image text recognition method as described in any one of claims 1 to 3, characterized in that, The method further includes: The recognition model is converted into the target format, and the recognition model is quantized using an int8 quantization function with a deep learning inference optimizer.

5. The image text recognition method as described in claim 1, characterized in that, The method further includes: Create a video memory space for storing a first image and / or a second image, wherein the first image is a processed image and the second image is an image to be processed; When the video memory space receives the second image, and the video memory space stores the first image, the second image is used to cover the first image.

6. An image text recognition device, characterized in that, The image text recognition device includes a central processing unit, an image processor, and a memory. The memory stores a recognition program, and when the recognition program is executed by the processor, it implements the image text recognition method as described in any one of claims 1 to 5.

7. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by one or more image processors, implements the image text recognition method as described in any one of claims 1 to 5.

Images