Text recognition method, system and storage medium

Abstract

The present application discloses a text recognition method, system and storage medium, which relate to text recognition technology. The specific scheme includes: the control end sends the acquired image to be recognized to the computing device of the cloud service end; the image to be recognized contains at least one text information to be recognized; the control end obtains at least one area to be recognized in the image to be recognized determined by the computing device and containing text information; the control end determines the corresponding sub-image according to each area to be recognized, and sends each sub-image to the computing device; the sub-image is the image to be recognized contained in the area to be recognized; the control end obtains the text information determined by the computing device from each sub-image. The present application can reduce the computing resource cost and time cost required for text information recognition on the basis of meeting the recognition accuracy requirements for the text information in the image to be recognized.

Description

Technical Field

[0001] The present application relates to text recognition technology, and in particular to a text recognition method, system and storage medium. Background Art

[0002] With the development of Internet technology, text recognition technology for recognizing text information from images has an increasingly wide range of applications and can be widely used to recognize text information in document images, license plates, advertisements, menus and other images.

[0003] In the prior art, layout analysis or deep learning methods can usually be used to recognize text in images. However, the layout analysis method requires that the layout structure of the image to be recognized has strong regularity, and the foreground part containing text information and the background part containing other image information in the image are highly distinguishable. It cannot adapt to the images to be recognized with complex foreground and background parts and random layout structures. The deep learning method performs image processing, and the amount of calculation is very large when processing complex images to be recognized. No matter which method in the prior art is used for text recognition, it is difficult to meet the recognition needs in actual use, especially when the fonts are rich in variations and the background part interferes greatly with the foreground part. It is either difficult to meet the recognition accuracy requirements, or the computing resource cost and time cost consumed are too large to bear. Summary of the invention

[0004] In view of this, the main purpose of the present application is to provide a text recognition method, which can overcome the defects of the prior art and reduce the computing resource cost and time cost required for text information recognition while meeting the recognition accuracy requirements for text information in the image to be recognized.

[0005] In order to achieve the above objectives, the technical solution proposed in this application is:

[0006] In a first aspect, an embodiment of the present application provides a text recognition method, comprising the following steps:

[0007] The control end sends the acquired image to be recognized to the computing device of the cloud service end; the image to be recognized contains at least one text information to be recognized;

[0008] The control end obtains at least one to-be-recognized area containing the text information in the to-be-recognized image determined by the computing device;

[0009] The control end determines a corresponding sub-image according to each of the to-be-recognized areas, and sends each of the sub-images to the computing device; the sub-image is a portion of the to-be-recognized image contained in the to-be-recognized area.

[0010] The control end obtains the text information determined by the computing device from each of the sub-images.

[0011] In a possible implementation, the method further includes the computing device of the cloud service end scheduling by using the following steps:

[0012] At every preset time interval, the control end obtains each computing device provided by the cloud service end to determine the total load of the area to be identified and the text information;

[0013] When the total load is greater than a preset first load threshold, the control end applies to the cloud service end for allocating a predetermined number of the computing devices;

[0014] When the total load is less than a preset second load threshold, the control end applies to the cloud service end for releasing a predetermined number of the computing devices.

[0015] In a possible implementation manner, the total load is the total time taken by each computing device provided by the cloud server to determine the area to be identified and the text information.

[0016] In a possible implementation, the method further includes the computing device of the cloud service end scheduling by using the following steps:

[0017] When the control end obtains the image to be recognized, it applies to the cloud service end for allocation of at least one of the computing devices, and sends the obtained image to be recognized and each of the sub-images to the allocated computing device;

[0018] When the control end obtains the text information determined by the computing device, it applies to the cloud service end for releasing the at least one computing device.

[0019] In a possible implementation manner, the area to be identified is a parallelogram area;

[0020] The step of the control end determining the corresponding sub-image according to each of the to-be-identified areas comprises:

[0021] For each of the to-be-recognized areas, determining an intermediate image corresponding to the to-be-recognized area; the intermediate image is a parallelogram;

[0022] The intermediate image is rotated and stretched to obtain the sub-image; the sub-image is a rectangle.

[0023] In a possible implementation, the computing device includes a detection computing device and an identification computing device;

[0024] The step of sending the acquired image to be recognized to the computing device of the cloud service end is as follows:

[0025] The control end sends the acquired image to be identified to the detection computing device;

[0026] The step of sending each of the sub-images to the computing device is:

[0027] Each of the sub-images is sent in parallel to the recognition computing device.

[0028] In a second aspect, the embodiment of the present application further provides a text recognition method, comprising:

[0029] When the computing device of the cloud service end receives the image to be recognized sent by the control end, at least one area to be recognized containing text information is determined from the image to be recognized, and the area to be recognized is sent to the control end;

[0030] When the computing device receives the sub-images corresponding to each of the to-be-recognized areas, the computing device determines the text information contained in each of the sub-images, and sends the determined text information to the control end.

[0031] In a possible implementation, the computing device of the cloud service end is allocated using the following steps:

[0032] When receiving the application for allocating the computing device from the control end, the cloud service end selects a computing device that meets the preset rule from each of the computing devices owned by the cloud service end as a candidate computing device according to the preset rule;

[0033] Scoring each candidate computing device according to the computing resources of each candidate computing device;

[0034] At least one candidate computing device with the highest score among each of the candidate computing devices is determined as a target computing device, and the target computing device is allocated to the control end.

[0035] In a third aspect, an embodiment of the present application further provides a text recognition system, including: a control terminal and a cloud service terminal;

[0036] The cloud service end includes at least one computing device; the computing device is used to, when receiving an image to be recognized sent by the control end, determine at least one area to be recognized containing text information from the image to be recognized, and send the area to be recognized to the control end; and is also used to, when receiving a sub-image corresponding to each of the areas to be recognized, respectively determine the text information contained in each of the sub-images, and send the respectively determined text information to the control end;

[0037] The control end includes an identification device; the identification device is connected to each of the computing devices respectively; the identification device is used to send the image to be identified and each of the sub-images to the computing device of the cloud service end, and receive the area to be identified and the text information determined by the computing device; and is also used to determine the corresponding sub-image according to each area to be identified.

[0038] In a possible implementation manner, the cloud service end further includes: a main computing device;

[0039] The main computing device is connected to each of the computing devices; the main computing device is used to schedule the computing devices;

[0040] The control end further comprises: a resource scheduler; the resource scheduler is connected to the identification device and the main computing device respectively; the resource scheduler is used to apply to the main computing device for allocating or releasing the computing device.

[0041] In a possible implementation manner, the control end further includes: a service gateway; the service gateway is connected to the recognition device; and the service gateway is used to obtain the image to be recognized.

[0042] In a possible implementation, the computing device includes a detection computing device and an identification computing device;

[0043] The detection computing device and the identification computing device are connected to the main computing device and the identification device respectively;

[0044] The detection and calculation device is used to receive the image to be identified; and is also used to determine at least one area to be identified containing text information from the image to be identified;

[0045] The recognition and computing device is used to receive the sub-image; and is also used to determine the text information contained in the sub-image.

[0046] In a possible implementation, the detection computing device includes a graphics processor.

[0047] In a possible implementation manner, the identification and computing device includes a central processing unit.

[0048] In a fourth aspect, the present application also provides a computer-readable storage medium. The specific solution is:

[0049] A computer-readable storage medium stores computer instructions, which, when executed by a processor, can implement the above-mentioned first aspect and any possible implementation method of the first aspect, or implement the above-mentioned second aspect and the steps of any possible implementation method of the second aspect.

[0050] In a fifth aspect, the present application also provides an electronic device. The specific solution is:

[0051] An electronic device includes the above-mentioned computer-readable storage medium and a processor capable of executing the computer-readable storage medium.

[0052] In summary, the text recognition method proposed in the present application utilizes the computing device of the cloud service end to recognize the text information in the image to be recognized. First, the acquired image to be recognized is sent to the computing device, and the computing device determines at least one area to be recognized containing text information in the image to be recognized, so as to first frame the area to be recognized containing text information when the layout structure is arbitrary and the foreground and background parts of the image are complex. Then, the corresponding sub-image is determined according to the area to be recognized, and each sub-image is an image to be recognized contained in the area to be recognized. By dividing the entire image to be recognized containing a large amount of image information into sub-images containing less image information, the amount of calculation can be greatly reduced, and the pertinence of text recognition can be improved, thereby improving the accuracy of text recognition, so that the obtained text information can meet the requirements of recognition accuracy, and at the same time, the computing resource cost and time cost are greatly reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0054] Figure 1 A flowchart of a text recognition method provided in an embodiment of the present application;

[0055] Figure 2 A data flow diagram of a text recognition method provided in an embodiment of the present application;

[0056] Figure 3 A schematic diagram of a flow chart of scheduling of computing devices in a text recognition method provided in an embodiment of the present application;

[0057] Figure 4 A schematic diagram of determining a sub-image according to a region to be recognized in a method for character recognition provided in an embodiment of the present application;

[0058] Figure 5 A flowchart of another text recognition method provided in an embodiment of the present application;

[0059] Figure 6 A schematic diagram of a flow chart of scheduling a computing device in another text recognition method provided in an embodiment of the present application;

[0060] Figure 7 A schematic diagram of the structure of a text recognition system provided in an embodiment of the present application;

[0061] Figure 8 A flowchart of another text recognition method provided in an embodiment of the present application;

[0062] Fig. 9 A schematic diagram of a process for determining text information from an image to be recognized;

[0063] Fig.10 assigning schematics to computing devices;

[0064] Fig.11 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0065] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0066] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can, for example, be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0067] The recognition of text information in an image usually takes the text information as the foreground part of the image and other image information as the background part, and recognizes the text information from the image. With the development of Internet technology, the text recognition technology for recognizing text information from images has an increasingly wide range of applications, and can be widely used to recognize text information in images such as document images, license plates, advertisements, menus, etc.

[0068] Taking advertising images as an example, the recognition of text information in advertising images is particularly important when reviewing advertisements. In order to review false advertising, illegal advertising, infringing advertising and other issues in advertising, it is necessary to recognize the text information in advertising images. Advertising images are typical images with complex foreground and background parts, large interference of the background part on the foreground part, rich font changes, and random layout structure.

[0069] In the prior art, layout analysis or deep learning methods are used to recognize text in images. The layout analysis method requires that the layout structure has strong regularity, and cannot process images with complex foreground and background parts and random layout structures. Especially when the font changes greatly or the background part interferes with the foreground part greatly, the recognition accuracy of the text information is not high, and the recognition accuracy requirements of the text recognition cannot be met. Using the deep learning method for text recognition greatly increases the accuracy of text recognition. However, in order to meet the high recognition accuracy requirements, the deep learning method has a complex algorithm model and a high demand for computing resources. The computing resource cost and time cost consumed are too high and difficult to bear. Especially for application scenarios such as advertising review, it is often necessary to regularly process a large number of text recognition tasks. If the deployed computing equipment is insufficient, it will take too long to process a large number of text recognition tasks, and the text recognition tasks cannot be completed within the specified time. However, if a large number of computing devices are deployed, these large number of computing devices are limited and wasted in the interval time when a large number of text recognition tasks are not required.

[0070] In view of this, the core invention of the present application is to first send the acquired image to be identified to a computing device, and the computing device determines at least one area to be identified that contains text information in the image to be identified, so that when the layout structure is arbitrary and the foreground and background parts of the image are complex, the area to be identified that contains text information is first framed. Then, the corresponding sub-image is determined according to the area to be identified, and each sub-image is an image to be identified contained in the area to be identified. By dividing the entire image to be identified that contains a large amount of image information into sub-images that contain less image information, the amount of calculation can be greatly reduced, and the pertinence of text recognition can be improved, thereby improving the accuracy of text recognition, so that the obtained text information can meet the requirements of recognition accuracy. At the same time, the cost of computing resources and time costs are also greatly reduced.

[0071] Furthermore, by utilizing the computing equipment on the cloud server to recognize text information in the image to be recognized, when it is necessary to process large quantities of text recognition tasks on a regular basis, sufficient computing equipment can be requested on the cloud server, and redundant computing equipment can be released during the intervals when large quantities of text recognition tasks are not needed. In this way, computing equipment can be requested dynamically based on the number of text recognition tasks, thereby avoiding waste of computing resources while meeting the requirements for recognition accuracy and processing speed for processing large quantities of text recognition tasks.

[0072] To make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0073] Embodiment 1

[0074] Figure 1 The flowchart of the text recognition method provided in the embodiment of the present application is shown in FIG. Figure 1 As shown, this embodiment mainly includes:

[0075] S101: The control end sends the acquired image to be recognized to the computing device of the cloud service end; the image to be recognized contains at least one text information to be recognized.

[0076] The image to be recognized can be any image containing text information, including document images, license plates, advertisements, menus, etc. Figure 2 As shown, the image to be recognized contains at least one text information.

[0077] The embodiment of the present application utilizes a computing device on a cloud server as a device for detecting and recognizing text information from an image to be recognized.

[0078] S102: The control end obtains at least one area to be identified that contains the text information in the image to be identified and is determined by the computing device.

[0079] The computing device of the cloud server first determines at least one region to be identified containing text information from the image to be identified, such as Figure 2 As shown, the area to be recognized is an image area containing text information in the image to be recognized.

[0080] S103: The control end determines a corresponding sub-image according to each of the to-be-recognized areas, and sends each of the sub-images to the computing device; the sub-image is a portion of the to-be-recognized image contained in the to-be-recognized area.

[0081] After receiving the area to be identified, the control end determines the sub-image according to the area to be identified, such as Figure 2 As shown, the sub-image is a part of the image to be identified, which is the part of the image to be identified contained in the area to be identified. Each sub-image is sent to the computing device. When there are more than two sub-images, each sub-image can be sent to a different computing device, each sub-image can be sent to the same computing device, or a predetermined number of sub-images in each sub-image can be sent to one computing device.

[0082] S104: The control end obtains the text information determined by the computing device from each of the sub-images.

[0083] like Figure 2 As shown, the control end obtains the text information determined by the computing device from each sub-image, and the text information determined in each sub-image is the text information in the image to be recognized.

[0084] The embodiment of the present application utilizes the computing device of the cloud service end to recognize the text information in the image to be recognized, and can flexibly schedule the computing device, thereby avoiding the waste of computing resources on the basis of meeting the requirements of recognition accuracy and processing speed for processing large-scale text recognition tasks.

[0085] Therefore, in the text recognition method provided in the embodiment of the present application, the control end can adopt the following two possible implementation methods to schedule the computing device.

[0086] A: In one possible implementation, the computing device may be scheduled each time text recognition in an image is required.

[0087] Specifically, when the control end obtains the image to be recognized, it applies to the cloud service end for allocation of at least one of the computing devices, and sends the acquired image to be recognized and each of the sub-images to the allocated computing device. The computing device applied for allocation can meet the text recognition requirements in the acquired image to be recognized. When the computing device includes a detection computing device and a recognition computing device, when acquiring the image to be recognized, the computing device applied for allocation includes the detection computing device and the recognition computing device.

[0088] After the computing device determines the text information from the image to be recognized, that is, when the control end obtains the text information determined by the computing device, the cloud service end is requested to release the at least one computing device. That is, after the recognition of the text information in the image is completed, the computing device is released. When the computing device includes a detection computing device and an identification computing device, both the detection computing device and the identification computing device are released.

[0089] In this implementation, computing equipment is requested on demand only when text recognition is needed, which can avoid wasting computing resources to the greatest extent possible on the basis of meeting the text recognition needs.

[0090] B: In another possible implementation, since the process of allocating computing devices on the cloud server side often consumes more time resources than the text recognition process, the total load of computing devices on the cloud server side can be monitored regularly and the computing devices can be scheduled according to the total load. Figure 3 The steps shown to perform the scheduling of computing equipment include:

[0091] S301: At every preset time interval, the control end obtains each computing device provided by the cloud service end to determine the total load of the area to be identified and the text information.

[0092] In this embodiment, the control end periodically monitors the load level of the computing device of the cloud service end, that is, obtains the total load of each computing device of the cloud service end. Exemplarily, the above total load is the total time for each computing device provided by the cloud service end to determine the area to be identified and the text information. The load level of the computing device of the cloud service end can also be determined based on the load parameters such as the processor utilization, memory utilization, and operating speed of each computing device of the cloud service end.

[0093] Here, the preset time interval may be set according to actual needs, for example, the preset time interval may be set to 1 second to 1 minute.

[0094] S302: When the total load is greater than a preset first load threshold, the control end applies to the cloud service end for allocating a predetermined number of the computing devices.

[0095] When the total load is greater than a preset first load threshold, for example, the total load is the total time for each computing device provided by the cloud server to determine the area to be identified and the text information, the first load threshold is a preset time length, and when the total time for each computing device provided by the cloud server to determine the area to be identified and the text information is greater than the preset first load threshold, an application is made to the cloud server to allocate a predetermined number of computing devices.

[0096] S303: When the total load is less than a preset second load threshold, the control end applies to the cloud service end for releasing a predetermined number of the computing devices.

[0097] When the total load is less than the preset second load threshold, for example, the total load is the total time for each computing device provided by the cloud server to determine the area to be identified and the text information, and the second load threshold is the preset time length, when the total time for each computing device provided by the cloud server to determine the area to be identified and the text information is less than the preset second load threshold, the cloud server is requested to release a predetermined number of computing devices. Since the cloud server often consumes more time resources in the process of allocating computing devices, the control end usually adopts a more conservative strategy when applying for the release of computing devices, to ensure that the computing devices occupied by the control end are not less than the preset device quantity threshold, or to ensure that the idle computing devices among the computing devices occupied by the control end are not less than the preset idle device quantity threshold.

[0098] For the sake of beauty and attraction, the text information in many images will be displayed in cursive, artistic and various italic fonts. Therefore, preferably, the area to be identified is a parallelogram area, which can better adapt to the characteristics of the variable fonts of the text information in the image and make text recognition faster and more accurate.

[0099] When a parallelogram is used as the shape of the area to be identified, it is necessary to determine an intermediate image corresponding to each area to be identified; the intermediate image is a parallelogram. Figure 4 As shown in FIG. 1 , the font in the original image is in italics, so the parallelogram region to be identified can be detected, and the intermediate image of the parallelogram can be determined based on the parallelogram region to be identified. In order to facilitate the subsequent steps of performing text recognition based on the sub-image and determining the text information in the sub-image, and to improve the accuracy of the text information determined from the sub-image, as well as the speed and efficiency of determining the text information, after obtaining the intermediate image of the parallelogram, as shown in FIG. Figure 4 As shown, the intermediate image is rotated and stretched to obtain a sub-image; the sub-image is rectangular. During the rotation and stretching process, the italic font is also rotated and stretched to become a standard font. Exemplarily, the OpenCV image processing engine can be used to determine the intermediate image according to the area to be identified, and the intermediate image is rotated and stretched to obtain a sub-image.

[0100] Embodiment 2

[0101] like Figure 5 FIG. 1 is a flow chart of another text recognition method provided in an embodiment of the present application, which is applied to a cloud service end of a text recognition system, and includes:

[0102] S501: When the computing device of the cloud service end receives the image to be recognized sent by the control end, at least one area to be recognized containing text information is determined from the image to be recognized, and the area to be recognized is sent to the control end.

[0103] The actual detection operation of the area to be identified that contains text information in the image to be identified is performed by the computing device of the cloud service end. After the computing device of the cloud service end receives the image to be identified sent by the control end, any commonly used method can be used to determine the above-mentioned at least one area to be identified from the image to be identified. For example, any commonly used deep learning model can be used to determine the above-mentioned at least one area to be identified from the image to be identified, including but not limited to: target detector (Single Shot Detector, SSD), scene-oriented text detection algorithm, region convolutional neural network (Region Convolutional Neural Networks, R-CNN), fast region convolutional neural network (Fast Region Convolutional Neural Networks, Fast-RCNN), faster region convolutional neural network (Faster Region Convolutional Neural Networks, Faster-RCNN), mask region convolutional neural network (Mask Region Convolutional Neural Networks, Mask-RCNN), etc.

[0104] S502: When the computing device receives the sub-images corresponding to each of the to-be-identified areas, the computing device determines the text information contained in each of the sub-images, and sends the determined text information to the control end.

[0105] Here, the acquired sub-image is still the portion of the image to be recognized that is contained in the area to be recognized.

[0106] The actual recognition operation of the text information contained in the sub-image is performed by the computing device of the cloud server. After the computing device of the cloud server receives the sub-image sent by the control end, any commonly used method can be used to determine the text information from the sub-image. For example, any commonly used deep learning model can be used to determine the text information from the sub-image, including but not limited to: Region Convolutional Neural Networks (R-CNN), Fast Region Convolutional Neural Networks (Fast-RCNN), Faster Region Convolutional Neural Networks (Faster-RCNN), Mask Region Convolutional Neural Networks (Mask-RCNN), etc.

[0107] When scheduling the computing device on the cloud server, the control end uses two possible implementation methods to apply to the cloud server for allocating or releasing the computing device. Regardless of which possible implementation method the control end uses to apply to the cloud server for allocating or releasing the computing device, in the text recognition method provided in the embodiment of the present application, the cloud server can use the following method: Figure 6 The steps shown perform the allocation of computing devices.

[0108] S601: When receiving the application for allocating the computing device from the control end, the cloud service end selects the computing device that meets the preset rule from each of the computing devices owned by itself as the candidate computing device according to the preset rule.

[0109] Since the cloud service end may include many kinds of computing devices, the cloud service end first screens each computing device it owns according to preset rules. For example, it takes up more memory resources to perform text recognition in an image, so a memory resource threshold can be set to screen computing devices with memory resources greater than the preset memory resource threshold as alternative computing devices. In addition, in the case where the computing device includes a detection computing device and an identification computing device, preset rules can be set according to different requirements for detecting the area to be identified containing text information and identifying text information from a sub-image. For example, the detection computing device needs to identify at least one area to be identified containing text information in the entire image, so in order to ensure the detection efficiency, the detection computing device can usually be implemented using a computing device using a graphics processing unit (GPU) as a processor resource. For another example, the recognition computing device usually only needs to analyze the sub-image, and compared with the entire image, the sub-image range is greatly reduced and more targeted, so it can be implemented using a computing device using a CPU as a processor resource. Since the cost of a computing device using a CPU as a processor resource is much lower than that of a computing device using a GPU as a processor resource, the text recognition method provided in the embodiment of the present application can greatly reduce the hardware cost required for text recognition.

[0110] S602: Score each candidate computing device according to the computing resources of each candidate computing device.

[0111] The scoring of the candidate computing devices is mainly based on the computing resources of the candidate computing devices, which mainly include processor resources and memory resources. For example, each candidate computing device can be scored based on the ratio of idle computing resources to total computing resources in the candidate computing device. The higher the ratio of idle computing resources to total computing resources, the higher the score of the corresponding candidate computing device. Each candidate computing device can also be scored based on the utilization of processor resources and memory resources in the computing resources. The closer the utilization of processor resources and memory resources is, the higher the score of the corresponding candidate computing device is.

[0112] S603: Determine at least one candidate computing device with the highest score among each of the candidate computing devices as a target computing device, and assign the target computing device to the control end.

[0113] Specifically, the step of assigning the target computing device to the control end can use the Internet Protocol Address (IP) of the target computing device to bind to the control end, or send the IP of the target computing device to the control end, and the control end will bind the target computing device to the control end.

[0114] Embodiment 3

[0115] like Figure 7 As shown, the embodiment of the present application also provides a text recognition system 700, including a control terminal 701 and a cloud service terminal 702;

[0116] The cloud service end 702 includes at least one computing device 7021; the computing device 7021 is used to determine at least one area to be identified containing text information from the image to be identified when receiving the image to be identified sent by the control end, and send the area to be identified to the control end; and is also used to determine the text information contained in each sub-image when receiving the sub-images corresponding to each area to be identified, and send the determined text information to the control end;

[0117] The control end 701 includes an identification device 7011; the identification device 7011 is connected to each of the computing devices 7021 respectively; the identification device 7011 is used to send the image to be identified and each of the sub-images to the computing device 7021 of the cloud service end 702, and receive the area to be identified and the text information determined by the computing device 7021; and is also used to determine the corresponding sub-image according to each area to be identified.

[0118] The identification device 7011 in the control terminal 701 and the computing device 7021 in the cloud service terminal 702 can be connected via a wired method such as a network cable, or can be connected via a wireless method such as WIFI, Bluetooth, etc.

[0119] In a possible implementation, in order to give full play to the advantages of the cloud server 702 in flexibly allocating and scheduling the computing devices 7021, the cloud server 702 further includes a main computing device 7022, the main computing device 7022 is connected to each of the computing devices 7021; the main computing device 7022 is used to schedule the computing devices 7021;

[0120] The control end 701 also includes: a resource scheduler 7012; the resource scheduler 7012 is connected to the identification device 7011 and the main computing device 7022 respectively; the resource scheduler 7012 is used to apply to the main computing device 7022 for allocating or releasing the computing device 7021.

[0121] In a possible implementation, in order to communicate with the outside world, obtain the image to be recognized sent by the outside world and return the text information obtained by recognition, preferably, a unified and standard interface can be provided. Specifically, the control terminal 701 also includes: a service gateway 7013; the service gateway 7013 is connected to the recognition device 7011; the service gateway 7013 is used to obtain the image to be recognized. Of course, the recognition device 7011 can also directly connect to the outside world to obtain the image to be recognized and return the text information obtained by recognition. However, providing a unified interface for information interaction with the outside world can achieve isolation between users and internal functions of the program, improve the convenience of users using the system provided by the embodiment of the present application, and increase the maintainability of the system provided by the embodiment of the present application.

[0122] The computing device 7021 in the cloud service end 702 can adopt a unified device. In order to save hardware costs, different devices can also be selected according to different performance requirements when detecting text information and recognizing text information. Therefore, the computing device 7021 can be divided into a detection computing device and a recognition computing device. The detection computing device and the recognition computing device are connected to the main computing device 7022 and the recognition device 7011 respectively; the detection computing device is used to receive the image to be recognized; and is also used to determine at least one area to be recognized containing text information from the image to be recognized; the recognition computing device is used to receive the sub-image; and is also used to determine the text information contained in the sub-image. When detecting text information, since it is necessary to accurately detect the text information from the entire image, the amount of calculation is large and the required calculation speed is fast. Therefore, the detection computing device includes a graphics processor, and a graphics processor GPU is used as a processor resource. The recognition computing device only needs to perform recognition on the sub-image. Compared with the entire image, the image information in the sub-image is greatly reduced, and the sub-image is determined based on the area to be recognized that contains text information. Most of the image information in the sub-image is text information to be recognized. Therefore, the amount of calculation to determine the text information from the sub-image is greatly reduced, and there is no need to use an expensive GPU as a processor resource. The recognition computing device may include a central processing unit CPU, and use the CPU as a processor resource, thereby reducing the hardware cost of the entire text recognition system without affecting the computing efficiency and recognition accuracy.

[0123] For example, the present application embodiment further provides a text recognition method, which is implemented by the text identification system 700 in the third embodiment. Figure 8 As shown, including:

[0124] S801 : The service gateway 7013 in the control terminal 701 obtains the image to be recognized, and sends the image to be recognized to the recognition device 7011 .

[0125] The recognition device 7011 can use the image to be recognized as an enabling identifier, and when the image to be recognized is obtained, the detection and recognition of the text information in the image to be recognized can be started. The service gateway 7013 can also send a recognition instruction while sending the image to be recognized, and the recognition device 7011 starts the detection and recognition of the text information in the image to be recognized when receiving the recognition instruction.

[0126] S802: The identification device 7011 applies to the resource scheduler 7012 for allocation of the computing device 7021.

[0127] In order to save the implementation cost of the text identification system 700, for example, a computing device using a GPU as a processor resource can be used as a detection computing device for determining a region to be identified from an image to be identified, and a computing device using a CPU as a processor resource can be used as a recognition computing device for determining text information based on a sub-image. In this case, two computing devices are applied for separately, a detection computing device and a recognition computing device.

[0128] The core control module inside the control end 701 can be assumed by the recognition device 7011 or by the service gateway 7013. When the recognition device 7011 assumes the core control module inside the control end 701, after receiving the image to be recognized or the recognition instruction, the recognition device 7011 sends the instruction, command or message for applying for the allocation of the computing device 7021 to the resource scheduler 7012. When the service gateway 7013 assumes the core control module inside the control end 701, the service gateway 7013 needs to send the instruction, command or message for applying for the allocation of the computing device 7021 to the recognition device 7011. After receiving the instruction, command or message for applying for the allocation of the computing device 7021 sent by the service gateway 7013, the recognition device 7011 sends the instruction, command or message for applying for the allocation of the computing device 7021 to the resource scheduler 7012.

[0129] S803: The resource scheduler 7012 applies to the main computing device 7022 of the cloud server 702 for allocation of computing device 7021.

[0130] After receiving the instruction, command or message for applying for allocation of computing device 7021 from the identification device 7011 , the resource scheduler 7012 sends the instruction, command or message for applying for allocation of computing device 7021 to the main computing device 7022 of the cloud server 702 .

[0131] S804: The main computing device 7022 returns the allocated IP address of the computing device 7021 to the resource scheduler 7012.

[0132] Exemplarily, this may include detecting a computing device and identifying an IP address of the computing device.

[0133] S805: The resource scheduler 7012 returns the IP address of the computing device 7021 to the identification device 7011.

[0134] When the service gateway 7013 assumes the core control module inside the control terminal 701, after the resource scheduler 7012 returns the IP address of the computing device 7021 to the identification device 7011, the identification device 7011 returns the IP address of the computing device 7021 to the service gateway 7013.

[0135] S806: The recognition device 7011 sends the image to be recognized to the computing device 7021 of the cloud server 702, and receives the text information determined by the computing device 7021.

[0136] For example, Fig. 9 As shown, after the recognition device 7011 obtains the image to be recognized, it sends the image to be recognized to the detection computing device using the GPU as the processor resource, and receives at least one area to be recognized sent back by the detection computing device. Preferably, the area to be recognized is a parallelogram, so the detection computing device can send the positions of the four vertices of the area to be recognized in the image to be recognized. For example, the horizontal and vertical coordinate values ​​of the pixel points where the vertices are located can be used as the positions of the vertices in the image to be recognized. The recognition device 7011 cuts out the image to be recognized according to the positions of the four vertices of each area to be recognized in the image to be recognized, and determines a corresponding sub-image for each area to be recognized.

[0137] The recognition device 7011 sends each sub-image in parallel to the recognition computing device. It can send to multiple recognition computing devices in parallel and receive text information determined by each recognition computing device. If the computing power of the recognition computing device allows, each sub-image can also be sent to the same recognition computing device in parallel. The recognition computing device processes each sub-image in parallel and sends the text information determined by each sub-image to the recognition device 7011. The recognition device 7011 sends out the text information determined from the image to be recognized.

[0138] S807: The recognition device 7011 sends the text information determined from the image to be recognized to the service gateway 7013.

[0139] The service gateway 7013 sends the received text information to the requester.

[0140] S808: The identification device 7011 applies to the resource scheduler 7012 for releasing the computing device 7021.

[0141] S809: The resource scheduler 7012 applies to the main computing device 7022 to release the computing device 7021.

[0142] Exemplarily, when a computing device using a GPU as processor resources is used as a detection computing device for determining an area to be identified from an image to be identified, and a computing device using a CPU as processor resources is used as a recognition computing device for determining text information based on a sub-image, the detection computing device and the recognition computing device are released together.

[0143] When the identification device 7011 assumes the core control module inside the control end 701, the identification device 7011 sends an instruction, command or message to the resource scheduler 7012 to apply for the release of the computing device 7021. When the service gateway 7013 assumes the core control module inside the control end 701, the service gateway 7013 needs to send an instruction, command or message to the identification device 7011 to apply for the release of the computing device 7021. After receiving the instruction, command or message sent by the service gateway 7013 to apply for the release of the computing device 7021, the identification device 7011 sends the instruction, command or message to the resource scheduler 7012 to apply for the release of the computing device 7021. After receiving the instruction, command or message sent by the identification device 7011 to apply for the release of the computing device 7021, the resource scheduler 7012 applies to the cloud service end 702 to release the computing device 7021.

[0144] Illustratively, the present application embodiment further provides another method for character recognition, which is implemented by the character identification system 700 in the third embodiment, and includes:

[0145] Step 1: The service gateway 7013 in the control terminal 701 obtains the image to be recognized, and sends the image to be recognized to the recognition device 7011.

[0146] Step 2: The recognition device 7011 sends the image to be recognized to the computing device 7021 of the cloud service end 702 and receives the text information determined by the computing device 7021.

[0147] Step 3: The recognition device 7011 sends the text information determined from the image to be recognized to the service gateway 7013.

[0148] Step 4: The service gateway 7013 sends the text message to the requester.

[0149] The method can monitor the total load of the computing devices 7021 of the cloud server 702 at regular intervals, schedule the computing devices 7021 according to the total load, and dynamically adjust and allocate the computing devices 7021 based on the load conditions, thereby improving the utilization efficiency of computing resources in the character classification system 700. Since there are more than two computing devices 7021 allocated to the recognition device 7011 in the cloud server 702, before sending the image to be recognized to the detection computing device and sending the sub-image to the recognition computing device, the recognition device 7011 can evaluate the load of each computing device 7021, send the image to be recognized to the detection computing device with the smallest load, and send the sub-image to the recognition computing device with the smallest load.

[0150] The cloud server 702 may include a main computing device 7022 and a computing device 7021. Preferably, the cloud server 702 may use Kubernet as the main computing device 7022. The main computing device 7022 is connected to multiple computing devices 7021. After receiving a request for assigning a computing device 7021, the main computing device 7022 determines the identifiable area from the image to be identified and determines the text information from the sub-image according to the two tasks, such as Fig.10 As shown, a Pod for text detection and a Pod for text recognition are created. The Pod for text detection is bound to a computing device with a GPU as a processor resource, and the Pod for text recognition is bound to a computing device with a CPU as a processor resource. Afterwards, the bound computing device 7021 is used as the computing device 7021 assigned to the control terminal 701, and the IP address of the bound computing device 7021 is sent to the resource scheduler 7012.

[0151] When the embodiments of the present application are actually implemented, especially for the application scenario of recognizing text information in a large number of images at a fixed time, assuming that the average text information recognition processing time of a single image is required to be within 100ms, it is difficult to meet the above computing performance requirements on a single computing device with a CPU as a processor resource in the prior art, and all of them are processed by a computing device with a GPU as a processor resource, which has a high hardware cost. When the method provided in the embodiments of the present application is used to process the above text information recognition task:

[0152] 1. The detection computing device using GPU as processor resources first quickly detects the area to be identified containing text information in the image to be identified, and returns the detected area to be identified to the recognition device. For general images to be identified, the average time consumption is about 20ms.

[0153] 2. The recognition device cuts the entire image to be recognized into multiple sub-images containing text information according to the area to be recognized returned by the detection computing device, and then sends them to the recognition computing device in parallel.

[0154] 3. The recognition computing device using the CPU as the processor resource completes the recognition of the text information in the sub-image. Since each sub-image has fewer pixels, the recognition computing device using the CPU as the processor resource can also complete the recognition of the text information in each sub-image within 50ms on average.

[0155] The time-consuming task of identifying text information in the image to be identified is decomposed into the identification of text information in multiple sub-images that can be processed concurrently, making full use of the computing power of the recognition computing device cluster with CPU as the processor resource, and meeting the time-consuming requirements of the application scenario of identifying text information in a large number of images at a time. And on the basis of ensuring the recognition accuracy, it can realize the rapid recognition of text information in the image to be identified by relying only on a small number of detection computing devices with GPU as the processor resource and a large number of cheap recognition computing devices with CPU as the processor resource.

[0156] Embodiment 4

[0157] A computer-readable medium storing instructions, which, when executed by a processor, causes the processor to execute the steps of the method provided in Embodiment 1 or Embodiment 2. In practical applications, the computer-readable medium may be included in the device / apparatus / system described in the above embodiments, or may exist independently without being assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, and when the one or more programs are executed, the steps of the method provided in Embodiment 1 or Embodiment 2 are performed.

[0158] According to the embodiments disclosed in the present application, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof, but it is not used to limit the scope of protection of the present application. In the embodiments disclosed in the present application, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, apparatus, or device.

[0159] In addition to being implemented by a data processing program, the method steps described in this application can also be implemented by hardware, for example, by logic gates, switches, application specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers, etc. Therefore, such hardware that can implement the method described in this application can also constitute this application.

[0160] Embodiment 5

[0161] The present application also provides an electronic device, which may be a computer or a server, and may be applied to the control end or cloud service end of the third embodiment of the system described above. Fig.11 As shown, it shows an electronic device 1100 provided by an embodiment of the device of the present application.

[0162] The electronic device may include one or more processors 1101 with processing cores, and one or more computer-readable storage media 1102. The electronic device may also include a power supply 1103 and an input / output unit 1104. Those skilled in the art will appreciate that Fig.11 The electronic device is not limited thereto and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0163] in:

[0164] The processor 1101 is the control part of the electronic device, which connects various parts using various interfaces and lines, and executes the steps of the method provided in Example 1 or Example 2 by running or executing the software program stored in the computer-readable storage medium 1102.

[0165] The computer-readable storage medium 1102 may be used to store software programs, namely, the programs involved in the methods provided in Embodiment 1 or Embodiment 2.

[0166] The processor 1101 executes various functional applications and data processing by running the software program stored in the computer-readable storage medium 1102. The computer-readable storage medium 1102 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data used according to the needs of the electronic device, etc. In addition, the computer-readable storage medium 1102 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the computer-readable storage medium 1102 may also include a memory controller to provide the processor 1101 with access to the computer-readable storage medium 1102.

[0167] The electronic device also includes a power supply 1103 for supplying power to each component. Preferably, the power supply 1103 can be logically connected to the processor 1101 through a power management system, so as to manage charging, discharging, power consumption and other functions through the power management system. The power supply 1103 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators and other arbitrary components.

[0168] The server may also include an input and output unit 1104, which can be used, for example, to receive input digital or character information, and to generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control; for example, it can be used to display information input by the user or information provided to the user and various graphical user interfaces of the server, which can be composed of graphics, text, icons, videos and any combination thereof.

[0169] The flow chart and block diagram in the accompanying drawings of the present application show the possible architecture, function and operation of the system, method and computer program product according to the various embodiments disclosed in the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the box can also occur in the order of the standards in different figures. For example, the boxes represented by two connections can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0170] Those skilled in the art will appreciate that the features described in the various embodiments and / or claims of the present application may be combined and / or combined in a variety of ways, even if such combinations and / or combinations are not explicitly described in the present application. In particular, without departing from the spirit and teachings of the present application, the features described in the various embodiments and / or claims of the present application may be combined and / or combined in a variety of ways, and all of these combinations and / or combinations fall within the scope disclosed in the present application.

[0171] The present application uses specific embodiments to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core ideas of the present application, and is not used to limit the present application. For those skilled in the art, changes can be made in the specific implementation methods and application scopes based on the ideas, spirits and principles of the present application, and any modifications, equivalent substitutions, improvements, etc. made by them should be included in the scope of protection of the present application.

Claims

1. A text recognition method, It is characterized in that include: The control end sends the acquired image to be recognized to the computing device of the cloud service end; the image to be recognized contains at least one text information to be recognized; The control end obtains at least one to-be-recognized area containing the text information in the to-be-recognized image determined by the computing device; The area to be identified is determined by the computing device from the image to be identified based on a deep learning model; The control terminal determines a corresponding sub-image according to each of the to-be-recognized areas, and sends each of the sub-images to the computing device; the sub-image is a portion of the to-be-recognized image contained in the to-be-recognized area; The control end obtains the text information determined by the computing device from each of the sub-images.

2. The method according to claim 1, It is characterized in that The method further includes scheduling the computing device of the cloud service end by using the following steps: At every preset time interval, the control end obtains each computing device provided by the cloud service end to determine the total load of the area to be identified and the text information; When the total load is greater than a preset first load threshold, the control end applies to the cloud service end for allocating a predetermined number of the computing devices; When the total load is less than a preset second load threshold, the control end applies to the cloud service end for releasing a predetermined number of the computing devices.

3. The method according to claim 2, It is characterized in that The total load is the total time taken by each computing device provided by the cloud server to determine the area to be identified and the text information.

4. The method according to claim 1, It is characterized in that The method further includes scheduling the computing device of the cloud service end by using the following steps: When the control end obtains the image to be recognized, it applies to the cloud service end for allocation of at least one of the computing devices, and sends the obtained image to be recognized and each of the sub-images to the allocated computing device; When the control end obtains the text information determined by the computing device, it applies to the cloud service end for releasing the at least one computing device.

5. The method according to claim 1, It is characterized in that The area to be identified is a parallelogram area; The step of the control end determining the corresponding sub-image according to each of the to-be-identified areas comprises: For each of the to-be-recognized areas, determining an intermediate image corresponding to the to-be-recognized area; the intermediate image is a parallelogram; The intermediate image is rotated and stretched to obtain the sub-image; the sub-image is a rectangle.

6. The method according to claim 1, It is characterized in that The computing device includes a detection computing device and an identification computing device; The step of sending the acquired image to be recognized to the computing device of the cloud service end is as follows: The control terminal sends the acquired image to be identified to the detection computing device; The step of sending each of the sub-images to the computing device is: Each of the sub-images is sent in parallel to the recognition computing device.

7. A method for character recognition, It is characterized in that include: When the computing device of the cloud service end receives the image to be recognized sent by the control end, at least one region to be recognized containing text information is determined from the image to be recognized, and the region to be recognized is sent to the control end; the region to be recognized is determined by the computing device from the image to be recognized based on the deep learning model; When the computing device receives the sub-images corresponding to each of the to-be-identified areas, the computing device determines the text information contained in each of the sub-images, and sends the determined text information to the control terminal; The sub-image is sent to the computing device after being determined by the control terminal according to the distribution of the area to be identified, and the sub-image is the part of the image to be identified contained in the area to be identified.

8. The method according to claim 7, It is characterized in that The computing device of the cloud service end is allocated using the following steps: When receiving the application for allocating the computing device from the control end, the cloud service end selects a computing device that meets the preset rule from each of the computing devices owned by the cloud service end as a candidate computing device according to the preset rule; Scoring each candidate computing device according to the computing resources of each candidate computing device; At least one candidate computing device with the highest score among each of the candidate computing devices is determined as a target computing device, and the target computing device is allocated to the control end.

9. A text recognition system, It is characterized in that include: Control side and cloud service side; The cloud service end includes at least one computing device; the computing device is used to determine at least one area to be identified containing text information from the image to be identified when receiving the image to be identified sent by the control end, and send the area to be identified to the control end; It is also used to determine the text information contained in each sub-image when receiving the sub-images corresponding to each of the to-be-recognized areas, and send the determined text information to the control end; the to-be-recognized areas are determined by the computing device from the to-be-recognized images based on the deep learning model; The control terminal includes an identification device; The recognition device is connected to each of the computing devices respectively; the recognition device is used to send the image to be recognized and each of the sub-images to the computing device of the cloud service end, and receive the area to be recognized and the text information determined by the computing device; It is also used to determine the corresponding sub-image according to each of the areas to be identified.

10. The system according to claim 9, It is characterized in that The cloud service end also includes: a main computing device; The main computing device is connected to each of the computing devices; the main computing device is used to schedule the computing devices; The control end further comprises: a resource scheduler; the resource scheduler is connected to the identification device and the main computing device respectively; the resource scheduler is used to apply to the main computing device for allocating or releasing the computing device.

11. The system according to claim 9, It is characterized in that The control end further includes: a service gateway; the service gateway is connected to the recognition device; the service gateway is used to obtain the image to be recognized.

12. The system according to claim 10, It is characterized in that The computing device includes a detection computing device and an identification computing device; The detection computing device and the identification computing device are connected to the main computing device and the identification device respectively; The detection computing device is used to receive the image to be identified; Also used to determine at least one area to be identified containing text information from the image to be identified; The recognition computing device is used to receive the sub-image; It is also used to determine the text information contained in the sub-image.

13. The system according to claim 12, It is characterized in that The detection computing device includes a graphics processor.

14. The system according to claim 12, It is characterized in that The identification computing device includes a central processing unit.

15. A computer-readable storage medium storing computer instructions, It is characterized in that When the instructions are executed by a processor, the steps of the method described in any one of claims 1 to 6 or claims 7 to 8 can be implemented.

16. An electronic device, It is characterized in that The electronic device comprises the computer-readable storage medium as claimed in claim 15, and further comprises a processor capable of executing the computer-readable storage medium.

Images