Text image processing method, device, equipment and medium

Abstract

The present invention provides a text image processing method, device, equipment and medium, which involve machine learning technology, computer vision technology and deep learning technology. The text image processing method includes: obtaining a text image; processing the text image on a client to obtain an intermediate processing result; sending the intermediate processing result to a server; and receiving the text image processing result from the server, wherein the text image processing result is obtained by processing the intermediate processing result by the server.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence, specifically to machine learning technology, computer vision technology and deep learning technology, and in particular to a text image processing method, a text image processing device, an electronic device, a computer-readable storage medium and a computer program product. Background Art

[0002] Artificial intelligence is a discipline that studies how to use computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.). It includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, as well as machine learning / deep learning, big data processing technology, knowledge graph technology, and other major directions.

[0003] Currently, many mobile phone software can provide optical character recognition (OCR) capabilities, but the existing methods of using mobile phones to perform OCR are inefficient.

[0004] The methods described in this section are not necessarily methods that have been previously conceived or employed. Unless otherwise indicated, it should not be assumed that any method described in this section is considered to be prior art simply because it is included in this section. Similarly, unless otherwise indicated, the issues mentioned in this section should not be considered to have been recognized in any prior art. Summary of the invention

[0005] The present disclosure provides a text image processing method, a text image processing device, an electronic device, a computer-readable storage medium, and a computer program product.

[0006] According to one aspect of the present disclosure, a text image processing method is provided. The method includes: obtaining a text image; processing the text image at a client to obtain an intermediate processing result; sending the intermediate processing result to a server; and receiving the text image processing result from the server, wherein the text image processing result is obtained by the server processing the intermediate processing result.

[0007] According to one aspect of the present disclosure, a text image processing method is provided, which includes: receiving an intermediate processing result of a text image from a client; processing the intermediate processing result at a server to obtain a text image processing result; and sending the text image processing result to the client.

[0008] According to another aspect of the present disclosure, a text image processing device is provided. The device includes: an acquisition unit configured to acquire a text image; a first processing unit configured to process the text image on a client to obtain an intermediate processing result; a first sending unit configured to send the intermediate processing result to a server; and a first receiving unit configured to receive the text image processing result from the server, wherein the text image processing result is obtained by the server processing the intermediate processing result.

[0009] According to another aspect of the present disclosure, a text image processing device is provided. The device includes: a second receiving unit configured to receive an intermediate processing result of a text image from a client; a second processing unit configured to process the intermediate processing result on a server to obtain a text image processing result; and a second sending unit configured to send the text image processing result to the client.

[0010] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the above method.

[0011] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the above method.

[0012] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program implements the above method when executed by a processor.

[0013] According to one or more embodiments of the present disclosure, by performing preliminary processing on the text image on the client to obtain an intermediate processing result, and then using the server to further process the intermediate processing result to obtain the text image processing result, the computing power on the terminal and the computing power on the server are fully utilized, thereby reducing the time consumption of the text image processing process and improving processing efficiency.

[0014] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings exemplarily illustrate the embodiments and constitute a part of the specification, and together with the text description of the specification, are used to explain the exemplary implementation of the embodiments. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0016] Figure 1 A schematic diagram showing an exemplary system in which the various methods described herein may be implemented according to an embodiment of the present disclosure;

[0017] Figure 2 A flowchart of a text image processing method according to an exemplary embodiment of the present disclosure is shown;

[0018] Figure 3A A schematic diagram showing the time consumption of a serial text image recognition method according to an exemplary embodiment of the present disclosure;

[0019] Figure 3B A schematic diagram showing the time consumption of a waterfall text image recognition method according to an exemplary embodiment of the present disclosure;

[0020] Figure 4A A schematic diagram showing a waterfall text image recognition method according to an exemplary embodiment of the present disclosure is shown;

[0021] Figure 4B A schematic diagram showing a neural network split and deployed on a client and a server for text image recognition according to an exemplary embodiment of the present disclosure is shown;

[0022] Figure 5 A schematic diagram showing the time consumption and amount of data generated of different network layers of a neural network according to an exemplary embodiment of the present disclosure;

[0023] Figure 6 A flowchart of a text image processing method according to an exemplary embodiment of the present disclosure is shown;

[0024] Figure 7 A structural block diagram of a text image processing apparatus according to an exemplary embodiment of the present disclosure is shown;

[0025] Figure 8 A structural block diagram of a text image processing apparatus according to an exemplary embodiment of the present disclosure is shown; and

[0026] Fig. 9 A structural block diagram of an exemplary electronic device that can be used to implement the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0027] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0028] In the present disclosure, unless otherwise specified, the use of the terms "first", "second", etc. to describe various elements is not intended to limit the positional relationship, timing relationship, or importance relationship of these elements, and such terms are only used to distinguish one element from another element. In some examples, the first element and the second element may refer to the same instance of the element, and in some cases, based on the description of the context, they may also refer to different instances.

[0029] The terms used in the description of various examples in this disclosure are only for the purpose of describing specific examples and are not intended to be limiting. Unless the context clearly indicates otherwise, if the number of elements is not specifically limited, the element can be one or more. In addition, the term "and / or" used in this disclosure covers any one of the listed items and all possible combinations.

[0030] In the related art, existing methods either complete text image processing on the client or on the server, but the former is limited by insufficient performance and configuration on the client, while the latter is limited by network transmission speed.

[0031] To solve the above problems, the present invention performs preliminary processing on the text image on the client to obtain an intermediate processing result, and then uses the server to further process the intermediate processing result to obtain the text image processing result, thereby making full use of the computing power on the terminal and the computing power on the server, thereby reducing the time consumption of the text image processing process and improving the processing efficiency.

[0032] The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0033] Figure 1 FIG. 1 is a schematic diagram of an exemplary system 100 in which various methods and apparatuses described herein may be implemented according to an embodiment of the present disclosure. Figure 1 , the system 100 includes one or more client devices 101, 102, 103, 104, 105, and 106, a server 120, and one or more communication networks 110 coupling the one or more client devices to the server 120. The client devices 101, 102, 103, 104, 105, and 106 may be configured to execute one or more applications.

[0034] In an embodiment of the present disclosure, the server 120 may run one or more services or software applications that enable the text image processing method to be performed.

[0035] In some embodiments, server 120 may also provide other services or software applications that may include non-virtualized environments and virtualized environments. In some embodiments, these services may be provided as web-based services or cloud services, such as provided to users of client devices 101, 102, 103, 104, 105, and / or 106 under a software as a service (SaaS) network.

[0036] exist Figure 1 In the configuration shown, the server 120 may include one or more components that implement the functions performed by the server 120. These components may include software components, hardware components, or a combination thereof that can be executed by one or more processors. Users operating client devices 101, 102, 103, 104, 105, and / or 106 may in turn utilize one or more client applications to interact with the server 120 to utilize the services provided by these components. It should be understood that a variety of different system configurations are possible, which may differ from the system 100. Therefore, Figure 1 is one example of a system for implementing the various methods described herein and is not intended to be limiting.

[0037] The user can use the client devices 101, 102, 103, 104, 105 and / or 106 to perform human-computer interaction. The client device can provide an interface that enables the user of the client device to interact with the client device. For example, a camera in the client device can capture text images in real time. The client device can also output information to the user via the interface. For example, the client can output text image processing results to the user. Figure 1 Only six client devices are depicted, but one skilled in the art will appreciate that the present disclosure may support any number of client devices.

[0038] Client devices 101, 102, 103, 104, 105 and / or 106 may include various types of computer devices, such as portable handheld devices, general-purpose computers (such as personal computers and laptop computers), workstation computers, wearable devices, smart screen devices, self-service terminal devices, service robots, game systems, thin clients, various messaging devices, sensors or other sensing devices, etc. These computer devices may run various types and versions of software applications and operating systems, such as MICROSOFT Windows, APPLE iOS, UNIX-like operating systems, Linux or Linux-like operating systems (such as GOOGLE Chrome OS); or include various mobile operating systems, such as MICROSOFT Windows Mobile OS, iOS, Windows Phone, Android. Portable handheld devices may include cellular phones, smart phones, tablet computers, personal digital assistants (PDAs), etc. Wearable devices may include head-mounted displays (such as smart glasses) and other devices. Game systems may include various handheld game devices, Internet-enabled game devices, etc. Client devices are capable of executing various different applications, such as various Internet-related applications, communication applications (such as email applications), short message service (SMS) applications, and may use various communication protocols.

[0039] The network 110 may be any type of network known to those skilled in the art that may support data communications using any of a variety of available protocols, including but not limited to TCP / IP, SNA, IPX, etc. By way of example only, the one or more networks 110 may be a local area network (LAN), an Ethernet-based network, a token ring, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., Bluetooth, WIFI), and / or any combination of these and / or other networks.

[0040] Server 120 may include one or more general purpose computers, dedicated server computers (e.g., PC (personal computer) servers, UNIX servers, mid-range servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and / or combination. Server 120 may include one or more virtual machines running virtual operating systems, or other computing architectures involving virtualization (e.g., one or more flexible pools of logical storage devices that may be virtualized to maintain a server's virtual storage device). In various embodiments, server 120 may run one or more services or software applications that provide the functionality described below.

[0041] The computing units in the server 120 may run one or more operating systems including any of the above operating systems and any commercially available server operating systems. The server 120 may also run any of a variety of additional server applications and / or middle-tier applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, etc.

[0042] In some implementations, server 120 may include one or more applications to analyze and consolidate data feeds and / or event updates received from users of client devices 101, 102, 103, 104, 105, and 106. Server 120 may also include one or more applications to display data feeds and / or real-time events via one or more display devices of client devices 101, 102, 103, 104, 105, and 106.

[0043] In some embodiments, the server 120 may be a server of a distributed system, or a server combined with a blockchain. The server 120 may also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. A cloud server is a host product in a cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and virtual private servers (VPS) services.

[0044] The system 100 may also include one or more databases 130. In some embodiments, these databases may be used to store data and other information. For example, one or more of the databases 130 may be used to store information such as audio files and video files. The databases 130 may reside in various locations. For example, the data repository used by the server 120 may be local to the server 120, or may be remote from the server 120 and may communicate with the server 120 via a network-based or dedicated connection. The databases 130 may be of different types. In some embodiments, the database used by the server 120 may be a database, such as a relational database. One or more of these databases may store, update, and retrieve data to and from the database in response to a command.

[0045] In some embodiments, one or more of the databases 130 may also be used by applications to store application data. The databases used by the applications may be different types of databases, such as a key-value store, an object store, or a conventional store backed by a file system.

[0046] Figure 1The system 100 may be configured and operated in various ways to enable the application of various methods and apparatuses described in the present disclosure.

[0047] According to one aspect of the present disclosure, a text image processing method is provided. Figure 2 As shown, the text image processing method includes: step S201, obtaining a text image; step S202, processing the text image on the client to obtain an intermediate processing result; step S203, sending the intermediate processing result to the server; and step S204, receiving the text image processing result from the server, wherein the text image processing result is obtained by the server processing the intermediate processing result.

[0048] Therefore, by performing preliminary processing on the text image on the client to obtain an intermediate processing result, and then using the server to further process the intermediate processing result to obtain the text image processing result, the computing power on the terminal and the computing power on the server are fully utilized, thereby reducing the time consumption of the text image processing process and improving processing efficiency.

[0049] The text image processing method disclosed in the present invention can be used for various text image processing tasks, such as text detection, text region segmentation, text recognition, etc. The present invention will use the text recognition task as an example to illustrate the technical solution of the present invention, but it is not intended to limit the scope of protection.

[0050] In the context of the present disclosure, the client may be, for example, a mobile phone, a tablet computer, a laptop computer, or other terminal having weaker computing power and fewer storage resources than the server. The server may be, for example, a cloud server or other type of server capable of quickly processing text images.

[0051] Existing text image processing methods can include two major categories of methods. The first is the traditional multi-stage processing method, which can first perform a series of preprocessing on the text image, and then further process it by a specific task processing model (for example, a neural network) to obtain the text image processing result; the second is an end-to-end method based on deep learning, which can directly input the text image into a trained deep learning neural network to obtain the text image processing result.

[0052] In an exemplary embodiment, during the text recognition process, as shown in Table 1, the size of the transmitted data is 4.57MB, and the upload speed is 2.28MB / s, so the requested transmission process takes 2.01s, the server recognition takes 2.17s, and the returned text image processing result size is 3.69KB and takes 0ms, with a total time of 4.18s.

[0053] Table 1

[0054] Request start time 2022-09-28 14:50:02 Request end time 2022-09-28 14:50:04 Response start time 2022-09-28 14:50:06 Response end time 2022-09-28 14:50:06 Duration 4.18s Request time 2.01s Response time 0ms Delay 2.17s speed 1.09MB / s Request speed 2.28MB / s Response speed 0B / s Request data volume 4.57MB(4,790,329bytes) Response data volume 3.69KB(3,783bytes) Total data volume 4.57MB(4,794,112bytes)

[0055] From the above data, we can see that data transmission takes up nearly half of the time (2.01s). Therefore, optimizing the transmission time can significantly improve the efficiency of the overall recognition process.

[0056] For the traditional multi-stage processing method, the amount of computation in the preprocessing process of text images is no more than that of convolution operations, so it will not cause the intermediate feature vectors to increase exponentially. With the hardware configuration of mobile phones currently on the market, this type of preprocessing is very fast, basically completed in milliseconds to tens of milliseconds, so the computational work of the image preprocessing part can be migrated to the client.

[0057] In some embodiments, the preprocessing of the text image may include grayscale, binarization, image noise reduction, tilt correction, text detection, text segmentation, etc. It should be noted that these steps may be performed using traditional non-machine learning methods to improve efficiency.

[0058] Normally, the speed of text processing is proportional to the amount of data that needs to be processed. Take text recognition as an example. For example, it takes 2 seconds to recognize a 4MB image, and it may take about 200 milliseconds to recognize a 400KB image. Therefore, the text image is segmented into multiple segmented images on the client, and then transmitted to the server in a waterfall flow. The server starts to recognize from the moment it receives the first segmented image, and then until the last segmented image is received and recognized, the entire transmission process and recognition process are carried out in parallel. Compared to Figure 3A In the example, the entire text image is uploaded and then recognized in a serial manner. Figure 3B The waterfall upload and recognition method in the process enables the server to start processing the text image after receiving the first text segmentation image, thus significantly shortening the time consumption of the text recognition task.

[0059] According to some embodiments, step S202, processing the text image at the client to obtain an intermediate processing result may include: segmenting the text image to obtain an intermediate processing result, wherein the intermediate processing result includes a plurality of text segmentation images.

[0060] According to some embodiments, segmenting a text image may be, for example, segmenting the text image into a plurality of sub-images containing text based on blank areas in the image. Segmenting the text image to obtain an intermediate processing result may include: segmenting the text image to obtain a plurality of text segmentation images. Each of the plurality of text segmentation images may include text to be recognized. Traditional text recognition tasks require segmenting the image first to segment different texts in the image into different areas, which coincides with the above-mentioned processing flow.

[0061] In some embodiments, multiple text segmentation images may be sent to the server in sequence. In response to receiving each of the multiple text segmentation images, the server may process the text segmentation image to obtain a text segmentation image processing result corresponding to the text segmentation image. Thus, waterfall flow sending and processing of text segmentation images is achieved, shortening the overall time consumption of the text processing process.

[0062] In some embodiments, the text segmentation image processing result obtained by the server processing each of the multiple text segmentation images may include a recognition result of the text to be recognized in the text segmentation image.

[0063] According to some embodiments, the text image processing result may be obtained by aggregating the multiple text segmentation image processing results corresponding to the multiple text segmentation images after the server completes processing of the multiple text segmentation images. For example, the text recognized in the multiple text segmentation images may be aggregated to obtain the OCR result corresponding to the text image.

[0064] In an exemplary embodiment, Figure 4A As shown, the mobile phone obtains a text image containing multiple text areas, and performs preprocessing steps such as grayscale, binarization, image noise reduction, tilt correction, and text segmentation on the text image, and then sends the processed intermediate processing results (i.e., multiple text segmentation images) to the server in a waterfall flow manner. The server processes each text segmentation image in turn to generate a text processing result.

[0065] For end-to-end methods based on deep learning, the deep neural network models used for text image processing are often relatively large, usually tens or even hundreds of megabytes. It is unrealistic to put such a large volume directly on the mobile phone itself, and even dynamic downloading takes a lot of time. In addition, the text image processing model relies on the resources of the Graphic Processing Unit (GPU) and the hardware configuration of the mobile phone. The operating efficiency is much worse than that of the cloud server. Therefore, most related technologies deploy the model on the server to run.

[0066] In some embodiments, a deep neural network is usually composed of multiple network layers, including a fully connected layer (FC), a convolution layer (conv) - extracting a set of feature maps from the image, a pooling layer (pool) - applying a predefined function to map and combine features, an activation layer - using a nonlinear function to operate on the input data to generate and output the same amount of data (depending on the activation function, it is divided into a sigmoid layer, a relu layer, an htanh layer, etc.), a normalization layer (norm), a soft maximization layer (softmax), an argmax layer, a random inactivation layer (dropout), etc. The size of data transmission between each layer is different, so reasonably planning the relationship between each layer of the network, placing part of the layer on the client, and placing the other part on the server for calculation, can reduce the size of the data packet transmitted over the network, thereby reducing the overall time consumption. Figure 4B As shown, the mobile phone can process the received text image using the first part of the network layer deployed on the mobile phone to obtain an intermediate processing result (for example, a feature vector output by the intermediate layer of the neural network). Furthermore, the mobile phone can send the intermediate processing result to the server, and the latter part of the network layer deployed on the server will further process it to obtain the final text image processing result (for example, a text recognition result).

[0067] According to some embodiments, a first part of a text image processing neural network may be deployed on a client, and a second part of a text image processing neural network may be deployed on a server. Step S202, processing a text image on a client to obtain an intermediate processing result may include: processing the text image using at least one network layer included in the first part of the text processing neural network deployed on the client to obtain an intermediate processing result. The text image processing result is obtained by processing the intermediate processing result using at least one network layer included in the second part of the text processing neural network deployed on the server. In this way, the computing power on the terminal and the computing power on the server are fully utilized, thereby reducing the time consumption of the text image processing process and improving processing efficiency.

[0068] According to some embodiments, the data volume of the intermediate processing result can be smaller than the data volume of the text image. Therefore, by deploying some layers of the deep neural network on the client, and sending the intermediate processing result with a smaller data volume than the text image to the server for further processing, the amount of data transmission between the client and the server can be reduced, thereby reducing the time consumption of data transmission, and further reducing the total time consumption of the text processing process. It can not only bring a better user experience to the user on the client, but also increase the throughput of the server and improve the utilization of resources.

[0069] According to some embodiments, the first part and the second part of the text image processing neural network may be determined in advance based on at least one of the amount of received data and the processing time of each network layer in the text image neural network. As described above, the data transmission size between each network layer is different, and the amount of calculation of each layer is also different, so the neural network can be divided into the first part and the second part at a location where the amount of data is relatively small, and the network layer with serious time consumption can be divided into the second part processed by the server as much as possible.

[0070] In an exemplary embodiment, taking AlexNet as an example, Figure 5 As shown in the figure, the data volume of the first few layers of AlexNet is relatively large, while the calculation time of the latter layers is relatively long. It is not until the pooling layer 5 (pool5) that the data volume is significantly reduced compared with the previous layers, and the time consumed by the calculation of the fully connected layer immediately afterwards accounts for a large part of the total time consumption. Therefore, intuitively speaking, the best split point is between pool5 and fully connected layer 6 (fc6). Actual experiments have also proved that splitting from here can obtain better time consumption results. Putting the fully connected layer with huge calculation volume on the server can obtain faster calculation results, while putting the convolutional layer and the pooling layer that can significantly reduce the amount of data on the client can obtain better data transmission results.

[0071] According to some embodiments, the rules for segmenting the first part and the second part may also be directly set based on the above observations. The first part of the text image processing neural network may include: all network layers before the first fully connected layer in the text image processing neural network; or the last pooling layer and all network layers before it in the text image processing neural network. It is understandable that the text image processing neural network may also be pre-segmented into the first part and the second part in other ways, which are not limited here.

[0072] According to another aspect of the present disclosure, a text image processing method is provided. Figure 6As shown, the method includes: step S601, receiving the intermediate processing result of the text image from the client; step S602, processing the intermediate processing result on the server to obtain the text image processing result; and step S603, sending the text image processing result to the client.

[0073] It is understandable that the intermediate processing result received in step S601 may be, for example, the result of the client executing Figure 2 The processing of the intermediate processing result in step S602 can refer to the previous description and will not be repeated here.

[0074] According to some embodiments, the intermediate processing result may include multiple text segmentation images obtained after segmenting the text image, and the multiple text segmentation images may be received from the client in sequence. Step S602, processing the intermediate processing result at the server to obtain the text image processing result may include: in response to receiving each text segmentation image in the multiple text segmentation images, processing the text segmentation image to obtain a text segmentation image processing result corresponding to the text segmentation image.

[0075] According to some embodiments, a first part of a text image processing neural network may be deployed on a client. A second part of a text image processing neural network may be deployed on a server. An intermediate processing result may be obtained by processing a text image using at least one network layer included in the first part of the text processing neural network deployed on the client. Step S602, processing the intermediate processing result on the server to obtain a text image processing result may include: processing the intermediate processing result using at least one network layer included in the second part of the text processing neural network deployed on the server to obtain a text image processing result.

[0076] According to some embodiments, the second part of the text image processing neural network may include: the first fully connected layer and all subsequent network layers in the text image processing neural network; or all network layers after the last pooling layer in the text image processing neural network.

[0077] According to another aspect of the present disclosure, a text image processing device is provided. Figure 7As shown, the device 700 includes: an acquisition unit 710, configured to acquire a text image; a first processing unit 720, configured to process the text image at the client to obtain an intermediate processing result; a first sending unit 730, configured to send the intermediate processing result to the server; and a first receiving unit 740, configured to receive the text image processing result from the server, the text image processing result is obtained by the server processing the intermediate processing result. It can be understood that the operations of units 710-740 in the device 700 are similar to the operations of steps S201-S204, and are not described in detail here.

[0078] According to another aspect of the present disclosure, a text image processing device is provided. Figure 8 As shown, the device 800 includes: a second receiving unit 810, configured to receive the intermediate processing result of the text image from the client; a second processing unit 820, configured to process the intermediate processing result on the server to obtain the text image processing result; and a second sending unit 830, configured to send the text image processing result to the client. It can be understood that the operations of units 810-830 in the device 800 are similar to the operations of steps S601-S603, and are not described in detail here.

[0079] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0080] According to an embodiment of the present disclosure, an electronic device, a readable storage medium and a computer program product are also provided.

[0081] refer to Fig. 9 , a block diagram of an electronic device 900 that can be used as a server or client of the present disclosure will now be described, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0082] like Fig. 9As shown, the device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the device 900 can also be stored. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0083] Multiple components in the device 900 are connected to the I / O interface 905, including: an input unit 906, an output unit 907, a storage unit 908, and a communication unit 909. The input unit 906 can be any type of device that can input information to the device 900. The input unit 906 can receive input digital or character information, and generate key signal input related to user settings and / or function control of the electronic device, and can include but is not limited to a mouse, a keyboard, a touch screen, a trackpad, a trackball, a joystick, a microphone and / or a remote control. The output unit 907 can be any type of device that can present information, and can include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator and / or a printer. The storage unit 908 can include but is not limited to a disk, an optical disk. The communication unit 909 allows the device 900 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and can include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver and / or a chipset, such as Bluetooth TM devices, 802.11 devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0084] The computing unit 901 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning network algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 901 performs the various methods and processes described above, such as the text image processing method. For example, in some embodiments, the text image processing method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the text image processing method described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to perform the text image processing method in any other appropriate manner (e.g., by means of firmware).

[0085] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0086] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0087] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0088] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0089] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0090] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0091] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0092] Although the embodiments or examples of the present disclosure have been described with reference to the accompanying drawings, it should be understood that the above-mentioned methods, systems and devices are merely exemplary embodiments or examples, and the scope of the present invention is not limited by these embodiments or examples, but only by the claims after authorization and their equivalent scope. Various elements in the embodiments or examples can be omitted or replaced by their equivalent elements. In addition, each step can be performed in an order different from that described in the present disclosure. Further, the various elements in the embodiments or examples can be combined in various ways. It is important that with the evolution of technology, many elements described herein can be replaced by equivalent elements that appear after the present disclosure.

Claims

1. A text image processing method, comprising: Get text image; The text image is processed on the client to obtain an intermediate processing result, including: Preprocessing the text image, wherein the preprocessing includes at least one of grayscale, binarization, image noise reduction, tilt correction and text detection; Segmenting the preprocessed text image to obtain the intermediate processing result, wherein the intermediate processing result includes a plurality of text segmentation images, wherein each of the plurality of text segmentation images includes the text to be recognized; Sending the intermediate processing result to a server, wherein the plurality of text segmentation images are sent to the server in sequence, and the second and subsequent text segmentation images of the plurality of text segmentation images are sent after the previous text segmentation image is sent, and the server is configured to, in response to receiving each text segmentation image of the plurality of text segmentation images, perform processing on the text segmentation image and receiving the remaining text segmentation images in parallel; and A text image processing result is received from the server, wherein the text image processing result is obtained by aggregating a plurality of text segmentation image processing results corresponding to the plurality of text segmentation images after the server has processed the plurality of text segmentation images, wherein the text segmentation image processing result obtained by the server processing each of the plurality of text segmentation images includes a recognition result of a text to be recognized in the text segmentation image.

2. The method according to claim 1, wherein: The client is provided with a first part of a text image processing neural network, and the server is provided with a second part of the text image processing neural network. The text image is processed on the client to obtain an intermediate processing result, which includes: The text image is processed using at least one network layer included in the first part of the text processing neural network deployed on the client to obtain the intermediate processing result, wherein the text image processing result is obtained by processing the intermediate processing result using at least one network layer included in the second part of the text processing neural network deployed on the server.

3. The method according to claim 2, wherein: The first part and the second part of the text image processing neural network are determined in advance based on at least one of the amount of received data and the processing time of each network layer in the text image neural network.

4. The method according to claim 3, wherein: The data volume of the intermediate processing result is smaller than the data volume of the text image.

5. The method according to claim 2, wherein: The first part of the text image processing neural network includes: All network layers before the first fully connected layer in the text image processing neural network; or The last pooling layer and all previous network layers in the text image processing neural network.

6. A text image processing method, comprising: Receiving an intermediate processing result of a text image from a client, the intermediate processing result comprising a plurality of text segmentation images obtained by segmenting the preprocessed text image, the preprocessing comprising at least one of graying, binarization, image noise reduction, tilt correction and text detection, the plurality of text segmentation images being received from the client in sequence, and the second and subsequent text segmentation images of the plurality of text segmentation images are received after the previous text segmentation image is received, wherein each of the plurality of text segmentation images comprises text to be recognized; Processing the intermediate processing result on the server to obtain a text image processing result includes: In response to receiving each of the plurality of text segmentation images, performing processing of the text segmentation image and receiving of the remaining text segmentation images in parallel; and In response to determining that the plurality of text segmentation images have been processed, aggregating a plurality of text segmentation image processing results corresponding to the plurality of text segmentation images to obtain the text image processing result, wherein the text segmentation image processing result obtained by the server processing each of the plurality of text segmentation images includes a recognition result of the text to be recognized in the text segmentation image; and The text image processing result is sent to the client.

7. The method according to claim 6, wherein: The first part of the text image processing neural network is deployed on the client, and the second part of the text image processing neural network is deployed on the server. The intermediate processing result is obtained by processing the text image using at least one network layer included in the first part of the text processing neural network deployed on the client. The processing of the intermediate processing result on the server to obtain the text image processing result includes: The intermediate processing result is processed using at least one network layer included in the second part of the text processing neural network deployed on the server to obtain the text image processing result.

8. The method according to claim 7, wherein: The second part of the text image processing neural network includes: The first fully connected layer and all subsequent network layers in the text image processing neural network; or All network layers after the last pooling layer in the text image processing neural network.

9. A text image processing device, comprising: An acquisition unit, configured to acquire a text image; The first processing unit is configured to process the text image on the client to obtain an intermediate processing result, including: Preprocessing the text image, wherein the preprocessing includes at least one of grayscale, binarization, image noise reduction, tilt correction and text detection; Segmenting the preprocessed text image to obtain the intermediate processing result, wherein the intermediate processing result includes a plurality of text segmentation images, wherein each of the plurality of text segmentation images includes the text to be recognized; a first sending unit configured to send the intermediate processing result to a server, wherein the plurality of text segmentation images are sent to the server in sequence, and the second and subsequent text segmentation images of the plurality of text segmentation images are sent after the previous text segmentation image is sent, and the server is configured to, in response to receiving each text segmentation image of the plurality of text segmentation images, perform processing on the text segmentation image and receiving the remaining text segmentation images in parallel; and The first receiving unit is configured to receive a text image processing result from the server, wherein the text image processing result is obtained by aggregating a plurality of text segmentation image processing results corresponding to the plurality of text segmentation images after the server completes processing of the plurality of text segmentation images, wherein the text segmentation image processing result obtained by the server processing each of the plurality of text segmentation images includes a recognition result of a text to be recognized in the text segmentation image.

10. A text image processing device, comprising: A second receiving unit is configured to receive an intermediate processing result of a text image from a client, wherein the intermediate processing result includes a plurality of text segmentation images obtained by segmenting the preprocessed text image, wherein the preprocessing includes at least one of graying, binarization, image noise reduction, tilt correction, and text detection, wherein the plurality of text segmentation images are received from the client in sequence, and the second and subsequent text segmentation images of the plurality of text segmentation images are received after the previous text segmentation image is received, wherein each of the plurality of text segmentation images includes text to be recognized; The second processing unit is configured to process the intermediate processing result on the server to obtain a text image processing result, including: In response to receiving each of the plurality of text segmentation images, performing processing of the text segmentation image and receiving of the remaining text segmentation images in parallel; and In response to determining that the plurality of text segmentation images have been processed, aggregating a plurality of text segmentation image processing results corresponding to the plurality of text segmentation images to obtain the text image processing result, wherein the text segmentation image processing result obtained by the server processing each of the plurality of text segmentation images includes a recognition result of the text to be recognized in the text segmentation image; and The second sending unit is configured to send the text image processing result to the client.

11. An electronic device, comprising: at least one processor; as well as a memory communicatively coupled to the at least one processor; in The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-8.

13. A computer program product comprising a computer program, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images