Method and device for positioning and recognizing text information in an image

Abstract

The present application discloses a method and apparatus for locating and identifying text information in an image. The method for locating text information in an image comprises: obtaining an image to be detected containing text information; generating a mask image for locating text information in the image to be detected based on the image to be detected using a neural network-based text localization model; and determining, based on the mask image, position information of a text region in the image to be detected containing the text information. The operation of constructing the text localization model comprises: determining a generator network of a generative adversarial network used to construct the text localization model; and reducing at least a portion of network parameters of the generator network according to a preset ratio.

Description

Technical Field

[0001] With the rapid development of the mobile internet, the application of artificial intelligence (AI) on mobile devices has gradually become a new development direction and a major demand. The development of AI precisely meets these requirements, and as a result, the integration of AI and the mobile internet has attracted increasing attention from researchers. Identity authentication in mobile applications is one of these products. Common mobile authentication methods include facial recognition, liveness detection, and ID card optical character recognition (OCR). ID card OCR primarily recognizes text on ID cards to facilitate personal verification and is often used in fields such as security and finance.

[0002] Current ID card OCR recognition relies on traditional image processing methods: First, the ID card image undergoes preprocessing (e.g., cropping, grayscaling, and binarization) to remove background noise, resulting in a binary image that is converted to black and white and retains only the text area. The text area is then scanned in rows and columns, calculating the horizontal and vertical projections of the text area to determine the row and column baselines. This allows the specific coordinates of the text to be determined for subsequent OCR recognition.

[0003] The development of deep learning has led to the emergence of technologies that use neural networks to locate text on ID cards. These technologies often employ target location or segmentation models to locate text areas. This approach suffers from complex network structures, large models, difficulty translating to mobile devices, slow positioning speeds, low accuracy, insufficient robustness, and susceptibility to noise.

[0004] The above-mentioned existing technologies for using neural networks to locate ID card text have the following technical problems: the network structure is complex and the model is too large, making it difficult to apply to mobile terminals; the positioning speed is slow, the accuracy is low, the robustness is insufficient, and it is easily affected by noise interference. No effective solution has been proposed so far. Summary of the Invention

[0005] The embodiments of the present disclosure provide a method and device for locating and identifying text information in an image, so as to at least solve the technical problems in the prior art of using neural networks to locate identity card text, such as complex network structure and excessively large model, which makes it difficult to apply to mobile terminals, as well as slow positioning speed, low accuracy, insufficient robustness and susceptibility to noise interference.

[0006] According to one aspect of an embodiment of the present disclosure, a method for locating text information in an image is provided, comprising: obtaining an image to be detected containing text information; generating a mask image for locating text information in the image to be detected based on the image to be detected using a neural network-based text localization model; and determining, based on the mask image, position information of a text region containing text information in the image to be detected, wherein the operation of constructing the text localization model comprises: determining a generator network of a generative adversarial network used to construct the text localization model; and reducing at least a portion of network parameters of the generator network according to a preset ratio.

[0007] According to another aspect of an embodiment of the present disclosure, a method for identifying text information in an image is also provided, including: determining position information of a text area containing text information in the image to be detected; determining the text area in the detection image based on the position information; and identifying the text information in the text area.

[0008] According to another aspect of an embodiment of the present disclosure, a storage medium is further provided, the storage medium including a stored program, wherein when the program is run, a processor executes any one of the above methods.

[0009] According to another aspect of an embodiment of the present disclosure, a device for locating text information in an image is also provided, including: an acquisition module for acquiring an image to be detected containing text information; a generation module for generating a mask image for locating text information in the image to be detected based on the image to be detected using a text localization model based on a neural network; and a determination module for determining position information of a text area containing text information in the image to be detected based on the mask image, wherein the generation module includes: a determination submodule for determining a generator network of a generative adversarial network for constructing a text localization model; and a reduction submodule for reducing at least a portion of network parameters of the generator network according to a preset ratio.

[0010] According to another aspect of an embodiment of the present disclosure, a device for identifying text information in an image is also provided, including: a first determination module for determining position information of a text area containing text information in the image to be detected; a second determination module for determining the text area in the image to be detected based on the position information; and a recognition module for identifying text information in the text area.

[0011] According to another aspect of an embodiment of the present disclosure, a device for locating text information in an image is also provided, including: a first processor; and a first memory, connected to the first processor, for providing the first processor with instructions for processing the following processing steps: obtaining an image to be detected containing text information; using a neural network-based text localization model to generate a mask image for locating text information in the image to be detected based on the image to be detected; and determining, based on the mask image, position information of a text area containing text information in the image to be detected, wherein the operation of constructing the text localization model includes: determining a generator network of a generative adversarial network used to construct the text localization model; and reducing at least a portion of the network parameters of the generator network according to a preset ratio.

[0012] According to another aspect of an embodiment of the present disclosure, a device for identifying text information in an image is also provided, including: a second processor; and a second memory connected to the second processor, for providing the second processor with instructions for processing the following processing steps: determining position information of a text area containing text information in the image to be detected; determining the text area in the image to be detected based on the position information; and identifying the text information in the text area.

[0013] In the embodiment of the present disclosure, since the text positioning model is generated by reducing at least a portion of the network parameters of the generator network, the text positioning model is also simplified. And the computational complexity of the text positioning model is also reduced. Thus, the requirements of the mobile terminal are met. Since the network structure is simple, the positioning speed is also at the millisecond level, which meets the real-time requirements of the mobile terminal. In addition, since the text positioning model is constructed based on the generator network, the generator network can be trained using the corresponding discriminator network. Therefore, the discriminator network can enable the generator network to achieve high accuracy and robustness even after reducing at least a portion of the network parameters, and thus is not easily affected by noise interference. This solves the technical problems in the prior art of using neural networks to locate ID card text, such as complex network structure and excessively large models, which are difficult to apply to mobile terminals, as well as slow positioning speed, low accuracy, insufficient robustness and susceptibility to noise interference. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of this application. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation of the present disclosure. In the drawings:

[0015] Figure 1 is a hardware structure block diagram of a computing device for implementing the method according to embodiment 1 of the present disclosure;

[0016] Figure 2 is a flowchart of a method for locating text information in an image according to the first aspect of embodiment 1 of the present disclosure;

[0017] Figure 3 is a flowchart of a method for recognizing text information in an image according to the second aspect of Embodiment 1 of the present disclosure;

[0018] Figure 4 is a schematic diagram of generating a mask image for locating text information in an image to be detected according to the first aspect of embodiment 1 of the present disclosure;

[0019] Figure 5 is a schematic diagram of a network architecture diagram of a text localization model according to the first aspect of embodiment 1 of the present disclosure;

[0020] Figure 6 is a schematic diagram of a flowchart of text localization in an image to be detected according to the first aspect of embodiment 1 of the present disclosure;

[0021] Figure 7 is a schematic diagram of a method for locating text information in an image according to the first aspect of embodiment 1 of the present disclosure;

[0022] Figure 8 is a schematic diagram of an apparatus for locating text information in an image according to the first aspect of embodiment 2 of the present disclosure;

[0023] Figure 9 is a schematic diagram of an apparatus for recognizing text information in an image according to the second aspect of Embodiment 2 of the present disclosure;

[0024] Figure 10 is a schematic diagram of an apparatus for locating text information in an image according to the first aspect of Embodiment 3 of the present disclosure; and

[0025] Figure 11 It is a schematic diagram of a device for recognizing text information in an image according to the second aspect of Embodiment 3 of the present disclosure. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present disclosure.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] Example 1

[0029] According to this embodiment, a method embodiment for locating text information in an image is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0030] The method embodiment provided in this embodiment can be executed in a computing device similar to a mobile terminal. Figure 1 FIG1 shows a hardware structure block diagram of a computing device for locating text information in an image. Figure 1 As shown, the computing device may include one or more processors (the processor may include but is not limited to a microprocessor MCU or a programmable logic device FPGA, etc.), a memory for storing data, and a transmission device for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply and / or a camera. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0031] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuitry." The data processing circuitry may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuitry may be a single, independent processing module, or may be incorporated in whole or in part into any of the other components of the computing device. As described in the embodiments of the present disclosure, the data processing circuitry serves as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0032] The memory can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the method for locating text information in an image in the embodiment of the present disclosure. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, implementing the method for locating text information in an image of the above-mentioned application. The memory may include a high-speed random access memory and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely located relative to the processor, and these remote memories may be connected to the computing device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0033] The transmission device is used to receive or send data via a network. Specific examples of the aforementioned network may include a wireless network provided by a communications provider of the computing device. In one embodiment, the transmission device includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0034] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computing device.

[0035] It should be noted that, in some optional embodiments, the above Figure 1 The computing device shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware and software elements. Figure 1 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the computing device described above.

[0036] In the above operating environment, according to a first aspect of this embodiment, a method for locating text information in an image is provided. Figure 2 A schematic diagram of the process is shown in FIG. Figure 2 As shown, the method includes:

[0037] S202: Acquire an image to be detected containing text information;

[0038] S204: using a neural network-based text localization model to generate a mask image for locating text information in the image to be detected according to the image to be detected; and

[0039] S206: Determine the position information of the text area containing text information in the image to be detected based on the mask image, wherein the operation of constructing the text localization model includes: determining a generator network of a generative adversarial network used to construct the text localization model; and reducing at least a portion of the network parameters of the generator network according to a preset ratio.

[0040] Specifically, refer to Figure 4 As shown, in this embodiment, the computing device first obtains the image to be detected containing text information. The objects to be detected are, for example but not limited to: identity cards, bank cards, driving licenses, social security cards, bills, invoices, etc. Figure 4 As shown, the target object of this embodiment is an ID card, and the image to be detected is an ID card. The computing device first detects the ID card through the ID card detection algorithm and removes images that are not ID cards. In this process, the ID card images rotated 90°, 180°, and 270° are changed to the positive direction, and then the ID card images are uniformly scaled to a size of 256*256.

[0041] Furthermore, the computing device uses the text localization model of the neural network to generate a mask image for locating text information in the image to be detected based on the image to be detected. Figure 3 and 4 As shown, the computing device uses a neural network text localization model to generate a mask image based on the ID card image to be detected, which is used to locate the text information in the ID card image. The mask image is a 256*256 ID card area, with the text area in white and the rest in black. The text area includes, for example, the text area for name, gender, ethnicity, date of birth, address, and ID card number, as well as the text area for the issuing authority and expiration date.

[0042] Furthermore, the computing device determines the location of text areas within the ID card image to be inspected based on the mask image. This allows the location of text areas within the ID card image to be determined. For example, the computing device determines the location of text areas within the ID card image to be inspected based on the mask image. For example, the computing device determines the location of text areas containing name, gender, ethnicity, date of birth, address, ID card number, issuing authority, and expiration date.

[0043] The text localization model is constructed by determining a generator network of a generative adversarial network used to construct the text localization model, and then the computing device reduces at least a portion of the network parameters of the generator network according to a preset ratio.

[0044] For example, Figure 5 FIG. 4 shows a schematic diagram of a generative adversarial network for building a text localization model. Figure 5 The generated adversarial network shown in FIG can be, for example, a pix2pix network. In this embodiment, the text localization model used can be, for example, Figure 5 Based on the generator network shown, at least a portion of the network parameters of the generator network are reduced according to a preset ratio, thereby forming a text localization model.

[0045] As mentioned in the background, existing technologies for locating ID card text using neural networks often employ target location models or target segmentation models to locate text areas. This approach suffers from complex network structures, large models, and difficulty in mobile application. Furthermore, it suffers from slow positioning speeds, low accuracy, insufficient robustness, and susceptibility to noise.

[0046] To address this technical issue, this embodiment improves the text localization model used to generate mask images. During the construction of the text localization model, a generator network of a generative adversarial network used to construct the text localization model is first determined, and then at least a portion of the network parameters of the generator network are scaled down according to a preset ratio.

[0047] Compared to existing technologies, the text localization model is generated by reducing at least a portion of the network parameters of the generator network, simplifying the text localization model. This also reduces the computational complexity of the text localization model, thus meeting the requirements of mobile devices. Due to the simple network structure, localization speeds are also in the millisecond range, meeting the real-time requirements of mobile devices.

[0048] Furthermore, since the text localization model is built based on a generator network, the corresponding discriminator network can be used to train the generator network. This allows the discriminator network to achieve high accuracy and robustness even after reducing at least some of the network parameters, making it less susceptible to noise interference.

[0049] This solves the technical problems in the existing technology of using neural networks to locate ID card text, such as complex network structure and too large model, which makes it difficult to apply to mobile terminals, as well as slow positioning speed, low accuracy, insufficient robustness and susceptibility to noise interference.

[0050] Optionally, the operation of reducing at least a portion of the network parameters of the generator network according to a preset ratio includes: reducing the NGF parameters of the convolutional layer and the deconvolution layer of the generator network according to a preset ratio.

[0051] Specifically, the computing device reduces at least a portion of the network parameters of the generator network according to a preset ratio. The computing device may also reduce the ngf parameters of the convolutional layer and the deconvolutional layer of the generator network according to a preset ratio. For example, referring to Figure 5 As shown, the computing device reduces the NGF parameter in the original generative adversarial network from 64 to 16 based on a preset ratio, thus reducing some parameters. The NGF parameter can also be 32, 64, etc. This reduces the NGF parameter in the generator network and uses a lightweight convolutional approach, effectively reducing the model size and facilitating deployment for mobile ID card OCR recognition.

[0052] Optionally, the operation of reducing at least a portion of the network parameters of the generator network according to a preset ratio includes: reducing the size of the convolution kernels of the convolution layer and the deconvolution layer of the generator network according to a preset ratio.

[0053] Specifically, the computing device reduces at least a portion of the network parameters of the generator network according to a preset ratio. For example, the computing device can reduce the size of the convolution kernels of the convolution layer and deconvolution layer of the generator network according to a preset ratio. For example, the computing device changes the original 4*4 convolution kernel of the generative adversarial network to 3*3 according to a preset ratio. This reduces the size of the convolution kernels of the convolution layer and deconvolution layer in the generator network, and uses a lightweight convolution method to effectively reduce the model size, facilitating the deployment of mobile ID card OCR recognition.

[0054] Optionally, the operation of constructing the text localization model further includes: converting at least a portion of the convolutional layers and / or deconvolutional layers in the generator network into depth-separable convolutional layers and / or deconvolutional layers.

[0055] Specifically, when constructing the text localization model, the computing device may also convert at least a portion of the convolution and / or deconvolution in the generator network into depth-separable convolution and / or deconvolution. Figure 4 As shown, Figure 5 All convolutions and deconvolutions in the model are replaced with depthwise separable convolutions. This is because depthwise separable convolutions can reduce the number of parameters by about 1 / 9 compared to ordinary convolutions. This greatly reduces the size of the convolutions, making it easier to deploy mobile ID card OCR recognition.

[0056] Optionally, the method further includes: training the generator network using a discriminator network of a generative adversarial network.

[0057] Specifically, the computing device uses the discriminator network of the generative adversarial network to train the generator network. Due to the mutual verification mechanism between the generator and the discriminator, the target can be well located, making the text localization effect more adaptable.

[0058] According to the first aspect of this embodiment, since the text localization model is generated by reducing at least a portion of the network parameters of the generator network, the text localization model is simplified. Furthermore, the computational complexity of the text localization model is reduced, thereby meeting the requirements of mobile devices. Due to the simple network structure, the localization speed is also in the millisecond range, meeting the real-time requirements of mobile devices. Furthermore, since the text localization model is constructed based on the generator network, the generator network can be trained using the corresponding discriminator network. Thus, the discriminator network enables the generator network to achieve high accuracy and robustness even after reducing at least a portion of the network parameters, making it less susceptible to noise interference. This solves the technical problems in the prior art of using neural networks to locate ID card text, such as complex network structures and overly large models that are difficult to apply to mobile devices, as well as slow localization speeds, low accuracy, insufficient robustness, and susceptibility to noise interference.

[0059] Furthermore, according to a second aspect of this embodiment, a method for recognizing text information in an image is provided. Figure 3 A schematic diagram of the process is shown in FIG. Figure 3 As shown, the method includes:

[0060] S302: Determine position information of a text area containing text information in the image to be detected;

[0061] S304: Determine the text area in the image to be detected according to the position information; and

[0062] S306: Recognize text information in the text area.

[0063] Specifically, the computing device determines the location information of a text region containing text information in the ID card image to be detected. It can also determine the text region in the ID card image to be detected based on the location information and identify the text information in the text region. To obtain the coordinates of the text region, the computing device uses a connected domain analysis method based on the ID card image output by the text localization model to obtain the coordinates of the four vertices of the text region. After binarization, opening and closing operations, and obtaining the maximum connected domain, the coordinate position of the text region is obtained. The text region (ROI region) is cropped based on the location coordinates and a scaling ratio, and the text information is then identified in the text region.

[0064] According to the second aspect of this embodiment, the computing device locates the text information in the ID card image to be detected, determines the position information of the text area containing the text information, then determines the text area at the position information, and finally recognizes the text information in the text area, thereby obtaining the text information in the image to be detected.

[0065] In addition, reference Figure 1 As shown, according to a third aspect of this embodiment, a storage medium is provided, wherein the storage medium includes a stored program, wherein when the program is run, a processor executes any one of the above methods.

[0066] In addition, the following Figure 6 The flowchart shown is used to supplement the present embodiment:

[0067] Preprocess the training data: Before preprocessing, the ID card detection algorithm needs to be used to detect the ID card and remove the images that are not ID cards. In this process, the ID card images rotated 90°, 180°, and 270° will be turned into positive images, and then the images will be uniformly scaled to 256*256 size to obtain Figure 4 , and then Figure 4 The two images are used as input images. In actual applications, the present invention annotates 5,000 training samples, then performs data augmentation (including rotation, perspective transformation, illumination change, etc.) to 50,000 images, of which 2,000 are reserved for the test set. The original image and the annotated ground truth image are combined to form the training set.

[0068] Send it to the positioning network for text positioning: The positioning network of the present invention draws on the structure of the generative adversarial network pix2pix and performs a simplified transformation. The network structure is as follows Figure 5As shown in the figure, the training sample is first input into the Generator network, which undergoes 3 layers of convolution and 3 layers of deconvolution to finally output a 256*256*3 image. ngf is a parameter in the Generator network, which can be 32, 64, etc. Secondly, the image output by the Generator is sent to the Discriminator network, which undergoes 5 layers of convolution to finally output whether it is true (the real image is true) or false (the one generated by the Generator is false). L1 loss is used as the objective function, and the model is iterated until convergence. ndf is a parameter in the Discriminator, which can be 16, 32, 64, etc. The output of the positioning network is shown in the figure. Figure 4 As shown in the right image, the text area is white and other areas are black.

[0069] In order to facilitate mobile terminal deployment, the size of the network model needs to be reduced. Simply streamlining the network is not enough. Therefore, the present invention makes three changes: First, Figure 5 All ordinary convolutions such as CONV and DECONV are replaced with depthwise separable convolutions. The use of depthwise separable convolutions can reduce the number of parameters by about 1 / 9 compared to ordinary convolutions. Secondly, the ngf and ndf in the original pix2pix network are both 64, while the present invention uses 16, reducing some parameters. Thirdly, the convolution kernels of pix2pix are all 4*4, while the present invention changes them to 3*3. These three changes greatly reduce the model parameters, and the final model size is only about 350KB, which meets the requirements of the mobile terminal.

[0070] Connected domain analysis: For the image output by the positioning model, in order to obtain the coordinates of the text area, it is necessary to use the connected domain analysis method to obtain the coordinates of the four vertices of the text area. After binarization, opening and closing operations, and obtaining the maximum connected domain, the coordinate position of the text area can be obtained. According to the position coordinates and the scaling ratio, the text block area (ROI area) is cropped as the input for subsequent OCR recognition.

[0071] Therefore, according to this embodiment, since the text positioning model is generated by reducing at least a portion of the network parameters of the generator network, the text positioning model is also simplified. The computational complexity of the text positioning model is also reduced. This meets the requirements of the mobile terminal. Due to the simple network structure, the positioning speed is also at the millisecond level, meeting the real-time requirements of the mobile terminal. In addition, since the text positioning model is constructed based on the generator network, the generator network can be trained using the corresponding discriminator network. Therefore, the discriminator network can enable the generator network to achieve high accuracy and robustness even after reducing at least a portion of the network parameters, and thus is not easily affected by noise interference. This solves the technical problems in the prior art of using neural networks to locate ID card text, such as complex network structure and excessively large models, which are difficult to apply to mobile terminals, as well as slow positioning speed, low accuracy, insufficient robustness, and susceptibility to noise interference.

[0072] It should be noted that for the aforementioned method embodiments, for simplicity of description, they are all expressed as a series of action combinations. However, those skilled in the art should be aware that the present invention is not limited by the order of the actions described, because according to the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0073] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0074] Example 2

[0075] Figure 8 The apparatus 800 for locating text information in an image according to the first aspect of this embodiment is shown. The apparatus 800 corresponds to the method according to the first aspect of embodiment 1. Figure 8As shown, the device 800 includes: an acquisition module 810, used to acquire an image to be detected containing text information; a generation module 820, used to use a neural network-based text localization model to generate a mask image for locating text information in the image to be detected according to the image to be detected; and a determination module 830, used to determine the position information of the text area containing text information in the image to be detected according to the mask image, wherein the generation module 820 includes: a determination submodule, used to determine the generator network of the generative adversarial network used to construct the text localization model; and a reduction submodule, used to reduce at least a part of the network parameters of the generator network according to a preset ratio.

[0076] Optionally, the reduction submodule includes: a first reduction unit, used to reduce the NGF parameters of the convolution layer and the deconvolution layer of the generator network according to a preset ratio.

[0077] Optionally, the reduction submodule further includes: a second reduction unit, configured to reduce the size of the convolution kernels of the convolution layer and the deconvolution layer of the generator network according to a preset ratio.

[0078] Optionally, the generation module 820 further includes: converting at least a portion of the convolutional layers and / or deconvolutional layers in the generator network into depth-separable convolutional layers and / or deconvolutional layers.

[0079] Optionally, the apparatus 800 further includes: training the generator network using a discriminator network of a generative adversarial network.

[0080] also, Figure 9 The apparatus 900 for recognizing text information in an image according to the second aspect of this embodiment is shown. The apparatus 900 corresponds to the method according to the second aspect of embodiment 1. Figure 9 As shown, the device 900 includes: a first determination module 910, used to determine the position information of a text area containing text information in the image to be detected; a second determination module 920, used to determine the text area in the image to be detected based on the position information; and an identification module 930, used to identify text information in the text area.

[0081] Thus, according to this embodiment, the apparatus 800 for locating text information in an image and the apparatus 900 for recognizing text information in an image simplify the text localization model because it is generated by reducing at least a portion of the network parameters of the generator network. This also reduces the computational complexity of the text localization model, thereby meeting the requirements of mobile devices. Due to the simple network structure, localization speed is also in the millisecond range, meeting the real-time requirements of mobile devices. Furthermore, because the text localization model is constructed based on the generator network, the generator network can be trained using the corresponding discriminator network. Consequently, the discriminator network enables the generator network to achieve high accuracy and robustness even after reducing at least a portion of the network parameters, making it less susceptible to noise interference. This solves the technical problems of prior art using neural networks to locate ID card text, such as complex network structures and excessively large models that make them difficult to apply to mobile devices, as well as slow localization speeds, low accuracy, insufficient robustness, and susceptibility to noise interference.

[0082] Example 3

[0083] Figure 10 The apparatus 1000 for locating text information in an image according to the first aspect of this embodiment is shown. The apparatus 1000 corresponds to the method according to the first aspect of embodiment 1. Figure 10 As shown, the device 1000 includes: a processor 1010; and a memory 1020, which is connected to the processor 1010 and is used to provide the processor 1010 with instructions for processing the following processing steps: obtaining an image to be detected containing text information; using a neural network-based text localization model to generate a mask image for locating text information in the image to be detected according to the image to be detected; and determining the position information of the text area containing text information in the image to be detected based on the mask image, wherein the operation of constructing the text localization model includes: determining a generator network of a generative adversarial network used to construct the text localization model; and reducing at least a part of the network parameters of the generator network according to a preset ratio.

[0084] Optionally, the operation of reducing at least a portion of the network parameters of the generator network according to a preset ratio includes: reducing the NGF parameters of the convolutional layer and the deconvolution layer of the generator network according to a preset ratio.

[0085] Optionally, the operation of reducing at least a portion of the network parameters of the generator network according to a preset ratio also includes: reducing the size of the convolution kernels of the convolution layer and deconvolution layer of the generator network according to a preset ratio.

[0086] Optionally, the operation of constructing the text localization model further includes: converting at least a portion of the convolutional layers and / or deconvolutional layers in the generator network into depth-separable convolutional layers and / or deconvolutional layers.

[0087] Optionally, the memory 1020 is further configured to provide the processor 1010 with instructions for processing the following processing steps: training the generator network using the discriminator network of the generative adversarial network.

[0088] Figure 11 The apparatus 1100 for locating text information in an image according to the second aspect of this embodiment is shown. The apparatus 1100 corresponds to the method according to the second aspect of embodiment 1. Figure 11 As shown, the apparatus 1100 includes: determining position information of a text region containing text information in an image to be detected; determining the text region in the image to be detected based on the position information; and recognizing text information in the text region.

[0089] Therefore, according to this embodiment, the apparatus 1000 for locating text information in an image and the apparatus 1100 for recognizing text information in an image simplify the text localization model because it is generated by reducing at least a portion of the network parameters of the generator network. This also reduces the computational complexity of the text localization model, thereby meeting the requirements of mobile devices. Due to the simple network structure, the localization speed is also in the millisecond range, meeting the real-time requirements of mobile devices. Furthermore, because the text localization model is constructed based on the generator network, the generator network can be trained using the corresponding discriminator network. Consequently, the discriminator network enables the generator network to achieve high accuracy and robustness even after reducing at least a portion of the network parameters, making it less susceptible to noise interference. This solves the technical problems of prior art using neural networks to locate ID card text, such as complex network structures and excessively large models that make them difficult to apply to mobile devices, as well as slow localization speeds, low accuracy, insufficient robustness, and susceptibility to noise interference.

[0090] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0091] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0092] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0093] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0094] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0095] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk, etc., various media that can store program code.

[0096] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A method for locating text information in an image, characterized in that: include: Obtain an image to be detected containing text information; Using a neural network-based text localization model, a mask image for locating text information in the image to be detected is generated according to the image to be detected; as well as Determining, based on the mask image, position information of a text region containing the text information in the image to be detected, wherein the operation of constructing the text localization model includes: Determining a generator network of a generative adversarial network for constructing the text localization model; and Reducing at least a portion of network parameters of the generator network according to a preset ratio; The operation of reducing at least a portion of the network parameters of the generator network according to a preset ratio includes: Reduce the NGF parameters of the convolutional layer and deconvolution layer of the generator network according to a preset ratio; The operation of reducing at least a portion of the network parameters of the generator network according to a preset ratio further includes: Reduce the size of the convolution kernels of the convolution layer and deconvolution layer of the generator network according to a preset ratio; The operation of constructing the text positioning model further includes: Converting at least a portion of the convolutional layers and / or deconvolutional layers in the generator network into depth-separable convolutional layers and / or deconvolutional layers; The method also includes: training the generator network using a discriminator network of a generative adversarial network.

2. A method for identifying text information in an image, characterized in that: include: Determining position information of a text region containing text information in the image to be detected by the operation according to claim 1; Determining the text area in the image to be detected according to the position information; and The text information is recognized in the text area.

3. A storage medium, characterized in that: The storage medium includes a stored program, wherein when the program is run, the processor executes the method according to any one of claims 1 to 2.

4. A device for locating text information in an image, characterized in that: include: An acquisition module is used to acquire an image to be detected containing text information; A generating module, configured to generate a mask image for locating text information in the image to be detected based on the image to be detected using a text localization model based on a neural network; as well as a determination module, configured to determine, based on the mask image, location information of a text region containing the text information in the image to be detected, wherein the generation module comprises: a determination submodule, configured to determine a generator network of a generative adversarial network for constructing the text localization model; and a reduction submodule, configured to reduce at least a portion of network parameters of the generator network according to a preset ratio; Determine the module specifically for: Reduce the NGF parameters of the convolutional layer and deconvolution layer of the generator network according to a preset ratio; Reduce the size of the convolution kernels of the convolution layer and deconvolution layer of the generator network according to a preset ratio; Converting at least a portion of the convolutional layers and / or deconvolutional layers in the generator network into depth-separable convolutional layers and / or deconvolutional layers; The determination module is further used to: train the generator network using a discriminator network of a generative adversarial network.

5. A device for locating text information in an image, characterized in that: include: a first processor; as well as A first memory is connected to the first processor and is configured to provide the first processor with instructions for processing the following processing steps: Obtain an image to be detected containing text information; Using a neural network-based text localization model, a mask image for locating text information in the image to be detected is generated according to the image to be detected; as well as Determining, based on the mask image, position information of a text region containing the text information in the image to be detected, wherein the operation of constructing the text localization model includes: Determining a generator network of a generative adversarial network for constructing the text localization model; and Reducing at least a portion of network parameters of the generator network according to a preset ratio; The first memory is further configured to provide the first processor with instructions for processing the following steps: Reduce the NGF parameters of the convolutional layer and deconvolution layer of the generator network according to a preset ratio; Reduce the size of the convolution kernels of the convolution layer and deconvolution layer of the generator network according to a preset ratio; Convert at least a portion of the convolutional layers and / or deconvolutional layers in the generator network into depth-separable convolutional layers and / or deconvolutional layers; The generator network is trained using a discriminator network of a generative adversarial network.

Images