Image text detection method, device and equipment and readable storage medium

Abstract

The application discloses a kind of image text detection method, device, equipment and readable storage medium, the method includes: obtaining image to be detected, wherein, image to be detected contains text content and non-text content;By at least two kinds of preprocessing algorithm, feature processing is carried out to image to be detected, respectively obtain different types of feature image, wherein, preprocessing algorithm includes binary algorithm and mser algorithm, preprocessing algorithm and feature image are uniquely corresponding;All feature images and image to be detected are fused by features, and obtain multi-channel feature image;Multi-channel feature image is input into the text detection model trained to carry out identification, and the position information of text content is obtained.Based on the feature fusion of original image and binary image, mser algorithm processed image, and the image recognition of fused image, on the basis of retaining all features of original image, the situation that independent character appears false detection is avoided, and the accuracy of image text recognition result is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, and particularly relates to an image text detection method and device, equipment and a readable storage medium. BACKGROUND

[0002] The purpose of text detection is to obtain the position of characters in a picture, which is a pre-step of OCR (Optical Character Recognition). The more accurate the position of the text is detected, the more accurate the subsequent text recognition result will be.

[0003] The prior art usually includes two methods based on traditional image processing methods and deep learning methods.

[0004] Specifically:

[0005] 1. The traditional image processing method is usually used for relatively simple scenes, such as book scanning, screen capture, or other high-definition pictures. This processing method includes two implementation ideas:

[0006] 1.1. Apply morphological methods in image processing in computer vision, and the implementation steps are as follows:

[0007] a. Image preprocessing: denoising operation is performed on the image to improve the robustness of the method.

[0008] b. Binaryzation operation is performed on the image.

[0009] c. Image morphological transformation, dilation and erosion operation; dilation can connect the broken text area, and erosion can remove the relatively small noise in the image.

[0010] d. Connected domain processing is performed on the binaryzation image to form a text box, and then the text box belonging to the text is preliminarily screened according to the size of the text box.

[0011] e. The adjacent text boxes are connected into a text line, and finally the region position of the text line is output.

[0012] 1.2. The method of combining MSER (Maximally Stable Extremal Regions) and NMS (Non-Maxima Suppression) is used, and the implementation steps are as follows:

[0013] a. MSER processing: using the watershed idea to detect the image spots to obtain a plurality of text boxes that may be text.

[0014] b. Screening text area: preliminarily screening the text box belonging to the text according to the size of the text box.

[0015] c. NMS processing: the text boxes obtained in the second step are sorted according to the scores, and then the box with the highest score is retained, and then the remaining boxes are traversed to find the boxes with an overlap area (IOU) greater than a certain threshold with the current highest score box, and are deleted. Then continue this process to find another high-score box, and delete the boxes with an IOU greater than the threshold, and continue to loop until all boxes are processed.

[0016] d. Connect adjacent text boxes into text lines, and finally output the area position of the text lines.

[0017] The traditional image processing method has a high requirement for scene clarity: the method contains many parameters, such as different binarization methods, binarization thresholds, and text box size limits. If a more accurate detection effect is required, specific thresholds need to be matched for each scene. However, the actual scene has strong randomness, and the specific threshold set by humans cannot completely match the actual scene, and the universality is poor.

[0018] 2. The text detection method based on the deep learning model is to modify the bounding box regression module of the general object detection method to locate the text. The representative method is the TextBox algorithm, and its main steps are as follows (inference stage):

[0019] a. The feature extraction layer (backbone) extracts features from the input image;

[0020] b. The text-box layer extracts feature maps of different sizes;

[0021] c. Use the feature map to regress candidate boxes of different sizes;

[0022] d. Apply NMS (non-maximum suppression) to the candidate boxes output in the previous step to obtain the final detection result.

[0023] However, the above two text detection methods based on deep learning models have the following defects: a. A large number of training samples are required as supervision information during the training stage, and the cost of manual labeling and time cost of a large number of training samples is large. b. For isolated characters, especially 1, i, and other characters with small width or height, the feature learning for pixel classification is difficult due to small text and model downsampling, which easily leads to missed detection, resulting in low accuracy of text detection results.

[0024] There is no effective solution to the technical problem of low accuracy of text detection results caused by missing detection of isolated characters in the image in the prior art. SUMMARY

[0025] The application aims to provide an image text detection method, device, equipment and readable storage medium, which can solve the technical problem of low accuracy of text detection results caused by missing detection of isolated words in an image in the prior art.

[0026] An aspect of the application provides an image text detection method, which comprises: acquiring an image to be detected, wherein the image to be detected contains text content and non-text content; performing feature processing on the image to be detected through at least two preprocessing algorithms to obtain different types of feature images, wherein the preprocessing algorithms include a binarization algorithm and an mser algorithm, and the preprocessing algorithms are uniquely corresponding to the feature images; performing feature fusion on all the feature images and the image to be detected to obtain a multi-channel feature image; and inputting the multi-channel feature image into a trained text detection model for recognition to obtain position information of the text content.

[0027] Optionally, the feature fusion of all the feature images and the image to be detected to obtain the multi-channel feature image comprises: splicing scale channels of all the feature images and the image to be detected to obtain the multi-channel feature image; and / or adding corresponding elements at the same position of all the feature images and the image to be detected to obtain the multi-channel feature image.

[0028] Optionally, the text detection model is trained in the following manner: a batch of sample images are acquired, and the text content in the sample images is labeled to obtain sample images with text position labels; the sample images are processed through at least two preprocessing algorithms to obtain different types of sample feature images; the sample feature images and the corresponding sample images are fused to obtain multi-channel sample feature images; the multi-channel sample feature images are input into an initial deep learning network model for training to obtain predicted position information of the text content in the sample images; based on a loss value of the predicted position information and the text position labels, the parameters of the deep learning model are adjusted and the training is continued until the loss value converges, and the training is stopped to obtain the text detection model.

[0029] Optionally, the multi-channel sample feature image is input into an initial deep learning network model for training to obtain prediction position information of text content in the sample image, including: performing feature extraction on the multi-channel sample feature image through a backbone layer of the deep learning network to obtain multiple local features of the multi-channel sample feature image; performing up-sampling through a neck layer to obtain local features of the same size, and performing feature concatenation on the local features to obtain fused features; and performing prediction on the fused features through a head layer to obtain a training text probability map and a training text threshold map, wherein the training text probability map is a first closed region composed of text pixel points, and the training text threshold map is obtained by: after respectively expanding the boundary box of the first closed region inward and outward by n pixels, corresponding to obtain a contraction box and an expansion box, determining a middle region of the contraction box and the expansion box, and determining a second closed region composed of pixel points closest to the boundary box of the first closed region from the middle region.

[0030] Optionally, the multi-channel feature image is input into the trained text detection model for recognition to obtain the position information of the text content, including: performing feature recognition on the multi-channel feature image through the backbone layer, the neck layer and the head layer of the text prediction model to obtain a text probability map; and calculating the text position information of the multi-channel feature image through the text probability map.

[0031] Optionally, after obtaining the to-be-detected image, the method further includes: enhancing image properties of the to-be-detected image according to a preset first image enhancement algorithm, the image properties at least including one of the following: image brightness, image chroma, image contrast, image sharpness and image resolution; and enhancing image quality of the to-be-detected image according to a preset second image enhancement algorithm, the second image enhancement algorithm at least including one of the following: a Gaussian blur enhancement algorithm, a motion blur enhancement algorithm and a Gaussian noise enhancement algorithm.

[0032] Another aspect of the present application provides a device for detecting image text, the device comprising: an acquisition module configured to acquire a to-be-detected image, wherein the to-be-detected image contains text content and non-text content; a preprocessing module configured to perform feature processing on the to-be-detected image through at least two preprocessing algorithms to obtain different types of feature images, wherein the preprocessing algorithms include a binary algorithm and an mser algorithm, and the preprocessing algorithms are uniquely corresponding to the feature images; a feature fusion module configured to perform feature fusion on all the feature images and the to-be-detected image to obtain a multi-channel feature image; and an identification module configured to input the multi-channel feature image into a trained text detection model for recognition to obtain position information of the text content.

[0033] Optionally, the feature fusion module is specifically configured to: splice the scale channels of all the feature images and the to-be-detected image to obtain a multi-channel feature image; and / or add corresponding elements of the same positions of all the feature images and the to-be-detected image to obtain a multi-channel feature image.

[0034] Still another aspect of the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the image text detection method of any of the above embodiments when executing the computer program.

[0035] Still another aspect of the present application provides a computer storage medium having a computer program stored thereon, wherein the computer program is executable on a processor to implement the image text detection method of any of the above embodiments. Further, the computer readable storage medium can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs required by at least one function, etc.; and the data storage area can store data created according to the use of the blockchain node, etc.

[0036] The present application fuses features of the original image, the binarized image and the image processed by the mser algorithm, and performs image recognition on the fused image, thereby avoiding the false detection of independent characters and improving the accuracy of the image text recognition result on the basis of retaining all features of the original image. BRIEF DESCRIPTION OF DRAWINGS

[0037] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included to provide a description of preferred embodiments, and are not meant to limit the present application. Furthermore, the same reference numerals are used throughout the several drawings to refer to the same or like parts. In the drawings:

[0038] Figure 1 An optional flowchart of the image text detection method provided by the embodiment one of the present application is shown;

[0039] Figure 2 An optional flowchart of the training phase of the actual example of the image text detection method provided by the embodiment two of the present application is shown;

[0040] Figure 3 An optional flowchart of the inference phase of the actual example of the image text detection method provided by the embodiment two of the present application is shown;

[0041] Figure 4 A structure block diagram of the image text detection device provided by the embodiment three of the present application is shown; and

[0042] Figure 5 A block diagram of a computer device suitable for implementing the image text detection method according to the fourth embodiment of the present application is shown. DETAILED DESCRIPTION

[0043] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0044] It should be noted that in this document, the terms "comprise", "contain" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "comprises a" does not exclude the presence of another identical element in the process, method, article or device comprising the element.

[0045] Embodiment One

[0046] The present embodiment provides an image text detection method, Figure 1 A flowchart of the image text detection method is shown, as shown in Figure 1 The image text detection method can include steps S101 to S104, wherein:

[0047] Step S101, obtaining an image to be detected, wherein the image to be detected contains text content and non-text content;

[0048] The image to be detected is obtained by an image acquisition device, wherein the acquisition device can include a commonly used device such as a mobile phone camera, a high-speed camera, a camera, a scanner, a screen capture, etc.

[0049] Step S102, performing feature processing on the image to be detected by at least two preprocessing algorithms to obtain different types of feature images, wherein the preprocessing algorithms include a binarization algorithm and an mser algorithm, and the preprocessing algorithms are uniquely corresponding to the feature images;

[0050] When the image to be detected is obtained, feature processing is performed on the image to be detected by at least two preprocessing algorithms to obtain multiple types of feature images, wherein each preprocessing algorithm will obtain a corresponding feature image. In this actual example, in particular, this example does not limit the technical solution range of the present application.

[0051] The normalization operation is performed on the to-be-detected image, and a normalized picture is obtained after normalization to a uniform size. For example, the size of the normalized picture is 640*640*3 (W*H*C, where W is the picture width, H is the picture height, and C is the picture scale, which is usually 3).

[0052] The normalized image is subjected to a binarization operation to obtain a binarized picture. For example, the size of the binarized picture is 640*640*1.

[0053] The normalized image is subjected to an mser operation to obtain an mser picture. The text region in the mser picture is drawn into a new picture with all pixels being 0, where the text region is 1 and the background is 0. For example, the size of the image after mser processing is 640*640*1.

[0054] It should be noted that the pre-processing algorithm in the present application does not only include the binarization algorithm and the mser algorithm, but can also be a conventional image processing method, which is not limited herein.

[0055] In step S103, all feature images and the to-be-detected image are subjected to feature fusion to obtain a multi-channel feature image.

[0056] In this step, different types of images are subjected to feature fusion, i.e., the normalized to-be-detected image, the feature image obtained by the binarization algorithm, and the feature image obtained by the mser algorithm are subjected to feature fusion to obtain a multi-channel feature image. This step aims to retain the features of different types of images, maintain the integrity and diversity of the image features, avoid missing individual features of the to-be-detected image, and thus improve the accuracy of graphic text recognition.

[0057] In step S104, the multi-channel feature image is input into the trained text detection model for recognition to obtain the position information of the text content.

[0058] The multi-channel feature image containing different dimensional information is input into the trained text detection model for recognition, and thus the position information of the text content in the to-be-detected image can be obtained, effectively solving the problem of missing isolated text lines and improving the recall rate of the text detection model.

[0059] Preferably, step S103 can include steps S1031 to S1032, wherein:

[0060] In step S1031, the scale channels of all feature images and the to-be-detected image are spliced to obtain a multi-channel feature image; and / or

[0061] In the case of keeping the element value of each channel of the feature image and the image to be detected unchanged, the scale channel is directly superimposed or spliced, which is essentially equivalent to integrating multiple images into one image, but the features of each image still do not interfere with each other and exist independently. In the example of the above step S103, the final multi-channel image feature has a size of 640*640*5.

[0062] In step S1032, the elements at the same position of all feature images and the image to be detected are added to obtain a multi-channel feature image.

[0063] In addition, the feature fusion can also be realized in the following manner: in the case of keeping the channel number of the feature image and the image to be detected unchanged, the element values of the two features are added, and the feature dimension after addition does not change. This manner can reduce the operation amount of feature fusion, thereby improving the efficiency of image text recognition.

[0064] It should be noted that the above two fusion manners have no order or separate use requirement, and are randomly selected according to the actual application scene.

[0065] Preferably, the text detection model is trained in the following manner, which can include steps A1 to A4, wherein:

[0066] In step A1, a batch of sample images are obtained, and the text content in the sample images is labeled to obtain sample images with text position labels;

[0067] The text position label can be the upper left and lower right coordinate values of each pixel point contained in the text content of the sample image. The labeling manner can be labeled by artificial means or labeled by using a general position labeling model, which is not limited here.

[0068] In step A2, at least two preprocessing algorithms are used to process the sample images to obtain different types of sample feature images;

[0069] In step A3, the sample feature images and the corresponding sample images are fused to obtain a multi-channel sample feature image;

[0070] The preprocessing algorithms in the training process and the feature fusion are consistent with the processing manner of the image to be detected, which will not be described here.

[0071] In step A4, the multi-channel sample feature image is input into an initial deep learning network model for training to obtain the predicted position information of the text content in the sample image. Based on the loss value of the predicted position information and the text position label, the parameters of the deep learning model are adjusted and the training is continued until the loss value converges, and the text detection model is obtained.

[0072] The initial deep learning network model can be a DBNet network structure, wherein the model comprises a backbone layer, a neck layer, and a head layer; and the threshold map loss function can adopt a smooth-L1 loss function.

[0073] The predicted position information of the text content can be pixel point position coordinate values corresponding to the text content.

[0074] The training of the image text detection model is an iterative process, that is, a process of repeatedly adjusting the parameters of the deep learning model. When the number of iterations reaches a preset number threshold, or the loss function during iteration is less than a preset loss threshold, it is indicated that the training has met the requirements, thereby obtaining the image text detection model.

[0075] Preferably, step A4 can comprise steps A41 to A44, wherein:

[0076] Step A41, performing feature extraction on the multi-channel sample feature image through the backbone layer of the deep learning network to obtain a plurality of local features of the multi-channel sample feature image;

[0077] The backbone layer can adopt three residual modules of a resnet neural network as convolution layers to perform bottom or basic feature extraction on the multi-channel sample feature image, so as to extract all original features in the sample image in the multi-channel sample image.

[0078] The backbone layer can be replaced by other backbone network structures, such as resnet series networks, mobilenet series networks, swin transformer, etc., which are not limited herein.

[0079] Step A42, performing up-sampling through the neck layer to obtain local features of the same size, and performing feature concatenation on the local features to obtain fusion features;

[0080] For the obtained local features, the size of the local features is identified based on the corresponding channel of the neck layer, and the local features of the same size are concatenated to obtain fusion features. The feature concatenation can be a fusion of cross-modal data in the feature space through series connection, superposition or element multiplication.

[0081] The neck can be replaced by other feature fusion structures, such as FPN-PAN structure, BiFPN structure, ASFF structure, etc., which are not limited herein.

[0082] Step A43, the training text probability graph is a first closed region composed of text pixel points, and the training text threshold graph is obtained by the following method: after the boundary box of the first closed region is respectively expanded inward and outward by n pixels, the corresponding shrinkage box and expansion box are obtained, the middle region of the shrinkage box and the expansion box is determined, and a second closed region composed of the pixel points closest to the boundary box of the first closed region is determined from the middle region.

[0083] The training text probability graph is obtained by obtaining all text pixels from the original image region and outputting a closed region composed of all text pixels.

[0084] The training text threshold graph is obtained by the following method: after the boundary box of the first closed region is respectively expanded inward and outward by n pixels, the corresponding shrinkage box and expansion box are obtained, the middle region of the shrinkage box and the expansion box is determined, and a second closed region composed of the pixel points closest to the boundary box of the first closed region is determined from the middle region.

[0085] The training text probability graph and the training text threshold graph are set to more accurately locate the text position, and the two cooperate with each other to improve the accuracy of the text prediction model.

[0086] The head layer can be replaced by other feature decoding layers, such as replacing the two-dimensional deconvolution applied in DBNet with a Resize network, which is not limited here.

[0087] Step A44, the text position information in the multi-channel sample feature image is calculated by the training text probability graph and the training text threshold graph.

[0088] Specifically, the calculation process can be realized by the following formula:

[0089]

[0090] P i，j , T i，j are the pixel values of the coordinates (i, j) in the training text probability graph and the training text threshold graph, respectively, B i，j is the pixel value of the corresponding coordinate calculated.

[0091] The deep learning model of the embodiment can not only help users accurately distinguish text regions and backgrounds, but also separate closely connected text instances, improve the accuracy of image text recognition, and avoid the situation of missing independent characters.

[0092] Preferably, step S104 can include steps S1041 to S1042, wherein:

[0093] Step S1041, performing feature recognition on the multi-channel feature image through the backbone layer, neck layer and head layer of the text prediction model to obtain a text probability map;

[0094] The recognition principle of this step is the same as steps A41 to A43 in the training process, and will not be repeated here.

[0095] Step S1042, calculating the text position information of the multi-channel feature image through the text probability map.

[0096] Specifically, this calculation process can be implemented in the following way:

[0097]

[0098] P i，j respectively, the pixel value of the pixel with coordinates (i, j) in the text probability map, B i，j is the pixel value of the corresponding coordinates calculated.

[0099] Preferably, after obtaining the image to be detected, the method further includes steps B1 to B2, wherein:

[0100] Step B1, enhancing the image properties of the image to be detected according to a preset first image enhancement algorithm, the image properties including at least one of the following: image brightness, image chroma, image contrast, image sharpness and image resolution;

[0101] Step B2, enhancing the image quality of the image to be detected according to a preset second image enhancement algorithm, the second image enhancement algorithm including at least one of the following: Gaussian blur enhancement algorithm, motion blur enhancement algorithm, Gaussian noise enhancement algorithm.

[0102] Since the state values contained in the sample image and / or the image to be detected are all in ideal conditions, and the obtained image to be detected may have blur phenomenon, if the picture in ideal state is directly trained, the subsequent recognition effect will be limited, and only the image with standard state value can be recognized. The preset image enhancement algorithm is used to enhance the image from two dimensional information of image properties and image quality, thereby improving the training effect of the picture direction recognition model.

[0103] It should be noted that the purposes of the above steps B1 and B2 are to obtain the image based on the preset image enhancement algorithm after obtaining the image to be detected or the sample image, so as to improve the training effect of the image text detection model, and there is no special processing time sequence requirement, that is, in specific implementation, the above step B1 can be executed first, then the above step B2 is executed, or the above step B2 is executed first, then the above step B1 is executed, or steps B1 and B2 can be executed at the same time.

[0104] The embodiment is based on feature fusion of the original image, the binarized image and the image processed by the mser algorithm, and image recognition is performed on the fused image, thereby avoiding the false detection of independent characters on the basis of retaining all features of the original image, and improving the accuracy of the image text recognition result.

[0105] Embodiment two

[0106] Embodiment two of the present application also provides a whole process of an image text detection method. The whole process includes a training phase and an inference phase, and specifically:

[0107] The training phase is as shown in Figure 2 , and is realized by the following steps:

[0108] Step 1, the image acquisition device takes a picture, and the acquisition device can include a commonly used device such as a mobile phone, a high-speed camera, a camera, a scanner and a screen capture; a training set is constructed based on the image collected by the acquisition device.

[0109] Step 2, normalizing the collected image to obtain a normalized picture of a uniform size: for example, 640*640*3 (W*H*C, wherein W is the picture width; H is the picture height; and C is the picture scale, usually 3).

[0110] Step 3, performing binarization on the normalized image to obtain a binarized picture, and the size is 640*640*1.

[0111] Step 4, performing mser operation on the normalized image to obtain an mser picture, and drawing the obtained text region to a new pixel picture all of which are 0, wherein the text region is 1 and the background is 0, and the size is 640*640*1.

[0112] Step 5, performing feature fusion on the new images obtained in steps 2-4, and the fusion method is to splice the scale channels in the images, and the final new image feature size is 640*640*5.

[0113] Step 6, input to the DBNet classic network.

[0114] Step 6.1, obtain probability map and threshold map feature layers through backbone+neck+head layers, wherein the probability map is a first closed region composed of text pixel points, and the threshold map is obtained by the following method: after the boundary box of the first closed region is respectively expanded inward and outward by n pixels, a shrinkage box and an expansion box are obtained, the middle region of the shrinkage box and the expansion box is determined, and a second closed region composed of the pixel points closest to the boundary box of the first closed region is determined from the middle region.

[0115] Step 6.2, calculate the binary map through the differentiable module, wherein the differentiable module is obtained by the following formula, wherein Pij and Tij are pixel values of coordinates (i, j) in the probability map and the threshold map feature map, and Bij is the pixel value of the corresponding coordinate calculated.

[0116]

[0117] Step 6.3, calculate the loss of probability map, threshold map and binary map and real label

[0118] Step 7, repeat the operations 1-6 until the training is completed.

[0119] The inference stage is as shown in Figure 3 , which is realized by the following method:

[0120] Step 1, the image acquisition device takes a picture, and the acquisition device can include: mobile phone shooting, high-speed camera, camera shooting, scanner, screen capture and other common devices.

[0121] Step 2, normalize the collected image to obtain a normalized picture of uniform size: for example, 640*640*3 (W*H*C, wherein W is the picture width; H is the picture height; C is the picture size, usually 3).

[0122] Step 3, perform binaryzation operation on the normalized image to obtain a binaryzation picture, and the size is: 640*640*1.

[0123] Step 4, perform MSER operation on the normalized image to obtain an mser picture, and draw the obtained text region to a new pixel picture all of which are 0, wherein the text region is 1 and the background is 0, and the size is: 640*640*1.

[0124] Step 5, the new image obtained in step 2-4 is fused with features, the fusion method is to splice the scale channel in the image, and finally the new image feature size is: 640*640*5.

[0125] Step 6, input to DBNet classic network

[0126] Step 6.1, obtain the probability map feature layer through the backbone+neck+head layer.

[0127] Step 7, obtain the box information (text position) through the probability map feature layer.

[0128] Example three

[0129] The embodiment three of the application further provides an image text detection device corresponding to the image text detection method provided in the above embodiment one, and the corresponding technical features and technical effects will not be described in detail in this embodiment, and the related parts can be referred to the above embodiment one. Specifically, Figure 4 The structure block diagram of the image text detection device is shown. As Figure 4 shown, the image text detection device 400 includes an acquisition module 401, a preprocessing module 402, a feature fusion module 403 and an identification module 404, wherein:

[0130] The acquisition module 401 is used for acquiring a to-be-detected image, wherein the to-be-detected image contains text content and non-text content;

[0131] The preprocessing module 402 is connected with the acquisition module 401, and is used for performing feature processing on the to-be-detected image through at least two preprocessing algorithms, wherein the preprocessing algorithms include a binarization algorithm and an mser algorithm, and the preprocessing algorithms correspond to the feature images uniquely;

[0132] The feature fusion module 403 is connected with the preprocessing module 402, and is used for performing feature fusion on all feature images and the to-be-detected image to obtain a multi-channel feature image;

[0133] The identification module 404 is connected with the feature fusion module 403, and is used for inputting the multi-channel feature image into a trained text detection model for identification to obtain position information of the text content.

[0134] Optionally, the feature fusion module is specifically used for: splicing scale channels of all feature images and the to-be-detected image to obtain the multi-channel feature image; and / or adding corresponding elements at the same position of all feature images and the to-be-detected image to obtain the multi-channel feature image.

[0135] Optionally, the apparatus further comprises a training module, specifically configured to: obtain batch sample images, and label text content in the sample images to obtain sample images with text position labels; perform feature processing on the sample images by at least two pre-processing algorithms to obtain sample feature images of different types respectively; perform feature fusion on the sample feature images and corresponding sample images to obtain multi-channel sample feature images; input the multi-channel sample feature images into an initial deep learning network model for training to obtain predicted position information of the text content in the sample images, adjust parameters of the deep learning model based on a loss value of the predicted position information and the text position labels and continue training until the loss value converges, and stop training to obtain a text detection model.

[0136] Optionally, the training module is further configured to: perform feature extraction on the multi-channel sample feature images by a backbone layer of the deep learning network to obtain multiple local features of the multi-channel sample feature images; perform up-sampling by a neck layer to obtain local features of the same size, and perform feature concatenation on the local features to obtain fusion features; and perform prediction on the fusion features by a head layer to obtain a training text probability map and a training text threshold map, wherein the training text probability map is a first closed region composed of text pixel points, and the training text threshold map is obtained by: after respectively expanding a bounding box of the first closed region inward and outward by n pixels, obtaining a contracted box and an expanded box, determining a middle region of the contracted box and the expanded box, and determining a second closed region composed of pixel points closest to the bounding box of the first closed region from the middle region.

[0137] Optionally, the recognition module is specifically configured to: perform feature recognition on the multi-channel feature images by the backbone layer, the neck layer and the head layer of the text prediction model to obtain a text probability map; and calculate text position information of the multi-channel feature images by the text probability map.

[0138] Optionally, the apparatus further comprises an image enhancement module, specifically configured to: perform enhancement processing on image attributes of the to-be-detected image according to a preset first image enhancement algorithm, the image attributes at least including one of the following: image brightness, image chroma, image contrast, image sharpness and image resolution; and perform enhancement processing on image quality of the to-be-detected image according to a preset second image enhancement algorithm, the second image enhancement algorithm at least including one of the following: a Gaussian blur enhancement algorithm, a motion blur enhancement algorithm and a Gaussian noise enhancement algorithm.

[0139] Embodiment Four

[0140] Figure 5A block diagram of a computer device suitable for implementing the image text detection method according to the fourth embodiment of the present application is shown. In this embodiment, the computer device 500 can be a smartphone, a tablet computer, a notebook computer, a desktop computer, a rack server, a blade server, a tower server, or a rack-mounted server (including a single server or a server cluster composed of multiple servers), etc. that executes programs. As shown in the figure, the computer device 500 according to this embodiment includes, but is not limited to, a memory 501, a processor 502, and a network interface 503 that are communicatively connected via a system bus. It should be noted that Figure 5 only the computer device 500 with components 501-503 is shown, but it should be understood that all the components shown are not required to be implemented, and more or fewer components can be alternatively implemented. Figure 5

[0141] In this embodiment, the memory 503 includes at least one type of computer-readable storage medium, which includes a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., an SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 501 can be an internal storage unit of the computer device 500, such as a hard disk or a memory of the computer device 500. In other embodiments, the memory 501 can also be an external storage device of the computer device 500, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device 500. Of course, the memory 501 can include both the internal storage unit and the external storage device of the computer device 500. In this embodiment, the memory 501 is generally used to store an operating system and various application software installed on the computer device 500, such as program codes of the image text detection method, etc.

[0142] The processor 502 can be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip in some embodiments. The processor 502 is generally used to control the overall operation of the computer device 500. For example, it performs control and processing related to data interaction or communication of the computer device 500, etc. In this embodiment, the processor 502 is used to run the program codes of the steps of the image text detection method stored in the memory 501.

[0143] ​In the embodiment, the image text detection method stored in the memory 501 can also be divided into one or more program modules and executed by one or more processors (the processor 502 in the embodiment) to complete the present application.

[0144] The network interface 503 can include a wireless network interface or a wired network interface, which is generally used to establish a communication link between the computer device 500 and other computer devices. For example, the network interface 503 is used to connect the computer device 500 with an external terminal through a network, establish a data transmission channel and a communication link between the computer device 500 and the external terminal, and the like. The network can be an intranet, the Internet, a Global System of Mobile communication (GSM), a Wideband Code Division Multiple Access (WCDMA), a 4G network, a 5G network, Bluetooth, Wi-Fi, and the like wireless or wired network.

[0145] Embodiment five

[0146] The embodiment also provides a computer readable storage medium, including a flash memory, a hard disk, a multimedia card, a card type memory (for example, an SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read only memory (ROM), an electrically erasable programmable read only memory (EEPROM), a programmable read only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, a server, an App application market, and the like, in which a computer program is stored, and the computer program is executed by a processor to realize the steps of the image text detection method.

[0147] Obviously, those skilled in the art should understand that each module or each step of the above-mentioned embodiment of the present application can be realized by a general computing device, which can be concentrated on a single computing device or distributed on a network composed of multiple computing devices, and alternatively, each module or each step can be realized by program code executable by a computing device, so that each module or each step can be stored in a storage device and executed by a computing device, and in some cases, the steps shown or described can be executed in different order, or each module or each step can be manufactured into an individual integrated circuit module or a plurality of modules or steps can be manufactured into a single integrated circuit module. Therefore, the embodiment of the present application is not limited to any specific combination of hardware and software.

[0148] It should be noted that the serial numbers of the embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0149] Through the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be realized by means of software and necessary general hardware platforms, and of course, can also be realized by hardware, but in many cases, the former is a better embodiment.

[0150] The above are only preferred embodiments of the present application, and do not limit the patent scope of the present application, and any equivalent structure or equivalent process transformation made by using the content of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method of detecting image text, characterized by, The method comprises: acquiring an image to be detected, wherein the image to be detected contains text content and non-text content; performing feature processing on the image to be detected by at least two preprocessing algorithms to obtain different types of feature images, wherein the preprocessing algorithms include a binarization algorithm and an mser algorithm, and the preprocessing algorithms are uniquely corresponding to the feature images; performing feature fusion on all the feature images and the image to be detected to obtain a multi-channel feature image; inputting the multi-channel feature image into a trained text detection model for recognition to obtain position information of the text content; the feature fusion on all the feature images and the image to be detected to obtain a multi-channel feature image comprises: splicing scale channels of all the feature images and the image to be detected to obtain a multi-channel feature image; and / or adding corresponding elements at the same position of all the feature images and the image to be detected to obtain a multi-channel feature image.

2. The method of claim 1, wherein, The text detection model is trained in the following manner: acquiring a batch of sample images, labeling text content in the sample images to obtain sample images with text position labels; performing feature processing on the sample images by at least two preprocessing algorithms to obtain different types of sample feature images; performing feature fusion on the sample feature images and corresponding sample images to obtain multi-channel sample feature images; inputting the multi-channel sample feature images into an initial deep learning network model for training to obtain predicted position information of the text content in the sample images, adjusting parameters of the deep learning model based on a loss value of the predicted position information and the text position labels and continuing training until the loss value converges, thereby obtaining a text detection model.

3. The method of claim 2, wherein, the inputting of the multi-channel sample feature images into the initial deep learning network model for training to obtain predicted position information of the text content in the sample images comprises: performing feature extraction on the multi-channel sample feature images by a backbone layer of the deep learning network to obtain multiple local features of the multi-channel sample feature images; performing up-sampling by a neck layer to obtain local features of the same size, and performing feature concatenation on the local features to obtain fusion features; performing prediction on the fusion features by a head layer to obtain a training text probability map and a training text threshold map, wherein the training text probability map is a first closed region composed of text pixel points, and the training text threshold map is obtained in the following manner: after respectively expanding a bounding box of the first closed region inward and outward by n pixels, a contracted box and an expanded box are obtained, a middle region of the contracted box and the expanded box is determined, and a second closed region composed of pixel points closest to the bounding box of the first closed region is determined from the middle region; calculating text position information in the multi-channel sample feature images by the training text probability map and the training text threshold map.

4. The method of claim 3, wherein, The method comprises the following steps: The text prediction model comprises a backbone layer, a neck layer and a head layer. The text prediction model comprises a backbone layer, a neck layer and a head layer.

5. The method of claim 1, wherein, The text prediction model comprises a backbone layer, a neck layer and a head layer. After obtaining the image to be detected, the method further comprises: According to the preset first image enhancement algorithm, the image attribute of the image to be detected is enhanced, and the image attribute at least includes one of the following: image brightness, image chroma, image contrast, image sharpness and image resolution.

6. An image text detection apparatus characterized by comprising: According to the preset second image enhancement algorithm, the image quality of the image to be detected is enhanced, and the second image enhancement algorithm at least includes one of the following: Gaussian blur enhancement algorithm, motion blur enhancement algorithm and Gaussian noise enhancement algorithm. The device comprises: An acquisition module is configured to acquire an image to be detected, wherein the image to be detected contains text content and non-text content. A preprocessing module is configured to perform feature processing on the image to be detected by at least two preprocessing algorithms to obtain different types of feature images, wherein the preprocessing algorithms include a binary algorithm and an mser algorithm, and the preprocessing algorithms are uniquely corresponding to the feature images. A feature fusion module is configured to perform feature fusion on all feature images and the image to be detected to obtain a multi-channel feature image. An identification module is configured to input the multi-channel feature image into a trained text detection model for identification to obtain position information of the text content.

7. A computer device comprising: The feature fusion module is specifically configured to splice scale channels of all feature images and the image to be detected to obtain a multi-channel feature image, and / or add corresponding elements at the same position of all feature images and the image to be detected to obtain a multi-channel feature image.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, A memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of any one of claims 1 to 5 when executing the computer program. The computer program is executed by the processor to implement the method of any one of claims 1 to 5.

Images