Text detection method, device, readable medium and electronic device
By using semantic segmentation kernel expansion algorithm and two-way fusion feature extraction in educational scenarios, the problem of low detection accuracy caused by irregular shape of unit text areas in the picture and the adjacent and compactness of multiple unit text areas is solved, and a higher text detection accuracy.
Patent Information
- Application Number
- CN202210613091.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-31
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2042-05-31
AI Technical Summary
In educational scenarios, the unit text areas in the picture are irregular in shape and multiple unit text areas are adjacent and compact, resulting in a low accuracy of text detection for traditional image processing methods and deep learning methods.
A text detection method is adopted. By inputting the image to be detected to preset text detection model, feature extraction and bidirectional fusion, combined with the kernel expansion algorithm of semantic segmentation, gradually expanding pixels until the final predicted text area map is obtained, improving detection accuracy .
It effectively improves the accuracy of text detection of irregular shapes of unit text areas and compact pictures adjacent to multiple unit text areas, and improves the accuracy of detection.
Smart Images

Figure CN115063806B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular, to a text detection method, device, readable medium and electronic device. Background Art
[0002] In the field of education, text detection technology is widely used, for example: intelligent grading, question bank recording and arrangement, AI learning machines, etc. For example, in the question bank recording and arrangement scenario: text detection can accurately locate the position of text in the test paper, avoiding manual text entry by the recording and arrangement teacher, and effectively improving the efficiency of question bank recording and arrangement.
[0003] For images in educational scenes, the unit text area has irregular shapes and multiple unit text areas are close together. These characteristics lead to low detection accuracy when using traditional image processing methods based on connected domain contour detection or deep learning methods based on predefined anchor boxes for text detection. Summary of the invention
[0004] The purpose of the present disclosure is to provide a text detection method, device, readable medium and electronic device to improve the accuracy of text detection in pictures with irregular unit text areas and multiple unit text areas adjacent and compact.
[0005] In order to achieve the above object, the present disclosure provides a text detection method, the method comprising:
[0006] Input the image to be detected into a preset text detection model to obtain multiple predicted text area maps with different unit text area sizes; perform pixel expansion on the target predicted text area map under the supervision of the supervised predicted text area map to obtain an intermediate expansion map, wherein the target predicted text area map is a predicted text area map with the smallest unit text area size among the multiple predicted text area maps, and the supervised predicted text area map is a predicted text area map with a unit text area size ranking adjacent to the target predicted text area map among the multiple predicted text area maps, and a size larger than that of the target predicted text area map;
[0007] For the obtained intermediate expansion map, pixel expansion is performed under the supervision of the new supervised predicted text region map until the new supervised predicted text region map is the predicted text region map with the largest unit text region size among the multiple predicted text region maps, and the obtained intermediate expansion map is the final predicted text region map;
[0008] Among them, the new supervised predicted text area map is a predicted text area map in which the size of the unit text area in the multiple predicted text area maps is adjacent to the previous supervised predicted text area map and the size is larger than the previous supervised predicted text area map.
[0009] Optionally, inputting the image to be detected into a preset text detection model to obtain a plurality of predicted text area maps with different unit text area sizes includes the following steps performed based on the preset text detection model:
[0010] Perform feature extraction on the image to be detected to obtain multiple target feature maps, which are derived from different layers of feature extraction;
[0011] Performing bidirectional fusion on the multiple target feature maps to obtain multiple first fused feature maps;
[0012] For each first fused feature map, the receptive field is expanded by a 3x3 convolution kernel, two consecutive 3x3 convolution kernels, and three consecutive 3x3 convolution kernels to obtain a second fused feature map;
[0013] Upsampling and concatenating the plurality of second fused feature maps to obtain a third fused feature map, wherein the size of the third fused feature map is the same as the maximum size of the plurality of second fused feature maps;
[0014] Reducing the number of channels of the third fused feature map to obtain a fourth fused feature map;
[0015] Processing the fourth fused feature map through multiple 1x1 convolution kernels to obtain multiple fifth fused feature maps, wherein the number of the 1x1 convolution kernels is the same as the number of the predicted text area maps, and the number of channels of each of the fifth fused feature maps is 2 and the number of the fifth fused feature maps is the same as the number of the predicted text area maps;
[0016] Upsampling the plurality of fifth fused feature maps to obtain a plurality of sixth fused feature maps, wherein a size of each of the sixth fused feature maps is the same as that of the image to be detected;
[0017] A plurality of sixth fusion feature maps are predicted to obtain a plurality of predicted text area maps with different unit text area sizes.
[0018] Optionally, the preset text detection model is trained in the following manner:
[0019] For each image in the image set, all unit text areas in the image are annotated according to an annotation instruction input by a user to obtain a first annotated image, and all unit text areas in the first annotated image are indented multiple times according to different indentation distances to obtain multiple second annotated images, wherein the annotation instruction is the coordinates of all unit text areas in the image sequentially represented according to a fixed order;
[0020] Based on the image set, a multi-branch model is trained to obtain a preset text detection model.
[0021] Optionally, the preset text detection model is trained in the following manner:
[0022] For each image in the image set, all unit text areas in the image are annotated according to an annotation instruction input by a user to obtain a first annotated image, and all unit text areas in the first annotated image are indented multiple times according to different indentation distances to obtain multiple second annotated images, wherein the annotation instruction is the coordinates of all unit text areas in the image sequentially represented according to a fixed order;
[0023] Based on the image set, a multi-branch model is trained to obtain a multi-branch detection model;
[0024] The multi-branch detection model is reconstructed to obtain a single-branch detection model, and the single-branch detection model is used as a preset text detection model.
[0025] The present disclosure also provides a text detection device, the text detection device comprising:
[0026] A detection module is used to input the image to be detected into a preset text detection model to obtain multiple predicted text area maps with different unit text area sizes; a post-processing module is used to perform pixel expansion on the target predicted text area map under the supervision of a supervised predicted text area map to obtain an intermediate expansion map, wherein the target predicted text area map is a predicted text area map with the smallest unit text area size among the multiple predicted text area maps, and the supervised predicted text area map is a predicted text area map with a size order of unit text areas in the multiple predicted text area maps that is adjacent to the target predicted text area map and has a size larger than the target predicted text area map; for the obtained intermediate expansion map, pixel expansion is performed under the supervision of a new supervised predicted text area map until the new supervised predicted text area map is the predicted text area map with the largest unit text area size among the multiple predicted text area maps, and the obtained intermediate expansion map is the final predicted text area map; wherein the new supervised predicted text area map is a predicted text area map with a size order of unit text areas in the multiple predicted text area maps that is adjacent to the previous supervised predicted text area map and has a size larger than the previous supervised predicted text area map.
[0027] Optionally, the detection module is specifically used to perform feature extraction on the image to be detected based on the preset text detection model to obtain multiple target feature maps, and the multiple target feature maps are derived from different layers of feature extraction; bidirectionally fuse the multiple target feature maps to obtain multiple first fused feature maps; for each first fused feature map, the receptive field is expanded by a 3x3 convolution kernel, two consecutive 3x3 convolution kernels, and three consecutive 3x3 convolution kernels to obtain a second fused feature map; upsample and splice the multiple second fused feature maps to obtain a third fused feature map, and the size of the third fused feature map is the same as the maximum size of the multiple second fused feature maps. The number of channels of the third fused feature map is the same as that of the image to be detected; the number of channels of the third fused feature map is reduced to obtain a fourth fused feature map; the fourth fused feature map is processed by multiple 1x1 convolution kernels to obtain multiple fifth fused feature maps, wherein the number of the 1x1 convolution kernels is the same as the number of the predicted text area maps, the number of channels of each of the fifth fused feature maps is 2 and the number of the channels is the same as the number of the predicted text area maps; the multiple fifth fused feature maps are upsampled to obtain multiple sixth fused feature maps, and the size of each of the sixth fused feature maps is the same as that of the image to be detected; the multiple sixth fused feature maps are predicted to obtain multiple predicted text area maps with different unit text area sizes.
[0028] Optionally, the preset text detection model is trained in the following manner:
[0029] For each image in the image set, all unit text areas in the image are annotated according to an annotation instruction input by a user to obtain a first annotated image, and all unit text areas in the first annotated image are indented multiple times according to different indentation distances to obtain multiple second annotated images, wherein the annotation instruction is the coordinates of all unit text areas in the image sequentially represented according to a fixed order;
[0030] Based on the image set, a multi-branch model is trained to obtain a preset text detection model.
[0031] Optionally, the preset text detection model is trained in the following manner:
[0032] For each image in the image set, all unit text areas in the image are annotated according to an annotation instruction input by a user to obtain a first annotated image, and all unit text areas in the first annotated image are indented multiple times according to different indentation distances to obtain multiple second annotated images, wherein the annotation instruction is the coordinates of all unit text areas in the image sequentially represented according to a fixed order;
[0033] Based on the image set, a multi-branch model is trained to obtain a multi-branch detection model;
[0034] The multi-branch detection model is reconstructed to obtain a single-branch detection model, and the single-branch detection model is used as a preset text detection model.
[0035] The present disclosure also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which implements the steps of the above method when executed by a processor.
[0036] The present disclosure also provides an electronic device, comprising:
[0037] a memory having a computer program stored thereon;
[0038] A processor is used to execute the computer program in the memory to implement the steps of the above method.
[0039] Through the above technical solution, the image to be detected is detected and post-processed to obtain the final predicted text area map, which is a kernel expansion algorithm based on semantic segmentation. Therefore, the technical solution provided by the present disclosure can well solve the problem of low detection accuracy caused by irregular shape of unit text area and multiple unit text areas being adjacent and compact, and improve the accuracy of text detection.
[0040] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings:
[0042] Figure 1 It is a flowchart of a text detection method according to an embodiment of the present disclosure.
[0043] Figure 2 It is a flowchart of further fusing features of a first fused feature map to obtain a second fused feature map according to an embodiment of the present disclosure.
[0044] Figure 3 It is a block diagram of a text detection device according to an embodiment of the present disclosure.
[0045] Figure 4 It is a block diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION
[0046] The specific implementation of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present disclosure, and is not used to limit the present disclosure.
[0047] It should be noted that all actions of acquiring signals, information or data in the present disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the device is located and with the authorization given by the owner of the corresponding device.
[0048] The present disclosure provides a text detection method. Figure 1 FIG. 1 is a flowchart of a text detection method according to an embodiment of the present disclosure. Figure 1 As shown, the method includes:
[0049] Step S10, inputting the image to be detected into a preset text detection model to obtain a plurality of predicted text area maps with different unit text area sizes.
[0050] The unit character area can be a character row or a character column. The plurality of predicted character area maps with different unit character area sizes, that is, the number of unit character areas included in any predicted character area map is the same as the number of unit character areas included in other predicted character area maps, the shapes are similar, the sizes are different, and they correspond one to one. For example, step S10 obtains 5 predicted text area maps, namely the first predicted text area map, the second predicted text area map, the third predicted text area map, the fourth predicted text area map and the fifth predicted text area map; the first predicted text area map includes 3 unit text areas, recorded as unit text area A, unit text area B and unit text area E; then, the second predicted text area map also includes 3 unit text areas, recorded as unit text area A1, unit text area B1 and unit text area E1; then unit text area A corresponds to unit text area A1, has similar shape and different size, unit text area B corresponds to unit text area B1, has similar shape and different size, unit text area E corresponds to unit text area E1, has similar shape and different size; similarly, the third predicted text area map, the fourth predicted text area map and the fifth predicted text area map also include 3 unit text areas with similar shape, different size and one-to-one correspondence and different size, which will not be described in detail here. Step S20, pixel expansion is performed on the target predicted text area map under the supervision of the supervised predicted text area map to obtain an intermediate expansion map.
[0051] Among them, the target predicted text area map is a predicted text area map with the smallest unit text area size among multiple predicted text area maps, and the supervised predicted text area map is a predicted text area map with the unit text area sizes of multiple predicted text area maps being adjacent to the target predicted text area map and having a size larger than the target predicted text area map.
[0052] Step S30, for the obtained intermediate expansion map, pixel expansion is performed under the supervision of the new supervised predicted text area map until the new supervised predicted text area map is the predicted text area map with the largest unit text area size among the multiple predicted text area maps, and the obtained intermediate expansion map is the final predicted text area map.
[0053] Among them, the new supervised predicted text area map is a predicted text area map in which the size of the unit text area in the multiple predicted text area maps is adjacent to the previous supervised predicted text area map and the size is larger than the previous supervised predicted text area map.
[0054] Steps S20 and S30 can use a kernel expansion algorithm for pixel expansion, and the core idea of the kernel expansion algorithm is a breadth-first algorithm for undirected graphs. In order to facilitate the understanding of steps S20 and S30, the example of step S10 is used, that is, it is assumed that step S10 obtains 5 predicted text area maps, namely the first predicted text area map, the second predicted text area map, the third predicted text area map, the fourth predicted text area map and the fifth predicted text area map, and it is assumed that the unit text area size of these 5 predicted text area maps decreases in sequence, that is, the unit text area size of the fifth predicted text area map is the smallest, the unit text area size of the fourth predicted text area map is the second smallest, the unit text area size of the third predicted text area map is the third smallest, the unit text area size of the second predicted text area map is the fourth smallest, and the unit text area size of the first predicted text area map is the largest. Then step S20 and step S30 are as follows: perform pixel expansion on the fifth predicted text area map under the supervision of the fourth predicted text area map to obtain a first intermediate expansion map; perform pixel expansion on the first intermediate expansion map under the supervision of the third predicted text area map to obtain a second intermediate expansion map; perform pixel expansion on the second intermediate expansion map under the supervision of the second predicted text area map to obtain a third intermediate expansion map; perform pixel expansion on the third intermediate expansion map under the supervision of the first predicted text area map to obtain a fourth intermediate expansion map, and the fourth intermediate expansion map is the final predicted text area map.
[0055] Through the above technical solution, the image to be detected is detected and post-processed to obtain the final predicted text area map, which is a kernel expansion algorithm based on semantic segmentation. Therefore, the technical solution provided by the present disclosure can well solve the problem of low detection accuracy caused by irregular shape of unit text area and multiple unit text areas being adjacent and compact, and improve the accuracy of text detection.
[0056] Optionally, inputting the image to be detected into a preset text detection model to obtain a plurality of predicted text area maps with different unit text area sizes includes the following steps performed based on the preset text detection model:
[0057] Feature extraction is performed on the image to be detected to obtain multiple target feature maps, which are derived from different layers of feature extraction.
[0058] That is, the feature maps extracted by the backbone network based on the preset text detection model are generally divided by layers (stages), and are recorded as C1, C2, C3, C4, C5, C6, C7, etc., where the numbers are the same as the layer (stage) numbers, representing the number of times the size (resolution) is halved, such as C3 represents the feature map output by layer (stage) 3, with a resolution of 1 / 4 of the input image, and C6 represents the feature map output by layer (stage) 6, with a size (resolution) of 1 / 32 of the input image. Then multiple target feature maps are selected from feature maps C1, C2, C3, C4, C5, C6, C7, etc., for example, feature maps C3, C4, C5, C6 are selected as target feature maps.
[0059] The multiple target feature maps are bidirectionally fused to obtain multiple first fused feature maps.
[0060] Feature fusion can select the Feature Pyramid Networks (FPN) method, and then multiple target feature maps are bidirectionally fused with FPN to obtain multiple first fused feature maps, and the number of the first fused feature maps is the same as the number of the target feature maps. For example, target feature maps C3, C4, C5, and C6 are bidirectionally fused to obtain first fused feature maps P3, P4, P5, and P6.
[0061] For each first fused feature map, the receptive field is expanded by a 3x3 convolution kernel, two consecutive 3x3 convolution kernels, and three consecutive 3x3 convolution kernels to obtain a second fused feature map.
[0062] In order to achieve a higher detection accuracy for unit text regions of various sizes, this solution further performs feature fusion on each first fused feature map through a 3x3 convolution kernel, two consecutive 3x3 convolution kernels, and three consecutive 3x3 convolution kernels to obtain a second fused feature map, such as Figure 2 Through this step, the problem of large span of unit text area size in the image to be detected can be effectively solved.
[0063] The multiple second fused feature maps are upsampled and concatenated to obtain a third fused feature map, and the size of the third fused feature map is the same as the maximum size of the multiple second fused feature maps.
[0064] For example, the second feature fusion map obtained according to the first fusion feature map P3, P4, P5, P6 is recorded as P3', P4', P5', P6'. The second feature fusion map P4' is upsampled by 2 times, the second feature fusion map P5' is upsampled by 4 times, and the second feature fusion map P6' is upsampled by 8 times, and then spliced with the second feature fusion map P3' to obtain the third fusion feature map F1. The size of the third fusion feature map F1 is the same as that of the second feature fusion map P3'.
[0065] The number of channels of the third fused feature map is reduced to obtain a fourth fused feature map.
[0066] For example, the number of channels of the third fused feature map F1 can be reduced by using a 3*3 convolution kernel to obtain the fourth fused feature map F2, so as to reduce the amount of calculation. For example, the number of channels of the third fused feature map F1 is changed from 1024 to 256.
[0067] The fourth fused feature map is processed by multiple 1x1 convolution kernels to obtain multiple fifth fused feature maps, wherein the number of the 1x1 convolution kernels is the same as the number of the predicted text area maps, and the number of channels of each of the fifth fused feature maps is 2 and the number of channels is the same as the number of the predicted text area maps.
[0068] Since there are only text classes and non-text classes when performing text detection, the number of channels of the fourth fused feature map F2 can be set to 2. In addition, since multiple predicted text area maps need to be obtained, multiple convolution kernels are required to process the fourth fused feature map separately. Following the above example, since 5 predicted text area maps (first predicted text area map, second predicted text area map, third predicted text area map, fourth predicted text area map and fifth predicted text area map) need to be obtained, the fourth fused feature map F2 can be processed by 5 1x1 convolution kernels to obtain 5 fifth fused feature maps F3.
[0069] The multiple fifth fusion feature maps are upsampled to obtain multiple sixth fusion feature maps, and the size of each of the sixth fusion feature maps is the same as that of the image to be detected.
[0070] Following the above example, the size of the fifth fused feature map F3 is the same as that of the feature map C3, which is 1 / 4 of the image to be detected. Therefore, each fifth fused feature map F3 needs to be upsampled by 4 times to obtain the sixth fused feature map F4 with the same size as the image to be detected.
[0071] A plurality of sixth fusion feature maps are predicted to obtain a plurality of predicted text area maps with different unit text area sizes.
[0072] For example, according to the activation function and the loss function, multiple sixth fusion feature maps are predicted. Following the above example, five sixth fusion feature maps F4 are predicted respectively to obtain five predicted text area maps (first predicted text area map, second predicted text area map, third predicted text area map, fourth predicted text area map and fifth predicted text area map).
[0073] Through the above technical solution, a bidirectional feature pyramid network is used for feature fusion, and the single-layer fused feature map is further fused to expand the receptive field, effectively solving the problem of large span of unit text area size in the image to be detected.
[0074] Optionally, the preset text detection model is trained in the following manner:
[0075] For each image in the image set, all unit text areas in the image are annotated according to the annotation instructions input by the user to obtain a first annotated image, and all unit text areas in the first annotated image are indented multiple times according to different indentation distances to obtain multiple second annotated images. The annotation instructions are the coordinates of all unit text areas in the image represented in sequence according to a fixed order.
[0076] That is, when annotating an image, firstly, the image is annotated according to the actual size of the unit text area in the image to obtain a first annotated image. Then, the first annotated image is indented according to different indentation distances, and a second annotated image is obtained each time the indentation is performed. The calculation formula for the indentation distance can be:
[0077]
[0078] Among them, d is the indentation distance of each unit text area in the first annotated image, r is the indentation factor of each indentation, for example, if there are 4 indentations, r = 0.8, 0.6, 0.4, 0.2 can be taken to obtain 4 second annotated images, s is the area of each unit text area in the first annotated image, l is the perimeter of each unit text area in the first annotated image. Following the above example, for each image to be detected, in order to obtain 5 predicted text area maps, it is necessary to generate a first annotated image and 4 second annotated images with different indentation distances for each image during training.
[0079] Based on the image set, a multi-branch model is trained to obtain a preset text detection model.
[0080] Through the above technical solution, a multi-branch model is used for training, so that the obtained preset text detection model has strong generalization ability. For example, the multi-branch model can connect 50 multi-branch network components in series.
[0081] Optionally, the preset text detection model is trained in the following manner:
[0082] For each image in the image set, all unit text areas in the image are annotated according to the annotation instructions input by the user to obtain a first annotated image, and all unit text areas in the first annotated image are indented multiple times according to different indentation distances to obtain multiple second annotated images. The annotation instructions are the coordinates of all unit text areas in the image represented in sequence according to a fixed order.
[0083] Based on the image set, a multi-branch model is trained to obtain a multi-branch detection model.
[0084] The multi-branch detection model is reconstructed to obtain a single-branch detection model, and the single-branch detection model is used as a preset text detection model.
[0085] Through the above technical solution, a multi-branch model is used for training in the training phase, and then the network reconstruction method is used to convert the trained multi-branch detection model into a single-branch detection model. In the inference phase, the single-branch detection model after network reconstruction is directly used for model inference, thereby achieving the effect of improving the inference speed and saving hardware resources.
[0086] Based on the above inventive concept, an embodiment of the present disclosure further provides a text detection device. Figure 3 As shown, the text detection device includes:
[0087] The detection module 10 is used to input the image to be detected into a preset text detection model to obtain multiple predicted text area maps with different unit text area sizes. The post-processing module 20 is used to perform pixel expansion on the target predicted text area map under the supervision of the supervised predicted text area map to obtain an intermediate expansion map, wherein the target predicted text area map is a predicted text area map with the smallest unit text area size among the multiple predicted text area maps, and the supervised predicted text area map is a predicted text area map with a unit text area size ranking adjacent to the target predicted text area map and a size larger than the target predicted text area map; for the obtained intermediate expansion map, pixel expansion is performed under the supervision of a new supervised predicted text area map until the new supervised predicted text area map is a predicted text area map with the largest unit text area size among the multiple predicted text area maps, and the obtained intermediate expansion map is the final predicted text area map; wherein the new supervised predicted text area map is a predicted text area map with a unit text area size ranking adjacent to the previous supervised predicted text area map and a size larger than the previous supervised predicted text area map.
[0088] Through the above technical solution, the image to be detected is detected and post-processed to obtain the final predicted text area map, which is a kernel expansion algorithm based on semantic segmentation. Therefore, the technical solution provided by the present disclosure can well solve the problem of low detection accuracy caused by irregular shape of unit text area and multiple unit text areas being adjacent and compact, and improve the accuracy of text detection.
[0089] Optionally, the detection module is specifically used to perform feature extraction on the image to be detected based on the preset text detection model to obtain multiple target feature maps, and the multiple target feature maps are derived from different layers of feature extraction; bidirectionally fuse the multiple target feature maps to obtain multiple first fused feature maps; for each first fused feature map, the receptive field is expanded by a 3x3 convolution kernel, two consecutive 3x3 convolution kernels, and three consecutive 3x3 convolution kernels to obtain a second fused feature map; upsample and splice the multiple second fused feature maps to obtain a third fused feature map, and the size of the third fused feature map is the same as the maximum size of the multiple second fused feature maps. The number of channels of the third fused feature map is the same as that of the image to be detected; the number of channels of the third fused feature map is reduced to obtain a fourth fused feature map; the fourth fused feature map is processed by multiple 1x1 convolution kernels to obtain multiple fifth fused feature maps, wherein the number of the 1x1 convolution kernels is the same as the number of the predicted text area maps, the number of channels of each of the fifth fused feature maps is 2 and the number of the channels is the same as the number of the predicted text area maps; the multiple fifth fused feature maps are upsampled to obtain multiple sixth fused feature maps, and the size of each of the sixth fused feature maps is the same as that of the image to be detected; the multiple sixth fused feature maps are predicted to obtain multiple predicted text area maps with different unit text area sizes.
[0090] Through the above technical solution, a bidirectional feature pyramid network is used for feature fusion, and the single-layer fused feature map is further fused to expand the receptive field, effectively solving the problem of large span of unit text area size in the image to be detected.
[0091] Optionally, the preset text detection model is trained in the following manner:
[0092] For each image in the image set, all unit text areas in the image are annotated according to the annotation instructions input by the user to obtain a first annotated image, and all unit text areas in the first annotated image are indented multiple times according to different indentation distances to obtain multiple second annotated images. The annotation instructions are the coordinates of all unit text areas in the image represented in sequence according to a fixed order.
[0093] Based on the image set, a multi-branch model is trained to obtain a preset text detection model.
[0094] Through the above technical solution, a multi-branch model is used for training, so that the obtained preset text detection model has strong generalization ability. For example, the multi-branch model can connect 50 multi-branch network components in series.
[0095] Optionally, the preset text detection model is trained in the following manner:
[0096] For each image in the image set, all unit text areas in the image are annotated according to the annotation instructions input by the user to obtain a first annotated image, and all unit text areas in the first annotated image are indented multiple times according to different indentation distances to obtain multiple second annotated images. The annotation instructions are the coordinates of all unit text areas in the image represented in sequence according to a fixed order.
[0097] Based on the image set, a multi-branch model is trained to obtain a multi-branch detection model.
[0098] The multi-branch detection model is reconstructed to obtain a single-branch detection model, and the single-branch detection model is used as a preset text detection model.
[0099] Through the above technical solution, a multi-branch model is used for training in the training phase, and then the network reconstruction method is used to convert the trained multi-branch detection model into a single-branch detection model. In the inference phase, the single-branch detection model after network reconstruction is directly used for model inference, thereby achieving the effect of improving the inference speed and saving hardware resources.
[0100] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.
[0101] Figure 4 1 is a block diagram of an electronic device 1900 according to an exemplary embodiment. For example, the electronic device 1900 may be provided as a server. Figure 4 The electronic device 1900 includes a processor 1922, which may be one or more, and a memory 1932 for storing a computer program executable by the processor 1922. The computer program stored in the memory 1932 may include one or more modules each corresponding to a set of instructions. In addition, the processor 1922 may be configured to execute the computer program to perform the above-mentioned text detection method.
[0102] In addition, the electronic device 1900 may further include a power supply component 1926 and a communication component 1950. The power supply component 1926 may be configured to perform power management of the electronic device 1900, and the communication component 1950 may be configured to implement communication of the electronic device 1900, for example, wired or wireless communication. In addition, the electronic device 1900 may further include an input / output (I / O) interface 1958. The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server 2003. TM , Mac OS X TM , Unix TM , Linux TM etc.
[0103] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the above-mentioned text detection method are implemented. For example, the non-transitory computer-readable storage medium can be the above-mentioned memory 1932 including program instructions, and the above-mentioned program instructions can be executed by the processor 1922 of the electronic device 1900 to complete the above-mentioned text detection method.
[0104] In another exemplary embodiment, a computer program product is further provided. The computer program product includes a computer program that can be executed by a programmable device. The computer program has a code portion for executing the above-mentioned text detection method when executed by the programmable device.
[0105] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings; however, the present disclosure is not limited to the specific details in the above embodiments. Within the technical concept of the present disclosure, a variety of simple modifications can be made to the technical solution of the present disclosure, and these simple modifications all fall within the protection scope of the present disclosure.
[0106] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.
[0107] In addition, various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.
Claims
1. A text detection method, characterized in that: The method comprises: Input the image to be detected into a preset text detection model to obtain multiple predicted text area maps with different unit text area sizes; perform pixel expansion on the target predicted text area map under the supervision of the supervised predicted text area map to obtain an intermediate expansion map, wherein the target predicted text area map is a predicted text area map with the smallest unit text area size among the multiple predicted text area maps, and the supervised predicted text area map is a predicted text area map with a unit text area size ranking adjacent to the target predicted text area map among the multiple predicted text area maps, and a size larger than that of the target predicted text area map; For the obtained intermediate expansion map, pixel expansion is performed under the supervision of the new supervised predicted text region map until the new supervised predicted text region map is the predicted text region map with the largest unit text region size among the multiple predicted text region maps, and the obtained intermediate expansion map is the final predicted text region map; Among them, the new supervised predicted text area map is a predicted text area map in which the size of the unit text area in the multiple predicted text area maps is adjacent to the previous supervised predicted text area map and the size is larger than the previous supervised predicted text area map.
2. The text detection method according to claim 1, characterized in that: Inputting the image to be detected into a preset text detection model to obtain a plurality of predicted text area maps with different unit text area sizes includes the following steps performed based on the preset text detection model: Perform feature extraction on the image to be detected to obtain multiple target feature maps, which are derived from different layers of feature extraction; Performing bidirectional fusion on the multiple target feature maps to obtain multiple first fused feature maps; For each first fused feature map, the receptive field is expanded by a 3x3 convolution kernel, two consecutive 3x3 convolution kernels, and three consecutive 3x3 convolution kernels to obtain a second fused feature map; Upsampling and concatenating the plurality of second fused feature maps to obtain a third fused feature map, wherein the size of the third fused feature map is the same as the maximum size of the plurality of second fused feature maps; Reducing the number of channels of the third fused feature map to obtain a fourth fused feature map; Processing the fourth fused feature map through multiple 1x1 convolution kernels to obtain multiple fifth fused feature maps, wherein the number of the 1x1 convolution kernels is the same as the number of the predicted text area maps, and the number of channels of each of the fifth fused feature maps is 2 and the number of the fifth fused feature maps is the same as the number of the predicted text area maps; Upsampling the plurality of fifth fused feature maps to obtain a plurality of sixth fused feature maps, wherein a size of each of the sixth fused feature maps is the same as that of the image to be detected; A plurality of sixth fusion feature maps are predicted to obtain a plurality of predicted text area maps with different unit text area sizes.
3. The text detection method according to claim 2, characterized in that: The preset text detection model is trained in the following way: For each image in the image set, all unit text areas in the image are annotated according to an annotation instruction input by a user to obtain a first annotated image, and all unit text areas in the first annotated image are indented multiple times according to different indentation distances to obtain multiple second annotated images, wherein the annotation instruction is the coordinates of all unit text areas in the image sequentially represented according to a fixed order; Based on the image set, a multi-branch model is trained to obtain a preset text detection model.
4. The text detection method according to claim 2, characterized in that: The preset text detection model is trained in the following way: For each image in the image set, all unit text areas in the image are annotated according to an annotation instruction input by a user to obtain a first annotated image, and all unit text areas in the first annotated image are indented multiple times according to different indentation distances to obtain multiple second annotated images, wherein the annotation instruction is the coordinates of all unit text areas in the image sequentially represented according to a fixed order; Based on the image set, a multi-branch model is trained to obtain a multi-branch detection model; The multi-branch detection model is reconstructed to obtain a single-branch detection model, and the single-branch detection model is used as a preset text detection model.
5. A text detection device, characterized in that: The text detection device comprises: A detection module is used to input the image to be detected into a preset text detection model to obtain multiple predicted text area maps with different unit text area sizes; a post-processing module is used to perform pixel expansion on the target predicted text area map under the supervision of a supervised predicted text area map to obtain an intermediate expansion map, wherein the target predicted text area map is a predicted text area map with the smallest unit text area size among the multiple predicted text area maps, and the supervised predicted text area map is a predicted text area map with a size order of unit text areas in the multiple predicted text area maps that is adjacent to the target predicted text area map and has a size larger than the target predicted text area map; for the obtained intermediate expansion map, pixel expansion is performed under the supervision of a new supervised predicted text area map until the new supervised predicted text area map is the predicted text area map with the largest unit text area size among the multiple predicted text area maps, and the obtained intermediate expansion map is the final predicted text area map; wherein the new supervised predicted text area map is a predicted text area map with a size order of unit text areas in the multiple predicted text area maps that is adjacent to the previous supervised predicted text area map and has a size larger than the previous supervised predicted text area map.
6. The text detection device according to claim 5, characterized in that: The detection module is specifically used to perform feature extraction on the image to be detected based on the preset text detection model to obtain multiple target feature maps, and the multiple target feature maps are derived from different layers of feature extraction; bidirectionally fuse the multiple target feature maps to obtain multiple first fused feature maps; for each first fused feature map, the receptive field is expanded by a 3x3 convolution kernel, two consecutive 3x3 convolution kernels, and three consecutive 3x3 convolution kernels to obtain a second fused feature map; upsample and splice the multiple second fused feature maps to obtain a third fused feature map, and the size of the third fused feature map is the same as the maximum size of the multiple second fused feature maps. The method comprises the following steps: reducing the number of channels of the third fused feature map to obtain a fourth fused feature map; processing the fourth fused feature map through multiple 1x1 convolution kernels to obtain multiple fifth fused feature maps, wherein the number of the 1x1 convolution kernels is the same as the number of the predicted text area maps, and the number of channels of each of the fifth fused feature maps is 2 and the number of the channels is the same as the number of the predicted text area maps; upsampling the multiple fifth fused feature maps to obtain multiple sixth fused feature maps, and the size of each of the sixth fused feature maps is the same as the image to be detected; predicting the multiple sixth fused feature maps to obtain multiple predicted text area maps with different unit text area sizes.
7. The text detection device according to claim 6, characterized in that: The preset text detection model is trained in the following way: For each image in the image set, all unit text areas in the image are annotated according to an annotation instruction input by a user to obtain a first annotated image, and all unit text areas in the first annotated image are indented multiple times according to different indentation distances to obtain multiple second annotated images, wherein the annotation instruction is the coordinates of all unit text areas in the image sequentially represented according to a fixed order; Based on the image set, a multi-branch model is trained to obtain a preset text detection model.
8. The text detection device according to claim 6, characterized in that: The preset text detection model is trained in the following way: For each image in the image set, all unit text areas in the image are annotated according to an annotation instruction input by a user to obtain a first annotated image, and all unit text areas in the first annotated image are indented multiple times according to different indentation distances to obtain multiple second annotated images, wherein the annotation instruction is the coordinates of all unit text areas in the image sequentially represented according to a fixed order; Based on the image set, a multi-branch model is trained to obtain a multi-branch detection model; The multi-branch detection model is reconstructed to obtain a single-branch detection model, and the single-branch detection model is used as a preset text detection model.
9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 4 are implemented.
10. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 4.
