Text detection method and device, computer device and storage medium

Abstract

The embodiment of the application belongs to the technical field of data processing, and relates to a character detection method and device, computer equipment and a storage medium. The character detection method comprises the following steps: obtaining a fusion feature map corresponding to a target picture; inputting the fusion feature map into a convolution network to obtain a character offset map, a character distribution map, a word distribution map and a word center line map of the target picture; determining a polygon bounding box according to the character offset map, the word distribution map and the character distribution map; identifying a region corresponding to the polygon bounding box intersecting with the word center line map as a character region; and decoding the character region to obtain a character detection result corresponding to the target picture. The application can improve the accuracy of the character detection result.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a text detection method, apparatus, computer equipment, and storage medium. Background Technology

[0002] Accurate and efficient text recognition from images is crucial for applications such as autonomous driving, map navigation, and image search. Text detection, as a prerequisite for text recognition, directly impacts the final recognition result.

[0003] Current text detection methods typically use special operators (such as color, texture, or specified rectangles) to detect text in images, achieving text detection in simple scenes. However, unlike scanned paper documents, text in natural scenes may exhibit characteristics such as perspective distortion, deformation, strong low light, and noisy backgrounds. Existing text detection methods do not yield ideal results for text recognition in natural scenes. Summary of the Invention

[0004] The purpose of this application is to provide a text detection method, apparatus, computer device, and storage medium to at least solve the problem of unsatisfactory text recognition results in natural scenes in the prior art.

[0005] To address the aforementioned technical problems, this application provides a text detection method, which employs the following technical solution:

[0006] Obtain the fused feature map corresponding to the target image; input the fused feature map into a convolutional network to obtain the character offset map, character distribution map, word distribution map, and word centerline map of the target image; determine polygon bounding boxes based on the character offset map, the word distribution map, and the character map; identify the regions corresponding to the polygon bounding boxes that intersect with the word centerline map as text regions; decode the text regions to obtain the text detection results corresponding to the target image.

[0007] Furthermore, the above-mentioned step of obtaining the fused feature map corresponding to the target image includes: obtaining the target image; using the ResNet50 feature extractor to extract features from the target image to obtain at least one initial feature map; and using an upsampling method to perform feature fusion processing on the initial feature map to obtain the fused feature map.

[0008] Furthermore, before using the ResNet50 feature extractor to extract features from the target image to obtain at least one initial feature map, the text detection method further includes: scaling and normalizing the target image.

[0009] Furthermore, the step of determining the polygon bounding box based on the character offset map, the word distribution map, and the character distribution map includes: binarizing the word distribution map to obtain a first word distribution map; filtering noise from the character offset map and the character distribution map based on the first word distribution map to obtain a first character offset map and a first character distribution map; binarizing the first character offset map to obtain a second character offset map; performing difference processing on the second character offset map and the first word distribution map to obtain a comprehensive word distribution map; extracting multiple character feature point coordinates from the first character distribution map; and determining the polygon bounding box based on the character feature point coordinates and the comprehensive word distribution map.

[0010] Furthermore, the step of identifying the region corresponding to the polygonal bounding box that intersects with the word centerline map as a text region includes: filtering noise from the word centerline map based on the first word distribution map to obtain a first word centerline map; the first word distribution map is obtained by binarizing the word distribution map; binarizing the first word centerline map to obtain a second word centerline map; and identifying the region corresponding to the polygonal bounding box that intersects with the second word centerline map as a text region.

[0011] Furthermore, before inputting the fused feature map into the convolutional network to obtain the character offset map, character distribution map, word distribution map, and word centerline map of the target image, the text detection method further includes: using the historical fused feature map of historical images as input, and the historical character offset map, historical character distribution map, historical word distribution map, and historical word centerline map as output, the convolutional network is trained using a weakly supervised strategy and a preset convolutional network model.

[0012] To address the aforementioned technical problems, this application also provides a text detection device, which employs the following technical solution:

[0013] The first acquisition module is used to acquire the fused feature map corresponding to the target image; the second acquisition module is used to input the fused feature map into a convolutional network to acquire the character offset map, character distribution map, word distribution map, and word centerline map of the target image; the determination module is used to determine the polygon bounding box based on the character offset map, the word distribution map, and the character distribution map; the text recognition module is used to identify the region corresponding to the polygon bounding box that intersects with the word centerline map as a text region; and the text decoding module is used to decode the text region to obtain the text detection result corresponding to the target image.

[0014] Furthermore, the first acquisition module includes an acquisition submodule, an extraction submodule, and a first processing submodule; the acquisition submodule is used to acquire a target image; the extraction submodule is used to extract features from the target image using a ResNet50 feature extractor to obtain at least one initial feature map; the first processing submodule is used to perform feature fusion processing on the initial feature map using an upsampling method to obtain a fused feature map.

[0015] Furthermore, the first acquisition module also includes a second processing submodule; the second processing submodule is used to perform scaling and normalization processing on the target image.

[0016] Furthermore, the determining module includes a word processing submodule, a filtering submodule, a character processing submodule, a difference processing submodule, a coordinate extraction submodule, and a determining submodule. The word processing submodule is used to binarize the word distribution map to obtain a first word distribution map. The filtering submodule is used to filter noise from the character offset map and the character distribution map based on the first word distribution map to obtain a first character offset map and a first character distribution map. The character processing submodule is used to binarize the first character offset map to obtain a second character offset map. The difference processing submodule is used to perform difference processing on the second character offset map and the first word distribution map to obtain a comprehensive word distribution map. The coordinate extraction submodule is used to extract the coordinates of multiple character feature points from the first character distribution map. The determining submodule is used to determine a polygonal bounding box based on the character feature point coordinates and the comprehensive word distribution map.

[0017] Furthermore, the text recognition module includes a noise filtering submodule, a centerline map processing submodule, and a recognition submodule; the noise filtering submodule is used to filter noise from the word centerline map based on the first word distribution map to obtain a first word centerline map; the first word distribution map is obtained by binarizing the word distribution map; the centerline map processing submodule is used to binarize the first word centerline map to obtain a second word centerline map; the recognition submodule is used to recognize the regions corresponding to the polygonal bounding boxes that intersect with the second word centerline map as text regions.

[0018] Furthermore, the text detection device also includes a training module; the training module is used to train the convolutional network by taking the historical fusion feature map of the historical image as input, and the historical character offset map, historical character distribution map, historical word distribution map, and historical word centerline map as output, and using a weakly supervised strategy and a preset convolutional network model.

[0019] To address the aforementioned technical problems, this application also provides a computer device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the text detection method described above.

[0020] To address the aforementioned technical problems, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the text detection method described above.

[0021] Compared with existing technologies, the embodiments of this application have the following main advantages: The fused feature map corresponding to the target image is input into a convolutional network to obtain the character offset map, character distribution map, word distribution map, and word centerline map of the target image. Then, polygonal bounding boxes are determined based on the character offset map, word distribution map, and character distribution map, and the regions corresponding to the polygonal bounding boxes that intersect with the word centerline map are identified as text regions. Finally, the text regions are decoded to obtain the text detection result corresponding to the target image. In this way, the character offset map and character distribution map corresponding to the character module, and the word distribution map and word centerline map corresponding to the word module, share the fused feature map of the target image. That is, both character and word supervision signals are used to detect text information in the image, improving the accuracy of the text detection result. Attached Figure Description

[0022] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is an exemplary system architecture diagram to which this application can be applied;

[0024] Figure 2 This is a flowchart of an embodiment of the text detection method of this application;

[0025] Figure 3 yes Figure 2 A flowchart of one embodiment of step S21;

[0026] Figure 4 yes Figure 2 A flowchart of an embodiment of step S23;

[0027] Figure 5 yes Figure 2 A flowchart of an embodiment of step S24;

[0028] Figure 6 This is a schematic diagram of the structure of one embodiment of the text detection device of this application;

[0029] Figure 7 This is a schematic diagram of the structure of one embodiment of the computer device according to this application. Detailed Implementation

[0030] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.

[0031] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0032] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0033] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0034] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social media platform software, etc.

[0035] Terminal devices 101, 102, and 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 players (Moving Picture Experts Group Audio Layer IV), laptops, and desktop computers, etc.

[0036] Server 105 can be a server that provides various services, such as a backend server that supports the pages displayed on terminal devices 101, 102, and 103.

[0037] It should be noted that the text detection method provided in this application embodiment can be applied to the server device 105 mentioned above, as well as to the terminal devices 101, 102, and 103 mentioned above. The server device 105 and the terminal devices 101, 102, and 103 can be collectively referred to as electronic devices. That is, the executing entity of the text detection method provided in this application embodiment can be a text detection device, and the text detection device can be the aforementioned electronic device (such as server device 105 or terminal devices 101, 102, and 103).

[0038] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0039] Continue to refer to Figure 2 The diagram illustrates a flowchart of an embodiment of the text detection method according to this application. The text detection method includes the following steps:

[0040] Step S21: Obtain the fusion feature map corresponding to the target image.

[0041] Specifically, Figure 3 This is a schematic flowchart illustrating a process for obtaining a fused feature map corresponding to a target image, as provided in an embodiment of this application. (Refer to...) Figure 3 As shown, it includes steps S211-S213.

[0042] Step S211: Obtain the target image.

[0043] Step S212: Use the ResNet50 feature extractor to extract features from the target image to obtain at least one initial feature map.

[0044] After obtaining the target image, the first step is to scale the target image.

[0045] Specifically, for a target image of size H*W, the target image is scaled by pixels with the shorter side length being the first value, and black pixels are pasted onto the two longer sides respectively, so that the pixel length of the longer side is divisible by the second value, thus obtaining the scaled target image.

[0046] Next, the scaled target image is normalized.

[0047] Specifically, the pixel values ​​of the scaled target image are first divided by the third value, and then the scaled target image is normalized in the BGR channel according to the preset mean and preset variance to obtain a pre-processed image of the preset size.

[0048] For example, for a target image of size H*W, the image is scaled by 800 pixels along its shorter side, and black pixels are pasted onto the two longer sides so that the pixel length of the longer sides is divisible by 32, resulting in a scaled target image. Then, the pixel values ​​of the scaled target image are divided by 255, and the image is normalized in the BGR channel according to the mean pixel_means = [0.406, 0.456, 0.485] and the variance pixel_stds = [0.225, 0.224, 0.229], resulting in a preprocessed image of size 3*H'*W'.

[0049] Finally, the ResNet50 feature extractor is used to extract features from the preprocessed image to obtain at least one initial feature map.

[0050] Specifically, the ResNet50 feature extractor is used to extract features from the preprocessed image, resulting in five feature maps of different sizes and channel depths: F0, F1, F2, F3, and F4. The channel depths and sizes of F0 are (128, 1 / 2 * H', 1 / 2 * W'), F1 are (256, 1 / 4 * H', 1 / 4 * W'), F2 are (512, 1 / 8 * H', 1 / 8 * W'), F3 are (1024, 1 / 16 * H', 1 / 16 * W'), and F4 are (2048, 1 / 32 * H', 1 / 32 * W').

[0051] In this embodiment, the shallow feature map contains more details and positional information, which is more conducive to detecting small text; the deep feature map contains more contextual semantic information, which is more conducive to detecting large text, and can further suppress false positive predictions and improve the accuracy of text detection.

[0052] Step S213: Use upsampling to perform feature fusion processing on the initial feature map to obtain a fused feature map.

[0053] Specifically, F1, F2, F3, and F4 are used as fusion sources, and upsampling is used to fuse shallow feature maps into deep feature maps. For example, the shallow feature maps are fused into deep feature maps using the instruction F1' = Up_Sample(Reduce_Channel(F2'), Reduce_Channel(F1)). Here, Up_Sample represents element-wise addition, and Reduce_Channel is a convolutional neural network containing convolutional layers, batch normalization layers, and non-linear activation layers. Dimensionality reduction of the feature maps can be achieved by changing the size of the convolutional kernels in the convolutional layers. Thus, the depth and size of F1' are (256, 1 / 4 * H', 1 / 4 * W'), the depth and size of F2' are (256, 1 / 8 * H', 1 / 8 * W'), the depth and size of F3' are (256, 1 / 16 * H', 1 / 16 * W'), and the depth and size of F4' are (256, 1 / 32 * H', 1 / 32 * W').

[0054] Then, F1', F2', F3', F4' and F0, F1, F2, F3, F4 are merged according to channels to obtain the initial fused feature map. Afterwards, a 1*1 convolutional kernel is used to reduce the dimensionality of the initial fused feature map to obtain the final fused feature map F5', with a depth and size of (256, 1 / 4*H', 1 / 4*W').

[0055] Step S22: Input the fused feature map into the convolutional network to obtain the character offset map, character distribution map, word distribution map, and word centerline map of the target image.

[0056] Optionally, before step S22, the text detection method further includes taking the historical fusion feature map of the historical image as input, and the historical character offset map, historical character distribution map, historical word distribution map, and historical word centerline map as output, and training a convolutional network using a weak supervision strategy and a preset convolutional network model.

[0057] The pre-defined convolutional network model includes convolutional layers, batch normalization layers, nonlinear activation layers, and deconvolutional layers.

[0058] The historical character offset map is a feature map composed of the first distance between the center of a character in a historical image and the center point of the historical image (the first distance value of the background is 0).

[0059] The historical character distribution map is a feature map composed of the second distance between each point in the historical image and the center of its corresponding character (the second distance value for the background is 0).

[0060] Optionally, both the first and second distances can be Euclidean distances.

[0061] The historical word distribution map is a feature map composed of text regions in historical images. For example, if a coordinate point is located within a text region, the label value of that coordinate point is 1; otherwise, it is 0.

[0062] A historical word centerline map is a feature map composed of the centerlines of text regions in a historical image. For example, after determining the historical word distribution map, first determine its geometric centerline (without width). Then, take 20 equal division points on this geometric centerline (the value can be determined according to the actual situation, such as the size of the historical word distribution map). Each division point is covered by a circle with an appropriate diameter (determined according to the actual size of the text region). Finally, calculate the average diameter of these 20 circles, and use the appropriate width of the average diameter (such as 70% of the average diameter) as the width of the centerline. Indent the beginning and end of the original geometric centerline by an appropriate length (such as 50% of the average diameter) to obtain the length of the centerline, thus obtaining the historical word centerline map.

[0063] Specifically, the process begins by using the historical fusion feature map of a historical image as input, and the historical character offset map, historical character distribution map, historical word distribution map, and historical word centerline map as output to train a pre-defined convolutional network model, resulting in a primary convolutional network. Next, the adjusted fusion feature map of the adjusted image is used as input to the primary convolutional network. Based on the adjusted word distribution map and adjusted word centerline map of the output adjusted image, the primary convolutional network is adjusted and iterated for a predetermined number of iterations to obtain the adjusted convolutional network. Then, the test fusion feature map of the test image is used as input to the adjusted convolutional network, outputting a test character offset map (called a pseudo-label), a test character distribution map (called a pseudo-label), a test word distribution map, and a test word centerline map. The quality of the character offset map and test character distribution map is evaluated using the test word distribution map and test word centerline map, and the positions corresponding to the character offset maps and test character distribution maps with lower scores are set as non-critical, resulting in the test convolutional network. Finally, the test convolutional network is iteratively trained using the pseudo-labels until the loss function converges, resulting in the final convolutional network.

[0064] In this embodiment, word distribution maps and word centerline maps are used as supervision signals to generate character pseudo-labels. Then, the positions corresponding to the lower-quality character pseudo-labels are set as non-emphasis. This reduces the number of iterations during the training process of the convolutional network and improves the training speed of the convolutional network.

[0065] Next, the fused feature map of the target image is input into the trained convolutional network. The output of the convolutional network is the character offset map, character distribution map, word distribution map, and word centerline map of the target image. The size of the character offset map is (H', W'), the size of the character distribution map is (2, H', W'), the size of the word distribution map is (H', W'), and the size of the word centerline map is (H', W').

[0066] Step S23: Determine the polygon bounding box based on the character offset map, word distribution map, and character distribution map.

[0067] Specifically, Figure 4 This is a schematic flowchart illustrating the process of determining a polygonal bounding box, provided as an embodiment of this application. (Refer to...) Figure 4 As shown, it includes steps S231-S236.

[0068] Step S231: Binarize the word distribution map to obtain the first word distribution map.

[0069] For example, a word distribution map can be binarized using a threshold of 0.5. Pixels of points in the word distribution map that are greater than or equal to 0.5 are set to 255, and pixels of points that are less than 0.5 are set to 0, thus obtaining the first word distribution map.

[0070] Step S232: Based on the first word distribution map, noise filtering is performed on the character offset map and the character distribution map to obtain the first character offset map and the first character distribution map.

[0071] Specifically, the points corresponding to the points with a pixel value of 0 in the first word distribution map are set to 0 in the character offset map and the character distribution map, thus obtaining the first character offset map and the first character distribution map.

[0072] Step S233: Binarize the first character offset image to obtain the second character offset image.

[0073] For example, the first character offset map is binarized using a threshold of 0.7. The pixels of points in the first character offset map that are greater than or equal to 0.7 are set to 255, and the pixels of points in the word distribution map that are less than 0.7 are set to 0, thus obtaining the second character offset map.

[0074] Step S234: Perform difference processing on the second character offset map and the first word distribution map to obtain a comprehensive word distribution map.

[0075] Specifically, the pixels of points with a pixel value of 0 in the second character offset map and a pixel value of 255 in the first word distribution map are set to 255 to obtain a comprehensive word distribution map.

[0076] Step S235: Extract the coordinates of multiple character feature points from the first character distribution map.

[0077] Specifically, a pre-trained extraction module can be used to extract the coordinates of multiple character feature points from the first character distribution map.

[0078] Step S236: Determine the polygonal bounding box based on the coordinates of character feature points and the comprehensive word distribution map.

[0079] Specifically, based on the coordinates of character feature points, the positions of a set of points on at least two boundaries are obtained from the comprehensive word distribution map. Then, the positions of all the points in the set are connected to obtain the polygonal bounding box of the text region of arbitrary shape.

[0080] Step S24: The region corresponding to the polygonal bounding box that intersects with the word center line diagram is identified as the text region.

[0081] Specifically, Figure 5 This is a schematic diagram illustrating a process for identifying text regions, provided as an embodiment of this application. (Refer to...) Figure 5 As shown, it includes steps S241-S243.

[0082] Step S241: Filter the noise from the word centerline map based on the first word distribution map to obtain the first word centerline map.

[0083] Specifically, the points corresponding to pixels with a value of 0 in the word centerline map and the first word distribution map are set to 0 to obtain the first word centerline map. The first word distribution map is obtained by binarizing the word distribution map.

[0084] Step S242: Binarize the center line map of the first word to obtain the center line map of the second word.

[0085] For example, the center line image of the first word is binarized using a threshold of 0.5. The pixels of points in the center line image of the first word that are greater than or equal to 0.5 are set to 255, and the pixels of points in the center line image of the first word that are less than 0.5 are set to 0, thus obtaining the center line image of the second word.

[0086] Step S243: The region corresponding to the polygonal bounding box that intersects with the center line of the second word is identified as the text region.

[0087] Step S25: Decode the text region to obtain the text detection result corresponding to the target image.

[0088] In this embodiment, the fused feature map corresponding to the target image is input into a convolutional network to obtain the character offset map, character distribution map, word distribution map, and word centerline map of the target image. Then, polygon bounding boxes are determined based on the character offset map, word distribution map, and character distribution map, and the regions corresponding to the polygon bounding boxes that intersect with the word centerline map are identified as text regions. Finally, the text regions are decoded to obtain the text detection result corresponding to the target image. This allows the character offset map and character distribution map of the character module, and the word distribution map and word centerline map of the word module, to share the fused feature map of the target image. That is, both character and word supervision signals are used to detect text information in the image, improving the accuracy of the text detection result.

[0089] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0090] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0091] Further reference Figure 6 As a response to the above Figure 2 To implement the method shown, this application provides an embodiment of a text detection device, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0092] like Figure 6 As shown, the text detection device 60 of this embodiment includes: a first acquisition module 61, a second acquisition module 62, a determination module 63, a text recognition module 64, and a text decoding module 65. Wherein:

[0093] The first acquisition module 61 is used to acquire the fused feature map corresponding to the target image; the second acquisition module 62 is used to input the fused feature map into a convolutional network to acquire the character offset map, character distribution map, word distribution map, and word centerline map of the target image; the determination module 63 is used to determine the polygon bounding box based on the character offset map, word distribution map, and character distribution map; the text recognition module 64 is used to identify the region corresponding to the polygon bounding box that intersects with the word centerline map as a text region; and the text decoding module 65 is used to decode the text region to obtain the text detection result corresponding to the target image.

[0094] In this embodiment, the fused feature map corresponding to the target image is input into a convolutional network to obtain the character offset map, character distribution map, word distribution map, and word centerline map of the target image. Then, polygon bounding boxes are determined based on the character offset map, word distribution map, and character distribution map, and the regions corresponding to the polygon bounding boxes that intersect with the word centerline map are identified as text regions. Finally, the text regions are decoded to obtain the text detection result corresponding to the target image. This allows the character offset map and character distribution map of the character module, and the word distribution map and word centerline map of the word module, to share the fused feature map of the target image. That is, both character and word supervision signals are used to detect text information in the image, improving the accuracy of the text detection result.

[0095] In some optional implementations of this embodiment, the first acquisition module 61 includes an acquisition submodule, an extraction submodule, and a first processing submodule; the acquisition submodule is used to acquire a target image; the extraction submodule is used to extract features from the target image using a ResNet50 feature extractor to obtain at least one initial feature map; the first processing submodule is used to perform feature fusion processing on the initial feature map using an upsampling method to obtain a fused feature map.

[0096] In this embodiment, the ResNet50 feature extractor is used to extract features from the target image, resulting in feature maps of different depths. Shallow feature maps contain more details and positional information, making them better at detecting small text, while deeper feature maps contain more contextual semantic information, making them better at detecting large text. This can further suppress false positive predictions and improve the accuracy of text detection.

[0097] In some optional implementations of this embodiment, the first acquisition module 61 further includes a second processing submodule; the second processing submodule is used to perform scaling and normalization processing on the target image.

[0098] In some optional implementations of this embodiment, the determining module 63 includes a word processing submodule, a filtering submodule, a character processing submodule, a difference processing submodule, a coordinate extraction submodule, and a determining submodule. The word processing submodule is used to binarize the word distribution map to obtain a first word distribution map. The filtering submodule is used to filter noise from the character offset map and the character distribution map based on the first word distribution map to obtain a first character offset map and a first character distribution map. The character processing submodule is used to binarize the first character offset map to obtain a second character offset map. The difference processing submodule is used to perform difference processing on the second character offset map and the first word distribution map to obtain a comprehensive word distribution map. The coordinate extraction submodule is used to extract the coordinates of multiple character feature points from the first character distribution map. The determining submodule is used to determine the polygon bounding box based on the coordinates of the character feature points and the comprehensive word distribution map.

[0099] In some optional implementations of this embodiment, the text recognition module 64 includes a noise filtering submodule, a centerline map processing submodule, and a recognition submodule; the noise filtering submodule is used to filter noise from the word centerline map based on the first word distribution map to obtain the first word centerline map; the first word distribution map is obtained by binarizing the word distribution map; the centerline map processing submodule is used to binarize the first word centerline map to obtain the second word centerline map; the recognition submodule is used to recognize the regions corresponding to the polygonal bounding boxes that intersect with the second word centerline map as text regions.

[0100] In some optional implementations of this embodiment, the text detection device further includes a training module; the training module is used to train a convolutional network by taking the historical fusion feature map of the historical image as input, and the historical character offset map, historical character distribution map, historical word distribution map, and historical word centerline map as output, and using a weak supervision strategy and a preset convolutional network model.

[0101] In this embodiment, word distribution maps and word centerline maps are used as supervision signals to generate character pseudo-labels. Then, the positions corresponding to the lower-quality character pseudo-labels are set as non-emphasis. This reduces the number of iterations during the training process of the convolutional network and improves the training speed of the convolutional network.

[0102] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 7 , Figure 7 This is a basic structural block diagram of the computer device in this embodiment.

[0103] The computer device 70 includes a memory 71, a processor 72, and a network interface 73 that are interconnected via a system bus. It should be noted that only the computer device 70 with components 71-73 is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0104] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.

[0105] The memory 71 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 71 may be an internal storage unit of the computer device 70, such as the hard disk or memory of the computer device 70. In other embodiments, the memory 71 may also be an external storage device of the computer device 70, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. Of course, the memory 71 may include both internal storage units and external storage devices of the computer device 70. In this embodiment, the memory 71 is typically used to store the operating system and various application software installed on the computer device 70, such as program code for a text detection method. In addition, the memory 71 can also be used to temporarily store various types of data that have been output or will be output.

[0106] In some embodiments, the processor 72 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 72 is typically used to control the overall operation of the computer device 70. In this embodiment, the processor 72 is used to run program code stored in the memory 71 or process data, for example, to run the program code of the text detection method.

[0107] The network interface 73 may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the computer device 70 and other electronic devices.

[0108] In this embodiment, the character offset map and character distribution map corresponding to the character module, and the word distribution map and word centerline map corresponding to the word module share the fusion feature map of the target image. That is, two kinds of supervision signals, characters and words, are used to detect text information in the image, which improves the accuracy of text detection results.

[0109] This application also provides another embodiment, namely, a computer-readable storage medium storing a text detection program, which can be executed by at least one processor to perform the steps of the text detection method as described above.

[0110] In this embodiment, the character offset map and character distribution map corresponding to the character module, and the word distribution map and word centerline map corresponding to the word module share the fusion feature map of the target image. That is, two kinds of supervision signals, characters and words, are used to detect text information in the image, which improves the accuracy of text detection results.

[0111] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0112] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, the purpose of providing these embodiments is to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.

Claims

1. A text detection method, characterized in that, Includes the following steps: Obtain the fused feature map corresponding to the target image; The fused feature map is input into a convolutional network to obtain the character offset map, character distribution map, word distribution map, and word centerline map of the target image; The polygon bounding box is determined based on the character offset map, the word distribution map, and the character map; The region corresponding to the polygonal bounding box that intersects with the word centerline diagram is identified as the text region. Decode the text region to obtain the text detection result corresponding to the target image; The step of determining the polygon bounding box based on the character offset map, the word distribution map, and the character distribution map includes: The word distribution map is binarized to obtain the first word distribution map; Based on the first word distribution map, noise filtering is performed on the character offset map and the character distribution map to obtain the first character offset map and the first character distribution map; The first character offset image is binarized to obtain the second character offset image; The difference between the second character offset map and the first word distribution map is calculated to obtain a comprehensive word distribution map. Extract the coordinates of multiple character feature points from the first character distribution map; The polygon bounding box is determined based on the coordinates of the character feature points and the comprehensive word distribution map.

2. The text detection method according to claim 1, characterized in that, The step of obtaining the fused feature map corresponding to the target image includes: Obtain the target image; The target image is subjected to feature extraction using the ResNet50 feature extractor to obtain at least one initial feature map; The initial feature map is subjected to feature fusion processing using an upsampling method to obtain a fused feature map.

3. The text detection method according to claim 2, characterized in that, Before using the ResNet50 feature extractor to extract features from the target image and obtain at least one initial feature map, the text detection method further includes: The target image is then scaled and normalized.

4. The text detection method according to claim 1, characterized in that, The step of identifying the region corresponding to the polygonal bounding box that intersects with the word centerline diagram as a text region includes: The word centerline map is obtained by filtering noise from the word centerline map based on the first word distribution map; the first word distribution map is obtained by binarizing the word distribution map. The centerline map of the first word is binarized to obtain the centerline map of the second word. The region corresponding to the polygonal bounding box that intersects with the center line of the second word is identified as the text region.

5. The text detection method according to claim 1, characterized in that, Before inputting the fused feature map into the convolutional network to obtain the character offset map, character distribution map, word distribution map, and word centerline map of the target image, the text detection method further includes: Using historical fusion feature maps of historical images as input, and historical character offset maps, historical character distribution maps, historical word distribution maps, and historical word centerline maps as output, the convolutional network is trained using a weakly supervised strategy and a pre-defined convolutional network model.

6. A character detection device, characterized in that, include: The first acquisition module is used to acquire the fusion feature map corresponding to the target image; The second acquisition module is used to input the fused feature map into the convolutional network to acquire the character offset map, character distribution map, word distribution map, and word centerline map of the target image; The determining module is used to determine the polygon bounding box based on the character offset map, the word distribution map, and the character map; The text recognition module is used to identify the region corresponding to the polygonal bounding box that intersects with the word centerline diagram as a text region; The text decoding module is used to decode the text region to obtain the text detection result corresponding to the target image; The determination module 63 includes a word processing submodule, a filtering submodule, a character processing submodule, a difference processing submodule, a coordinate extraction submodule, and a determination submodule; The word processing submodule is used to perform binarization on the word distribution map to obtain the first word distribution map; The filtering submodule is used to filter noise from the character offset map and the character distribution map based on the first word distribution map, so as to obtain the first character offset map and the first character distribution map. The character processing submodule is used to perform binarization processing on the first character offset image to obtain the second character offset image; The difference processing submodule is used to perform difference processing on the second character offset map and the first word distribution map to obtain a comprehensive word distribution map; The coordinate extraction submodule is used to extract the coordinates of multiple character feature points from the first character distribution map; The determination submodule is used to determine the polygon bounding box based on the coordinates of character feature points and the comprehensive word distribution map.

7. The text detection device according to claim 6, characterized in that, The first acquisition module includes an acquisition submodule, an extraction submodule, and a first processing submodule; The acquisition submodule is used to acquire the target image; The extraction submodule is used to extract features from the target image using a ResNet50 feature extractor to obtain at least one initial feature map; The first processing submodule is used to perform feature fusion processing on the initial feature map using an upsampling method to obtain a fused feature map.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the text detection method as described in any one of claims 1 to 5.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the text detection method as described in any one of claims 1 to 5.

Images