Text detection method and device, storage medium and electronic equipment
By dynamically fusing features of different scales in text detection and using spatial attention modules, the problems of low identification accuracy and complex calculation in the prior art are solved, and more efficient text area detection is achieved.
Patent Information
- Application Number
- CN202510437956.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-08
- Publication Date
- 2025-08-12
AI Technical Summary
The segmentation-based text detection method in the prior art cannot effectively utilize the features of text diversity, resulting in low recognition accuracy and complex calculations, making it difficult to accurately detect text areas in natural scenes and educational scenes.
By obtaining feature maps of multiple preset scales of the target image and inputting them into the pre-trained spatial attention model, dynamically fusing the feature map, the spatial attention module is used to improve the feature expression of key areas, inhibit useless features, and enhance the detection of the target area of interest.
It improves the accuracy and speed of text area detection, enhances the robustness of the model, and enables more accurate positioning of text areas.
Smart Images
Figure CN120472467A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer technology, and in particular to a text detection method, device, storage medium, and electronic device. Background Art
[0002] Detecting and locating text regions in images of given natural and educational scenes can facilitate the extraction of text containing rich language information, allowing downstream tasks of text region detection, such as text recognition, to proceed smoothly.
[0003] In related technologies, text region detection methods are mostly based on segmentation, which converts text detection into a single pixel classification judgment for solution. However, segmentation-based text detection methods fail to effectively utilize the diverse characteristics of text, treating each text instance as a separate instance without considering high-level features. This results in low recognition accuracy and inaccurate text detection results. Furthermore, segmentation-based text region detection methods are computationally complex, resulting in slow detection speeds and significant practical limitations. Summary of the Invention
[0004] The purpose of the present disclosure is to provide a text detection method, device, storage medium and electronic device.
[0005] To achieve the above-mentioned object, according to a first aspect of an embodiment of the present disclosure, a text detection method is provided, the method comprising: Acquire a plurality of first feature maps of preset scales of a target image, wherein the target image is an image of a text region containing text to be detected; Inputting the first feature map of each preset scale into a pre-trained spatial attention model to obtain a fused feature map, wherein the pixel information of each pixel point on the fused feature map is fused pixel information of multiple preset scales; Determining first probability information and first threshold information corresponding to the text to be detected based on the fused feature map, wherein the first probability information is used to represent the probability that the text to be detected is the target text, and the first threshold information is used to represent the probability threshold of the text to be detected being the target text; The text area of the text to be detected is determined according to the first probability information and the first threshold information.
[0006] Optionally, inputting the first feature map of each preset scale into a pre-trained spatial attention model to obtain a fused feature map includes: Performing scale fusion on the first feature maps of each preset scale to obtain a second feature map; After performing a pooling operation on the second feature map along the channel direction, an intermediate feature map is obtained; By performing a convolution operation on the intermediate feature map, a spatial weight of each pixel on the intermediate feature map is obtained; For each pixel point on the intermediate feature map, determine the target pixel value of the pixel point according to the spatial weight and pixel value of the pixel point; The fusion feature map is determined according to the target pixel value of each pixel.
[0007] Optionally, determining first probability information and first threshold information corresponding to the text to be detected according to the fused feature map includes: A convolution operation is performed on the fused feature map through a preset classification network to obtain first probability information and first threshold information corresponding to the text to be detected.
[0008] Optionally, the preset classification network is trained in the following manner: Classifying the target sample using the classification network to be trained to obtain second probability information and second threshold information of the sample image corresponding to the target sample; Using the pixel values of each pixel point in the threshold map, a differentiable binarization process is performed on the pixel values of each pixel point in the probability map to obtain an approximate binary feature map; the threshold map is a threshold map determined according to the second threshold information of the sample image corresponding to the target sample, and the probability map is a probability map determined according to the second probability information of the sample image corresponding to the target sample; Determine the loss function value of the approximate binary feature map, and use the loss function value to train the classification network to be trained through a back propagation algorithm to obtain a preset classification network.
[0009] Optionally, before inputting the first feature map of each preset scale into a pre-trained spatial attention model to obtain a fused feature map, the method further includes: For each of the first feature maps of a preset scale, inputting the first feature map of the preset scale into a pre-trained channel attention model to obtain a third feature map of the preset scale; Inputting the first feature map of each preset scale into a pre-trained spatial attention model to obtain a fused feature map includes: The third feature map of each preset scale is input into the spatial attention model to obtain the fused feature map.
[0010] Optionally, for each first feature map of a preset scale, inputting the first feature map of the preset scale into a pre-trained channel attention model to obtain a third feature map of the preset scale includes: After inputting the first feature map of the preset scale into the channel attention model, performing global average pooling on the first feature map of the preset scale to obtain a first weight for each channel of the first feature map of the preset scale; For each channel of the first feature map of the preset scale, weighted processing is performed on each pixel on the first feature map of the channel according to the first weight of the channel to obtain a weighted feature map of the channel; The third feature map of the preset scale is determined according to the weighted feature map corresponding to each channel.
[0011] Optionally, determining the text area of the text to be detected according to the first probability information and the first threshold information includes: Determining a binary image of the text to be detected according to the first probability information and the first threshold information; The text area is determined according to the binary image.
[0012] Optionally, acquiring a plurality of first feature maps of preset scales of the target image includes: The target image is input into a pre-trained feature extraction model to obtain the first feature maps of multiple different preset scales.
[0013] According to a second aspect of an embodiment of the present disclosure, the present disclosure provides a text detection device, the device comprising: A first acquisition module is used to acquire a plurality of first feature maps of preset scales of a target image, wherein the target image is an image of a text area containing text to be detected; a feature acquisition module, configured to input the first feature map of each preset scale into a pre-trained spatial attention model to obtain a fused feature map, wherein the pixel information of each pixel point on the fused feature map is fused pixel information of multiple preset scales; A first determination module is configured to determine first probability information and first threshold information corresponding to the text to be detected based on the fused feature map, wherein the first probability information is used to represent the probability that the text to be detected is the target text, and the first threshold information is used to represent the probability threshold of the text to be detected being the target text; The second determining module is configured to determine the text area of the text to be detected according to the first probability information and the first threshold information.
[0014] Optionally, the feature acquisition module is used to scale-fuse the first feature maps of each preset scale to obtain a second feature map; perform a pooling operation on the second feature map along the channel direction to obtain an intermediate feature map; obtain the spatial weight of each pixel on the intermediate feature map by performing a convolution operation on the intermediate feature map; for each pixel on the intermediate feature map, determine the target pixel value of the pixel according to the spatial weight and pixel value of the pixel; and determine the fused feature map according to the target pixel value of each pixel.
[0015] Optionally, the first determination module is used to perform a convolution operation on the fused feature map through a preset classification network to obtain first probability information and first threshold information corresponding to the text to be detected.
[0016] Optionally, the preset classification network is trained in the following manner: The target sample is classified by using the classification network to be trained to obtain the second probability information and the second threshold information of the sample image corresponding to the target sample; the pixel value of each pixel point in the threshold map is used to perform differentiable binarization processing on the pixel value of each pixel point in the probability map to obtain an approximate binary feature map; the threshold map is a threshold map determined according to the second threshold information of the sample image corresponding to the target sample, and the probability map is a probability map determined according to the second probability information of the sample image corresponding to the target sample; the loss function value of the approximate binary feature map is determined, and the classification network to be trained is trained by a back propagation algorithm using the loss function value to obtain a preset classification network.
[0017] Optionally, the device further comprises: A third determining module is configured to input, for each of the first feature maps of a preset scale, the first feature map of the preset scale into a pre-trained channel attention model to obtain a third feature map of the preset scale; The feature acquisition module is used to input the third feature map of each preset scale into the spatial attention model to obtain the fused feature map.
[0018] Optionally, the third determination module is used to input the first feature map of the preset scale into the channel attention model, perform global average pooling on the first feature map of the preset scale to obtain a first weight for each channel of the first feature map of the preset scale; for each channel of the first feature map of the preset scale, perform weighted processing on each pixel on the first feature map of the channel according to the first weight of the channel to obtain a weighted feature map of the channel; and determine the third feature map of the preset scale based on the weighted feature map corresponding to each channel.
[0019] Optionally, the second determination module is configured to determine a binary image of the text to be detected based on the first probability information and the first threshold information; and determine the text area based on the binary image.
[0020] Optionally, the first acquisition module is used to input the target image into a pre-trained feature extraction model to obtain the first feature maps of multiple different preset scales.
[0021] According to a third aspect of an embodiment of the present disclosure, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any one of the methods described in the first aspect of the embodiment of the present disclosure.
[0022] According to the fourth aspect of the embodiments of the present disclosure, the present disclosure provides an electronic device, comprising: a memory on which a computer program is stored; and a processor for executing the computer program in the memory to implement the steps of any one of the methods described in the first aspect of the embodiments of the present disclosure.
[0023] Through the above technical solution, a plurality of first feature maps of preset scales of a target image are obtained, wherein the target image is an image of a text region containing text to be detected; the first feature map of each preset scale is input into a pre-trained spatial attention model to obtain a fused feature map, wherein the pixel information of each pixel point on the fused feature map is the fused pixel information of multiple preset scales; first probability information and first threshold information corresponding to the text to be detected are determined based on the fused feature map, wherein the first probability information is used to characterize the probability that the text to be detected is the target text, and the first threshold information is used to characterize the probability threshold of the text to be detected being the target text; and the text region of the text to be detected is determined based on the first probability information and the first threshold information. In this way, when performing text detection, by dynamically fusing features of different scales, the robustness of the model at different scales is further enhanced, and the feature expression of key areas is enhanced by the spatial attention module, different degrees of attention are allocated to different areas in the feature map, the most useful features in the feature map are strengthened, and useless features are suppressed, thereby enhancing the specific target area of interest and weakening irrelevant background areas, thereby effectively improving the detection accuracy of the text area.
[0024] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following detailed description, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings: Figure 1The figure is a flowchart of a text detection method according to an exemplary embodiment.
[0026] Figure 2 It is a framework diagram of a feature extraction model according to an exemplary embodiment.
[0027] Figure 3 is a flowchart of step S102 according to an exemplary embodiment.
[0028] Figure 4 The flowchart of another text detection method is shown according to an exemplary embodiment.
[0029] Figure 5 It is a block diagram of a text detection device according to an exemplary embodiment.
[0030] Figure 6 is a block diagram of another text detection device according to an exemplary embodiment.
[0031] Figure 7 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION
[0032] The following describes the specific embodiments of the present disclosure in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present disclosure and are not intended to limit the present disclosure.
[0033] It should be noted that all actions of acquiring signals, information or data in the present disclosure are carried out in compliance with the corresponding data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.
[0034] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in a different order and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect. The terms "including" and their variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the description below.
[0035] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units. It should be noted that the modifications of "one" and "a plurality of" mentioned in this disclosure are illustrative and not restrictive. Those skilled in the art should understand that unless the context clearly indicates otherwise, they should be understood as "one or more".
[0036] Before introducing the text detection method, device, storage medium, and electronic device provided by the present disclosure, we first introduce the application scenarios involved in each embodiment of the present disclosure. The present disclosure can be applied to text detection scenarios. High-precision text detection plays an important role in promoting education, autonomous driving, scene analysis, industrial automation production, and other fields. As a precursor to text recognition, locating the text area in a specified natural scene image can help us effectively extract text containing rich language information, facilitating the smooth progress of downstream tasks of text detection.
[0037] Related technologies typically use segmentation-based text region detection, which converts text detection into a single-pixel classification decision. However, segmentation-based handwritten text detection fails to effectively leverage the diverse nature of text. Instead, it treats each text instance as a separate instance, ignoring high-level features. This results in low recognition accuracy and inaccurate text detection results.
[0038] In order to solve the above problems, the present disclosure provides a text detection method, device, storage medium and electronic device, which can further enhance the robustness of the model at different scales by dynamically fusing features of different scales during text detection, and improve the feature expression of key areas through a spatial attention module, allocate different degrees of attention to different areas in the feature map, strengthen the most useful features in the feature map, and suppress useless features, thereby enhancing specific target areas of interest while weakening irrelevant background areas, which can effectively improve the detection accuracy of text areas.
[0039] Figure 1 FIG. 1 is a flowchart of a text detection method according to an exemplary embodiment. Figure 1 As shown, the following steps are included: S101, obtaining a plurality of first feature maps of preset scales of a target image.
[0040] The target image is an image of a text area containing the text to be detected. The image can be a video image from an image acquisition device, composed of frames of images, or a single frame image or an image, or can be sourced from other devices. The text to be detected can be one or more lines of text, and the text can be natural language or numbers. This embodiment does not impose any restrictions on the language, shape, length, size, and color of the text, nor does it impose any restrictions on the source and acquisition method of the target image. Moreover, the image to be detected can be the original image of the above-mentioned video image, frame image, or image, or any variant image of the original image.
[0041] In this step, after acquiring the target image, the target image may be input into a pre-trained feature extraction model to obtain the first feature maps of multiple different preset scales.
[0042] For example, Figure 2 is a framework diagram of a feature extraction model according to an exemplary embodiment. Figure 2 As shown, the feature extraction model may include n convolutional layers, where n is a positive integer greater than 1, and n=5 is taken as an example. Each convolutional layer is connected in sequence, and the input data of the first convolutional layer is the target image F. When n is greater than or equal to 2, the input data of the nth convolutional layer is the output data of the n-1th convolutional layer, and the output data of the nth convolutional layer is 1 / 2 of the target image. n times the size of the feature map.
[0043] Secondly, the output data of the jth convolution layer is subjected to a 2-fold upsampling transposed convolution to obtain a 1 / 2 of the target image. j-1 The feature map of this size will be 1 / 2 of the target image. j-1 The feature map of times the size and the target image 1 / 2 output by the j-1th convolutional layer j-1 Add the feature maps of times the size to get the 1 / 2 of the target image corresponding to the j-1th convolution layer j-1 times the size of the merged feature map, where j is an integer greater than 2 and less than or equal to n.
[0044] In this way, F0, F1, F2, and F3 shown in the figure are the first feature maps of the four scales output by the feature extraction model. The above examples are only illustrative and the present disclosure does not limit this.
[0045] S102: Input the first feature map of each preset scale into a pre-trained spatial attention model to obtain a fused feature map.
[0046] Because different regions within a feature map contribute differently, a spatial attention operation is performed on the feature map to focus on the contributing regions within the same feature map. This generates a spatial attention feature map that focuses on regions of interest related to the target object. This fused feature map highlights the feature information of important regions within each of the multiple feature maps.
[0047] Figure 3 is a flowchart of step S102 according to an exemplary embodiment. Figure 3 As shown, the following steps are included: S1021: Scale-fusing the first feature map of each preset scale to obtain a second feature map.
[0048] Among them, the scale fusion can refer to scaling the first feature map of each preset scale to the same resolution, and then superimposing the first feature maps of each preset scale in the channel dimension through a splicing operation to obtain a second feature map after fusion, thereby increasing the number of channels of the feature map (equivalent to the number of features describing the image itself).
[0049] For example, Figure 2 Taking the first feature maps of the four preset scales F0, F1, F2, and F3 shown in FIG as an example, we can first obtain the first feature maps of the four preset scales of the target image according to the method of obtaining the first feature maps of the four preset scales as shown in FIG. Figure 2 The target image 1 / 2 is shown n times the size of the feature map and the target image 1 / 2 j-1 The merged feature map is multiplied by ni times, and the above feature maps are transposed and convolved with ni times upsampling to obtain n-1 1 / 2 of the target image. i The first feature map of each preset scale is scaled to the same resolution. Then, a splicing operation is performed to fuse the first feature maps of each preset scale to obtain a second feature map. i The first feature map of multiple sizes is a feature map of a preset size, and i is an integer less than or equal to n.
[0050] For example, the first feature map of the target image with a size of 1 / 4 is a feature map of a preset resolution. When the first feature maps of the target image with sizes of 1 / 4, 1 / 8, 1 / 16 and 1 / 32 are obtained respectively, the feature maps can be processed 2 times. 0 , 2 1 , 2 2 , 2 3The transposed convolution with multiple upsampling is performed to obtain multiple feature maps with the same resolution but different preset scales. The multiple feature maps can be expressed as (C×H×W), where C is the number of channels of the feature map, and H×W is the size of the feature map, which can be expressed by pixel resolution. Then, the multiple first feature maps with the same resolution but different preset scales can be superimposed in the channel dimension to obtain a second feature map that can be expressed as (N×C×H×W), where N is used to represent the number of first feature maps.
[0051] S1022: Perform a pooling operation on the second feature map along the channel direction to obtain an intermediate feature map.
[0052] In this step, after obtaining the above-mentioned second feature map, the second feature map can be pooled along the channel direction, wherein the above-mentioned pooling operation can be a maximum pooling operation and an average pooling operation; an intermediate feature map is obtained, and the intermediate feature map can be a feature map with a channel number of 1.
[0053] For example, after obtaining the second feature map represented as (N×C×H×W), a maximum pooling operation and an average pooling operation can be performed along the channel direction, and then they are connected to generate a valid feature descriptor, which can be represented as an intermediate feature map of (1×H×W).
[0054] S1023: Perform a convolution operation on the intermediate feature map to obtain the spatial weight of each pixel on the intermediate feature map.
[0055] For example, the intermediate feature map can be introduced into a convolutional network for convolution. The resulting feature map is then passed through an activation function to obtain the spatial weight of each pixel on the intermediate feature map. The activation function can be a sigmoid function, which outputs a 1×H×W feature map with pixel values between 0 and 1. This activation function is then added pixel by pixel to the intermediate feature map, followed by convolution and the sigmoid function to obtain an N×1×H×W feature map. This feature map is used to represent the spatial weight. This spatial weight characterizes the degree of information concentration at that image location.
[0056] For example, the second feature map can be processed by global maximum pooling and global mean pooling in the channel dimension to obtain compressed spatial information, and the shape is changed from the original 1024×38×38 to 1×38×38. The pooling results are then cascaded in the channel dimension to obtain compressed information with a shape of 2×38×38. The compressed spatial information is then aggregated using a two-dimensional convolution with a convolution kernel size of 7 to obtain aggregated information with a shape of 1×38×38 to obtain an intermediate feature map. The intermediate feature map is then added pixel by pixel with the second feature map and the spatial weight is obtained through sigmoid nonlinear activation.
[0057] S1024: For each pixel point on the intermediate feature map, determine a target pixel value of the pixel point according to the spatial weight and pixel value of the pixel point.
[0058] For example, after determining the spatial weight, the target pixel value of each pixel point on the intermediate feature map can be determined based on the numerical value of each pixel point on the feature map corresponding to the spatial weight. Furthermore, the spatial weight value of each pixel point can be multiplied by the numerical value of the pixel point on the intermediate feature map to obtain the target pixel value at the point, wherein the target pixel value refers to the pixel value after obtaining the representation of the importance of the information.
[0059] S1025: Determine the fused feature map according to the target pixel value of each pixel.
[0060] Among them, the pixel value of each pixel on the fusion feature map is the target pixel value.
[0061] S103: Determine first probability information and first threshold information corresponding to the text to be detected according to the fused feature map.
[0062] The first probability information is used to represent the probability that the text to be detected is the target text, and the first threshold information is used to represent the probability threshold that the text to be detected is the target text.
[0063] In this step, a convolution operation can be performed on the fused feature map through a preset classification network to obtain first probability information and first threshold information corresponding to the text to be detected, wherein the first probability information includes the probability that each pixel point in the fused feature map is a pixel point corresponding to the text to be detected, and the first threshold information includes the probability threshold that each pixel point in the fused feature map is a pixel point corresponding to the text to be detected, and the pixels in the first probability information correspond one-to-one to the pixels in the first threshold information.
[0064] In one implementation, the preset classification network can be composed of two groups of convolution modules with the same structure, which are responsible for outputting the probability map and the threshold map respectively. For example, each group of convolution modules can include a 3x3 convolution, two batch normalization operations and two transpose convolutions. In this way, after the fused feature map is input into the preset classification network, the preset classification network can output the first probability information and the first threshold information.
[0065] In addition, the preset classification network can be trained by the following steps: S1. Classify the target sample using the classification network to be trained to obtain second probability information and second threshold information of the sample image corresponding to the target sample.
[0066] Among them, the second probability information includes the probability that each pixel in the sample image is a pixel corresponding to the text, the second threshold information includes the probability threshold that each pixel in the sample image is a pixel corresponding to the text, and the pixels in the second probability information correspond one-to-one to the pixels in the second threshold information. The sample image can be obtained from a training set, and the training set can include multiple sample images, each sample image includes text to be recognized, and pre-annotated text position information, and the classification network to be trained is a pre-set classification network.
[0067] In addition, the classification network to be trained may generally include two branches, one of which is used to generate the second probability information, and the other branch is used to generate the second threshold information.
[0068] S2. Using the pixel value of each pixel point in the threshold map, perform differentiable binarization processing on the pixel value of each pixel point in the probability map to obtain an approximate binary feature map.
[0069] The threshold map is a threshold map determined according to the second threshold information of the sample image corresponding to the target sample, and the probability map is a probability map determined according to the second probability information of the sample image corresponding to the target sample.
[0070] It should be noted that the classification network to be trained includes a differentiable binarization processing module (Differentiable Binarization, DB), which can use the pixel value of each pixel point in the threshold map to perform differentiable binarization processing on the pixel value of each pixel point in the probability map, thereby finally obtaining an approximate binary feature map. Since the pixels in the approximate binary feature map are generated by differentiable binarization processing, they can be differentiated, thereby enabling the approximate binary feature map to be used for model training.
[0071] S3. Determine the loss function value of the approximate binary feature map, and use the loss function value to train the classification network to be trained through a back propagation algorithm to obtain a preset classification network.
[0072] In one possible approach of this step, the trained preset classification network can be obtained by calculating the gradient of the model parameters of the classification network to be trained using a back propagation algorithm based on the loss function value of the approximate binary feature map and then updating the model parameters of the classification network to be trained.
[0073] In another possible approach, the loss function value of the probability map and the loss function value of the threshold map of the sample image can be calculated separately, and then the loss function value of the probability map, the loss function value of the threshold map and the loss function value of the approximate binary feature map are weightedly summed to obtain the total loss function value. The gradient calculated by backpropagation of the total loss function value is then used in combination with a stochastic gradient descent optimizer to update the parameters of the classification network to be trained, thereby realizing the training of the classification network to be trained.
[0074] S104: Determine the text region of the text to be detected according to the first probability information and the first threshold information.
[0075] Optionally, a binary image of the text to be detected may be first determined according to the first probability information and the first threshold information; and then the text area may be determined according to the binary image.
[0076] For example, a threshold map can be determined based on the first threshold information corresponding to each pixel point, and then whether each pixel point in the image corresponding to the text to be detected belongs to a foreground pixel point can be determined based on the first probability information. The foreground pixel point is used to indicate that the pixel point belongs to the text area.
[0077] For example, for each pixel, if the first probability information corresponding to the pixel (such as pixel A) is greater than or equal to the probability threshold corresponding to the pixel (also pixel A) in the threshold map, then the pixel (i.e., pixel A) belongs to the foreground pixel (i.e., the pixel in the text area); otherwise, it belongs to the background pixel.
[0078] Through the above technical solution, a plurality of first feature maps of preset scales of a target image are obtained, wherein the target image is an image of a text area containing a text to be detected; the first feature map of each preset scale is input into a pre-trained spatial attention model to obtain a fused feature map, wherein the pixel information of each pixel on the fused feature map is the fused pixel information of multiple preset scales; first probability information and first threshold information corresponding to the text to be detected are determined based on the fused feature map, wherein the first probability information is used to characterize the probability that the text to be detected is the target text, and the first threshold information is used to characterize the probability threshold of the text to be detected being the target text; and the text area of the text to be detected is determined based on the first probability information and the first threshold information. In this way, when performing text detection, by dynamically fusing features of different scales, the robustness of the model at different scales is further enhanced, and the feature expression of key areas is enhanced by the spatial attention module, different degrees of attention are allocated to different areas in the feature map, the most useful features in the feature map are strengthened, and useless features are suppressed, thereby enhancing the specific target area of interest while weakening irrelevant background areas, which can effectively improve the detection accuracy of the text area.
[0079] In one embodiment, Figure 4 is a flowchart of another text detection method according to an exemplary embodiment. Figure 4 As shown, before inputting the first feature map of each preset scale into the pre-trained spatial attention model to obtain the fused feature map, the method further includes: S105: For each first feature map of the preset scale, input the first feature map of the preset scale into a pre-trained channel attention model to obtain a third feature map of the preset scale.
[0080] In this step, the first feature map of the preset scale can be first input into the channel attention model, and then the first feature map of the preset scale is globally average pooled to obtain the first weight of each channel of the first feature map of the preset scale.
[0081] For example, after obtaining multiple feature maps of different preset scales, for the first feature map represented as a (C×H×W) pair, a channel weight coefficient with a channel number of 1, represented as (C×1×1), is obtained through global average pooling. The channel weight coefficient is then multiplied pixel by pixel with the first feature map through a ReLU activation function to obtain the first weight of each channel on the first feature map.
[0082] Secondly, for each channel of the first feature map of the preset scale, each pixel on the first feature map of the channel is weighted according to the first weight of the channel to obtain a weighted feature map of the channel; then, the third feature map of the preset scale is determined according to the weighted feature map corresponding to each channel.
[0083] For example, after determining the first weight of each channel, the value of each pixel on the first feature map corresponding to the first weight can be weighted by channel to obtain a weighted feature map of the channel; then, the third feature map of the preset scale is determined based on the weighted feature map corresponding to each channel.
[0084] S106: Input the third feature map of each preset scale into the spatial attention model to obtain the fused feature map.
[0085] In this step, the step of obtaining the fused feature map can be performed according to the method in step S102, and the specific steps are not repeated here.
[0086] In this way, before obtaining the fused feature map, a channel attention model can be added. By weighting different channels, different channels can be enhanced or suppressed for different tasks according to the importance of each feature channel, that is, the weights of each channel of the image are dynamically modulated in the neural network, that is, channel attention, thereby enhancing the text feature weights to improve the network's representation ability, which can also show good results in dealing with complex scenes.
[0087] Figure 5 is a block diagram of a text detection device 500 according to an exemplary embodiment. Figure 5 As shown, the device includes: An acquisition module 501 is configured to acquire a plurality of first feature maps of preset scales of a target image, where the target image is an image of a text region containing text to be detected; A feature acquisition module 502 is configured to input the first feature map of each preset scale into a pre-trained spatial attention model to obtain a fused feature map, where the pixel information of each pixel point on the fused feature map is fused pixel information of multiple preset scales; A first determination module 503 is configured to determine first probability information and first threshold information corresponding to the text to be detected based on the fused feature map, wherein the first probability information is used to represent the probability that the text to be detected is the target text, and the first threshold information is used to represent the probability threshold of the text to be detected being the target text; The second determining module 504 is configured to determine the text region of the text to be detected according to the first probability information and the first threshold information.
[0088] Optionally, the feature acquisition module 502 is used to scale fuse the first feature map of each preset scale to obtain a second feature map; perform a pooling operation on the second feature map along the channel direction to obtain an intermediate feature map; obtain the spatial weight of each pixel point on the intermediate feature map by performing a convolution operation on the intermediate feature map; for each pixel point on the intermediate feature map, determine the target pixel value of the pixel point according to the spatial weight and pixel value of the pixel point; and determine the fused feature map according to the target pixel value of each pixel point.
[0089] Optionally, the first determination module 503 is configured to perform a convolution operation on the fused feature map through a preset classification network to obtain first probability information and first threshold information corresponding to the text to be detected.
[0090] Optionally, the preset classification network is trained in the following manner: The target sample is classified by using the classification network to be trained to obtain the second probability information and the second threshold information of the sample image corresponding to the target sample; the pixel value of each pixel point in the threshold map is used to perform differentiable binarization processing on the pixel value of each pixel point in the probability map to obtain an approximate binary feature map; the threshold map is a threshold map determined according to the second threshold information of the sample image corresponding to the target sample, and the probability map is a probability map determined according to the second probability information of the sample image corresponding to the target sample; the loss function value of the approximate binarization feature map is determined, and the classification network to be trained is trained by a back propagation algorithm using the loss function value to obtain a preset classification network.
[0091] Figure 6 is a block diagram of another text detection device according to an exemplary embodiment. Figure 6 As shown, the device also includes: A third determining module 505 is configured to input the first feature map of each preset scale into a pre-trained channel attention model to obtain a third feature map of the preset scale; The feature acquisition module 502 is used to input the third feature map of each preset scale into the spatial attention model to obtain the fused feature map.
[0092] Optionally, the third determination module 505 is used to input the first feature map of the preset scale into the channel attention model, perform global average pooling on the first feature map of the preset scale, and obtain a first weight for each channel of the first feature map of the preset scale; for each channel of the first feature map of the preset scale, perform weighted processing on each pixel on the first feature map of the channel according to the first weight of the channel to obtain a weighted feature map of the channel; and determine the third feature map of the preset scale according to the weighted feature map corresponding to each channel.
[0093] Optionally, the second determining module 504 is configured to determine a binary image of the text to be detected according to the first probability information and the first threshold information; and determine the text area according to the binary image.
[0094] Optionally, the acquisition module 501 is configured to input the target image into a pre-trained feature extraction model to obtain the first feature maps of multiple different preset scales.
[0095] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.
[0096] Through the above technical solution, when performing text detection, the robustness of the model at different scales is further enhanced by dynamically fusing features of different scales, and the feature expression of key areas is improved through the spatial attention module. Different degrees of attention are allocated to different areas in the feature map, strengthening the most useful features in the feature map and suppressing useless features, thereby enhancing specific target areas of interest while weakening irrelevant background areas, which can effectively improve the detection accuracy of text areas.
[0097] Figure 7 FIG. 7 is a block diagram of an electronic device 700 according to an exemplary embodiment. Figure 7 As shown, the electronic device 700 may include: a processor 701 , a memory 702 , and may further include one or more of a multimedia component 703 , an input / output (I / O) interface 704 , and a communication component 705 .
[0098] The processor 701 is used to control the overall operation of the electronic device 700 to complete all or part of the steps in the above-mentioned text detection method. The memory 702 is used to store various types of data to support the operation of the electronic device 700. This data may include, for example, instructions for any application or method operating on the electronic device 700, as well as application-related data such as contact information, sent and received messages, images, audio, video, etc. The memory 702 can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The multimedia component 703 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in the memory 702 or transmitted via the communication component 705. The audio component also includes at least one speaker for outputting audio signals. The input / output (I / O) interface 704 provides an interface between the processor 701 and other interface modules, such as a keyboard, a mouse, and buttons. These buttons may be virtual or physical. The communication component 705 is used for wired or wireless communication between the electronic device 700 and other devices. Wireless communication may include Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IoT, eMTC, or other 5G networks, or any combination thereof, without limitation. Accordingly, the communication component 705 may include a Wi-Fi module, a Bluetooth module, an NFC module, and the like.
[0099] In an exemplary embodiment, the electronic device 700 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-mentioned text detection method.
[0100] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided. When the program instructions are executed by a processor, the steps of the above-mentioned text detection method are implemented. For example, the computer-readable storage medium may be the aforementioned memory 702 including the program instructions. The program instructions may be executed by the processor 701 of the electronic device 700 to perform the above-mentioned text detection method.
[0101] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings. However, the present disclosure is not limited to the specific details of the above embodiments. Within the technical concept of the present disclosure, various simple modifications can be made to the technical solutions of the present disclosure, and these simple modifications all fall within the scope of protection of the present disclosure.
[0102] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.
[0103] In addition, the various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.
Claims
1. A text detection method, characterized in that: The method comprises: Acquire a plurality of first feature maps of preset scales of a target image, wherein the target image is an image of a text region containing text to be detected; Inputting the first feature map of each preset scale into a pre-trained spatial attention model to obtain a fused feature map, wherein the pixel information of each pixel point on the fused feature map is fused pixel information of multiple preset scales; Determining first probability information and first threshold information corresponding to the text to be detected based on the fused feature map, wherein the first probability information is used to represent the probability that the text to be detected is the target text, and the first threshold information is used to represent the probability threshold of the text to be detected being the target text; The text area of the text to be detected is determined according to the first probability information and the first threshold information.
2. The method according to claim 1, characterized in that Inputting the first feature map of each preset scale into a pre-trained spatial attention model to obtain a fused feature map includes: Performing scale fusion on the first feature maps of each preset scale to obtain a second feature map; After performing a pooling operation on the second feature map along the channel direction, an intermediate feature map is obtained; By performing a convolution operation on the intermediate feature map, a spatial weight of each pixel on the intermediate feature map is obtained; For each pixel point on the intermediate feature map, determine the target pixel value of the pixel point according to the spatial weight and pixel value of the pixel point; The fusion feature map is determined according to the target pixel value of each pixel.
3. The method according to claim 1, characterized in that Determining the first probability information and the first threshold information corresponding to the text to be detected according to the fused feature map includes: A convolution operation is performed on the fused feature map through a preset classification network to obtain first probability information and first threshold information corresponding to the text to be detected.
4. The method according to claim 3, characterized in that The preset classification network is trained in the following way: Classifying the target sample using the classification network to be trained to obtain second probability information and second threshold information of the sample image corresponding to the target sample; Using the pixel values of each pixel point in the threshold map, a differentiable binarization process is performed on the pixel values of each pixel point in the probability map to obtain an approximate binary feature map; the threshold map is a threshold map determined according to the second threshold information of the sample image corresponding to the target sample, and the probability map is a probability map determined according to the second probability information of the sample image corresponding to the target sample; Determine the loss function value of the approximate binary feature map, and use the loss function value to train the classification network to be trained through a back propagation algorithm to obtain a preset classification network.
5. The method according to claim 1, wherein Before inputting the first feature map of each preset scale into a pre-trained spatial attention model to obtain a fused feature map, the method further includes: For each of the first feature maps of a preset scale, inputting the first feature map of the preset scale into a pre-trained channel attention model to obtain a third feature map of the preset scale; Inputting the first feature map of each preset scale into a pre-trained spatial attention model to obtain a fused feature map includes: The third feature map of each preset scale is input into the spatial attention model to obtain the fused feature map.
6. The method according to claim 5, characterized in that For each of the first feature maps of the preset scale, inputting the first feature map of the preset scale into a pre-trained channel attention model to obtain a third feature map of the preset scale includes: After inputting the first feature map of the preset scale into the channel attention model, performing global average pooling on the first feature map of the preset scale to obtain a first weight for each channel of the first feature map of the preset scale; For each channel of the first feature map of the preset scale, weighted processing is performed on each pixel on the first feature map of the channel according to the first weight of the channel to obtain a weighted feature map of the channel; The third feature map of the preset scale is determined according to the weighted feature map corresponding to each channel.
7. The method according to claim 1, characterized in that The determining the text area of the to-be-detected text according to the first probability information and the first threshold information includes: Determining a binary image of the text to be detected according to the first probability information and the first threshold information; The text area is determined according to the binary image.
8. The method according to any one of claims 1 to 7, characterized in that The acquiring of a plurality of first feature maps of preset scales of the target image comprises: The target image is input into a pre-trained feature extraction model to obtain the first feature maps of multiple different preset scales.
9. A text detection device, characterized in that: The device comprises: A first acquisition module is used to acquire a plurality of first feature maps of preset scales of a target image, wherein the target image is an image of a text area containing text to be detected; a feature acquisition module, configured to input the first feature map of each preset scale into a pre-trained spatial attention model to obtain a fused feature map, wherein the pixel information of each pixel point on the fused feature map is fused pixel information of multiple preset scales; A first determination module is configured to determine first probability information and first threshold information corresponding to the text to be detected based on the fused feature map, wherein the first probability information is used to represent the probability that the text to be detected is the target text, and the first threshold information is used to represent the probability threshold of the text to be detected being the target text; The second determining module is configured to determine the text area of the text to be detected according to the first probability information and the first threshold information.
10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.
11. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 8.