Text detection method, device, electronic device and storage medium

Through feature pyramids and feature extraction networks, text detection is performed in dark light environments, noise problems caused by brightness enhancement and reduced accuracy of manual feature introduction, and high-accuracy text detection is achieved.

CN113936128BActive Publication Date: 2024-11-15INST OF AUTOMATION CHINESE ACAD OF SCI +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111057748.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-09-09
Publication Date
2024-11-15
Estimated Expiration
2041-09-09

AI Technical Summary

Technical Problem

The prior art has the presence of brightness enhancement in text detection in dark light environments, and the need to manually introduce manual features, reducing the accuracy of the detection.

Method used

The original light image is converted into a parameter matrix through the feature pyramid, the image brightness is adjusted to obtain the dark light image to be processed, and the feature extraction network is used for multi-layer feature extraction and feature completion, and text detection is performed by combining the output network output heat map and text prediction box for output text. , avoid brightness enhancement and artificial features introduction.

Benefits of technology

It improves the accuracy of text detection in dark light environments, enhances the accuracy of feature completion and resource utilization, and directly conducts text detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113936128B_ABST
    Figure CN113936128B_ABST
Patent Text Reader

Abstract

The present invention provides a text detection method, device, electronic device and storage medium, the method comprising: converting an original illuminated image into a corresponding parameter matrix through a feature pyramid, adjusting the image brightness of the original illuminated image based on the parameter matrix, and obtaining a dark-light image to be processed corresponding to the original illuminated image; performing multi-layer feature extraction on the original dark-light image corresponding to the original illuminated image and the dark-light image to be processed based on a feature extraction network, respectively, to obtain original features and supplementary features corresponding to the original dark-light image; performing feature completion on the original features based on the supplementary features, to obtain a feature-completed original dark-light image; outputting a heat map and a text prediction frame of the feature-completed original dark-light image based on an output network, and performing text detection based on the heat map and the text prediction frame. The present invention directly detects text in a dark-light environment by synthesizing the feature completion of the dark-light image and the original dark-light image, thereby improving the accuracy of text detection in a dark-light environment.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a text detection method, device, electronic equipment and storage medium. Background Art

[0002] The current text detection-related work is basically carried out for normal lighting conditions, and text detection work in dark light environments is very scarce. At present, dark light text detection mainly uses a technical method to enhance the brightness of the image before detection, that is, first use traditional image processing methods to enhance the image brightness, then use MSER to predict candidate results containing false detection objects, and then combine manual features and convolutional neural networks to learn the distribution of brightness-enhanced text, eliminate non-text candidates, and obtain the final detection result. However, in this method, the noise in the image is amplified by the image brightness enhancement method without reference. At the same time, manual features need to be introduced manually, which reduces the accuracy of text detection in dark light environments. Summary of the invention

[0003] The present invention provides a text detection method, device, electronic device and storage medium, aiming to improve the accuracy of text detection in a dark environment.

[0004] The present invention provides a text detection method, comprising:

[0005] The original light image is converted into a corresponding parameter matrix through a feature pyramid, and the image brightness of the original light image is adjusted based on the parameter matrix to obtain a dark light image to be processed corresponding to the original light image;

[0006] Based on the feature extraction network, multi-layer feature extraction is performed on the original dark-light image corresponding to the original illuminated image and the dark-light image to be processed, respectively, to obtain original features and supplementary features corresponding to the original dark-light image;

[0007] Performing feature completion on the original feature based on the supplementary feature to obtain a feature-completed original dark-light image;

[0008] The heat map and text prediction box of the original dark-light image with feature completion are output based on the output network, and text detection is performed based on the heat map and the text prediction box.

[0009] According to a text detection method provided by the present invention, the step of performing text detection based on the heat map and the text prediction box includes:

[0010] Determine, based on the heat map, a pixel score corresponding to each pixel point in the original dark-light image of the feature completion;

[0011] A corresponding positive sample area is determined based on each of the pixel scores, and text detection is performed based on the positive sample area and the text prediction box.

[0012] According to a text detection method provided by the present invention, the original features include a first part of original features and a second part of original features.

[0013] The step of performing feature completion on the original feature based on the supplementary feature to obtain the feature-completed original dark-light image comprises:

[0014] Inputting the supplementary features and the first part of the original features into a first feature fusion module, wherein the first feature fusion module includes a first merging function and a first convolutional layer;

[0015] Based on the first merging function and the first convolutional layer, completing the supplementary features into the first part of original features;

[0016] The second part of original features and their corresponding upper-level features are subjected to feature fusion to obtain the original dark-light image with feature completion.

[0017] According to a text detection method provided by the present invention, the second part of original features includes a first feature and a second feature, the first feature is at the bottom layer of the second feature,

[0018] The step of fusing the second part of original features and their corresponding upper-level features to obtain the feature-completed original dark-light image comprises:

[0019] Obtaining a first upper-level feature corresponding to the first feature, and inputting the first feature and the first upper-level feature into a second feature fusion module, wherein the second feature fusion module includes a second merging function and a second convolutional layer;

[0020] Based on the second merging function and the second convolutional layer, performing a first feature fusion on the first feature and the first previous-level feature;

[0021] Obtain a second upper-level feature corresponding to the second feature, and perform a second feature fusion on the second feature and the second upper-level feature based on the second merging function and the second convolutional layer;

[0022] Based on the first feature fusion and the second feature fusion, the original dark-light image with feature completion is obtained.

[0023] According to a text detection method provided by the present invention, the step of adjusting the image brightness of the original light image based on the parameter matrix to obtain a dark light image to be processed corresponding to the original light image comprises:

[0024] Determine the pixel value of each image point in the original illumination image, and iteratively adjust each pixel value based on the parameter matrix in combination with a preset iterative function to obtain an adjusted pixel value corresponding to each pixel value;

[0025] The image brightness of the original illuminated image is adjusted based on each of the adjusted pixel values ​​and the constraint information of the average brightness to obtain the dark-light image to be processed.

[0026] According to a text detection method provided by the present invention, the step of performing multi-layer feature extraction on the original dark-light image corresponding to the original illuminated image based on a feature extraction network to obtain the original features corresponding to the original dark-light image includes:

[0027] Based on the feature extraction network, multi-layer feature extraction is performed on the original dark-light image to obtain preset layer features corresponding to the original dark-light image, and the preset layer features are determined as the original features.

[0028] According to a text detection method provided by the present invention, before the step of converting an original light image into a corresponding parameter matrix through a feature pyramid and adjusting the image brightness of the original light image based on the parameter matrix to obtain a dark light image to be processed corresponding to the original light image, the method further includes:

[0029] Using a camera to capture an image with text data under a preset normal lighting environment, and obtaining a corresponding original lighting image;

[0030] An image corresponding to the original illumination image in a preset dark light environment is captured by the camera device to obtain an original dark light image corresponding to the original illumination image.

[0031] The present invention also provides a text detection device, comprising:

[0032] A synthesis module, used to convert the original light image into a corresponding parameter matrix through a feature pyramid, and adjust the image brightness of the original light image based on the parameter matrix to obtain a dark light image to be processed corresponding to the original light image;

[0033] An extraction module, configured to perform multi-layer feature extraction on an original dark-light image corresponding to the original illuminated image and the dark-light image to be processed based on a feature extraction network, respectively, to obtain original features and supplementary features corresponding to the original dark-light image;

[0034] A fusion module, used for completing the original features based on the supplementary features to obtain an original dark-light image with completed features;

[0035] The detection module is used to output the heat map and text prediction box of the original dark light image with feature completion based on the output network, and perform text detection based on the heat map and the text prediction box.

[0036] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any of the above-mentioned text detection methods are implemented.

[0037] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned text detection methods are implemented.

[0038] The present invention also provides a computer program product, which includes a computer program, and when the computer program is executed by the processor, the steps of any of the above text detection methods are implemented.

[0039] The text detection method, device, electronic device and storage medium provided by the present invention convert the original light image into the corresponding parameter matrix through the feature pyramid, and adjust the image brightness of the original light image based on the parameter matrix to obtain the dark light image to be processed corresponding to the original light image; perform multi-layer feature extraction on the original dark light image corresponding to the original light image and the dark light image to be processed based on the feature extraction network, respectively, to obtain the original features and supplementary features corresponding to the original dark light image; perform feature completion on the original features based on the supplementary features to obtain the feature-completed original dark light image; output the heat map and text prediction box of the feature-completed original dark light image based on the output network, and perform text detection based on the heat map and the text prediction box. It can be seen that the present invention synthesizes the dark light image, and performs feature completion on the original dark light image based on the synthesized dark light data, without enhancing the image brightness and introducing manual features manually, so as to directly detect the text in the dark light environment, thereby improving the accuracy of text detection in the dark light environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 It is one of the flowcharts of the text detection method provided by the present invention;

[0042] Figure 2This is the second flow chart of the text detection method provided by the present invention;

[0043] Figure 3 is a structural schematic diagram of a text detection device provided by the present invention;

[0044] Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] Combine the following Figures 1 to 4 The present invention describes a text detection method, device, electronic device and storage medium.

[0047] Specifically, the present invention provides a text detection method, referring to Figure 1 , Figure 1 This is one of the flow charts of the text detection method provided by the present invention.

[0048] The embodiments of the present invention provide embodiments of the text detection method. It should be noted that although the logical order is shown in the flow chart, under certain data, the steps shown or described can be completed in an order different from that shown here.

[0049] The method of the embodiment of the present invention takes an electronic device as an example of an execution subject. The text detection method provided by the embodiment of the present invention includes:

[0050] Step S10, converting the original light image into a corresponding parameter matrix through a feature pyramid, and adjusting the image brightness of the original light image based on the parameter matrix to obtain a dark light image to be processed corresponding to the original light image.

[0051] It should be noted that the text detection system in the embodiment of the present invention is one of the manifestations of electronic devices and does not limit electronic devices. The text detection system in this embodiment at least includes a feature pyramid, a feature extraction network and an output network.

[0052] It should be further explained that before processing the original illuminated image, the text detection system needs to use a camera device (camcorder, video camera, etc.) to take pictures in a normal lighting environment and a dark light environment to obtain a data set of original illuminated images in a normal lighting environment and original dark light images in a dark light environment, as shown in steps a to b.

[0053] Step a, photographing an image with text data under a preset normal lighting environment by a camera device to obtain a corresponding original lighting image;

[0054] Step b: photographing an image corresponding to the original illumination image in a preset dark light environment by the imaging device to obtain an original dark light image corresponding to the original illumination image.

[0055] Specifically, the text detection system uses a camera to shoot an image with text data under a preset normal lighting environment to obtain an original lighting image under the preset normal lighting environment. At the same time, the text data in the original lighting image needs to be labeled to obtain an original lighting image with text label data. Next, the text detection system needs to use a camera to shoot an image corresponding to the original lighting image under a preset dark light environment to obtain an original dark light image corresponding to the original lighting image under the preset dark light environment. At the same time, the text data in the original dark light image needs to be labeled to obtain an original dark light image with text label data. Among them, the preset normal lighting environment and the preset dark light environment are determined according to actual conditions, and this embodiment does not limit this. It should be further explained that the text detection method of this embodiment is based on a data set of original lighting images with text label data and original dark light images with text label data.

[0056] This embodiment collects a data set of original illuminated images with text label data and original dark-light images with text label data, and performs text detection through the data set of original illuminated images with text label data and original dark-light images with text label data, without the need for manual shooting and manual production of data sets, thereby ensuring the accuracy of the original illuminated images and the original dark-light images in the data set, thereby improving the accuracy of text detection in dark-light environments.

[0057] Furthermore, after obtaining the data set, the text detection system normalizes the original illuminated image through Unet (one of the specific forms of feature pyramid), normalizes each pixel in the original illuminated image to a parameter corresponding to 0 to 1, and all the normalized parameters constitute a parameter matrix corresponding to the original illuminated image, wherein it should be noted that the size of the parameter matrix is ​​consistent with the size of the original illuminated image. Next, the text detection system adjusts the image brightness of the original illuminated image based on the parameter matrix to obtain a dark light image to be processed corresponding to the original illuminated image, as shown in steps S101 to S102.

[0058] Step S101, determining the pixel value of each image point in the original illumination image, and iteratively adjusting each pixel value based on the parameter matrix combined with a preset iterative function to obtain an adjusted pixel value corresponding to each pixel value;

[0059] Step S102, adjusting the image brightness of the original illuminated image based on each of the adjusted pixel values ​​and constraint information of the average brightness to obtain the dark-light image to be processed.

[0060] Specifically, the text detection system determines the pixel value of each image point in the original illumination image, and iteratively adjusts each pixel value based on the parameter matrix combined with a preset iterative function. The preset iterative function is LE(I(x); α) = I(x) + A n I(x)(1-I(x)), I(x) is the pixel value in the original illuminated image at position x in the original illuminated image, A(n) is the parameter matrix of the nth iteration, and after adjustment of the parameter matrix and the preset iteration function, the adjusted pixel values ​​corresponding to each pixel value are obtained. Next, the text detection system adjusts each adjusted pixel value according to the constraint information of the average brightness to obtain a dark-light image to be processed corresponding to the original illuminated image, wherein it should be noted that, since the size of the dark-light image to be processed is the same as that of the original illuminated image before and after the process of synthesizing the dark-light image to be processed, the text label data in the dark-light image to be processed is the same as that in the original illuminated image, that is, the text label data in the dark-light image to be processed can be directly used.

[0061] Furthermore, the preset iterative function LE(I(x); α) = I(x) + A nThe specific analysis of I(x)(1-I(x)) is as follows: Let f(x)=x+αx(1-x), f′(x)=1+α-2αx, f″(x)=-2α, when α∈[-1,0], f″(x)=-2α>0>0, monotonically increasing, then f′(x)=1+α-2αx∈[1+a,1-a], f(x)=x+αx(1-x) monotonically increasing. f(x)∈[0,1]. When α∈[-1,0], f″(x)=-2α<0, and it is monotonically decreasing. Then f′(x)=1+α-2αx∈[1-a,1+a], and f(x)=x+αx(1-x) is monotonically increasing. f(x)∈[0,1]. In other words, the pixel value range of both input and output can be kept in the range of 0 to 1, and for the same parameter, the iterative function remains monotonic.

[0062] This embodiment synthesizes the dark-light image to be processed of the original light image through a parameter matrix and a preset iterative function, without the need to manually produce the dark-light image, thereby maximizing resources. At the same time, since the size of the dark-light image to be processed is the same as that of the original light image before and after synthesis, the text label data in the original light image is not changed, thereby ensuring the accuracy of the dark-light image to be processed, thereby ensuring the accuracy of text detection in a dark-light environment.

[0063] Step S20, performing multi-layer feature extraction on the original dark-light image corresponding to the original illuminated image and the dark-light image to be processed based on a feature extraction network, and obtaining original features and supplementary features corresponding to the original dark-light image respectively.

[0064] After the text detection system obtains the dark-light image to be processed corresponding to the original light image, it is necessary to input the original dark-light image corresponding to the original light image and the dark-light image to be processed into the feature extraction network, and perform multi-layer feature extraction on the original dark-light image and the dark-light image to be processed through the feature extraction network to obtain the original features of the original dark-light image and the corresponding supplementary features. In this embodiment, the Resnet50 feature extractor is used as one of the specific ways of the feature extraction network, as shown in step S201.

[0065] Step S201, performing multi-layer feature extraction on the original dark-light image based on the feature extraction network to obtain preset layer features corresponding to the original dark-light image, and determining the preset layer features as the original features.

[0066] Specifically, the text detection system performs multi-layer feature extraction on the original dark-light image through the Resnet50 feature extractor to obtain preset layer features corresponding to the original dark-light image. It should be noted that the preset layer features in this embodiment are 4 layers of features, that is, 4 layers of features of the original dark-light image are extracted through the Resnet50 feature extractor, and the 4 layers of features are determined as 4 layers of original features of the original dark-light image, wherein each layer of original features has its corresponding number of feature channels, and the number of feature channels is 2048, 1024, 512 and 256 from high to low, respectively, and the feature sizes of the number of feature channels from high to low are 1 / 32, 1 / 16, 1 / 8 and 1 / 4, respectively, relative to the original dark-light image. It can be further understood that the original features with a feature channel number of 2048 are the highest layer original features, the original features with a feature channel number of 256 are the lowest layer original features, and the original features with feature channel numbers of 1024 and 512 are the middle layer original features, which are sorted in order of the number of feature channels.

[0067] Similarly, the text detection system uses the Resnet50 feature extractor to perform multi-layer feature extraction on the dark-light image to be processed, and obtains the preset layer features corresponding to the dark-light image to be processed, that is, the 4 layers of features of the dark-light image to be processed are extracted by the Resnet50 feature extractor, and the 4 layers of features are determined as the 4 layers of supplementary features of the original dark-light image, wherein each layer of supplementary features has its corresponding number of feature channels, and the number of feature channels from high to low are 2048, 1024, 512 and 256 respectively, and the feature sizes from high to low are 1 / 32, 1 / 16, 1 / 8 and 1 / 4 respectively relative to the dark-light image to be processed. It can be further understood that the supplementary features with a feature channel number of 2048 are the highest layer supplementary features, the supplementary features with a feature channel number of 256 are the lowest layer supplementary features, and the supplementary features with feature channel numbers of 1024 and 512 are the middle layer supplementary features, which are sorted in order of the number of feature channels.

[0068] It should be further explained that before the text detection system performs multi-layer feature extraction on the dark-light image to be processed through the Resnet50 feature extractor, it is necessary to adjust the dark-light image to be processed pixel by pixel through steps S101 to S102, wherein the constraint information for adjusting the image brightness comes from the detection result of the dark-light image to be processed by the Resnet50 feature extractor. Since the parameters in the Resnet50 feature extractor are variable, the detection results of the Resnet50 feature extractor are different for different images.

[0069] This embodiment extracts the original features and supplementary features of the original dark-light image through a feature extraction network without the need to introduce features artificially. While ensuring that the extracted original features and supplementary features are complete, the accuracy of the original features and supplementary features is guaranteed, thereby ensuring the accuracy of text detection in a dark-light environment.

[0070] Step S30, completing the original features based on the supplementary features to obtain a feature-completed original dark-light image.

[0071] After the text detection system obtains the original features and supplementary features of the original dark-light image, it uses the top two layers of supplementary features in the supplementary features to complete the top two layers of original features in the original features. Then, the text detection system performs feature fusion on the remaining two layers of original features with their corresponding upper-level features to obtain the original dark-light image with feature completion.

[0072] Step S40, outputting a heat map and a text prediction box of the original dark-light image with feature completion based on the output network, and performing text detection based on the heat map and the text prediction box.

[0073] The text detection system outputs the heat map and text prediction box of the original dark light image with feature completion through the output network, wherein the text prediction box generates the corresponding text prediction box according to the text label data of the original dark light image with feature completion, that is, a certain position area is circled as the text prediction box according to the text label data of the original dark light image with feature completion. The heat map is a pixel score map, that is, the heat map contains the pixel score of each pixel point of the completed original dark light image, and the probability of belonging to text is determined according to the pixel score. The heat map is a single-channel map. Next, the text detection system performs text detection on the original dark light image with feature completion according to the heat map and the text prediction box, as shown in steps S401 to S402.

[0074] Step S401, determining the pixel score corresponding to each pixel point in the original dark-light image with feature completion based on the heat map;

[0075] Step S402: determining a corresponding positive sample area based on each of the pixel scores, and performing text detection based on the positive sample area and the text prediction box.

[0076] Specifically, the text detection system determines the pixel scores corresponding to each pixel of the original dark-light image of feature completion in the heat map, and determines whether there is a target pixel score, wherein the target pixel score is greater than the preset score threshold, that is, when the target pixel score is greater than the preset score threshold, it indicates that the pixel corresponding to the target pixel score is very likely to be text. Therefore, if it is determined that there is a target pixel score, the text detection system determines the pixel points (target pixel points) corresponding to all target pixel scores, and determines the positive sample area corresponding to the original dark-light image of feature completion according to all target pixel points, and the positive sample area is also the area where text may appear. It should be noted that in addition to outputting the above-mentioned heat map and text prediction box, the distance map of the original dark-light image of feature completion is also output. The distance map is a five-channel map, the first four channels are the four-side distances from each pixel to the positive sample area, and the fifth channel is the corresponding rotation angle of the positive sample area. Then, the text detection system performs text detection on the original dark-light image of feature completion according to the text prediction box of the positive sample area to obtain the text in the original dark-light image of feature completion.

[0077] Furthermore, for the heat map and the text prediction box, the loss function between the heat map and the text prediction box is as follows: L = L s +λ b L bbox , where L s is the loss function of the heat map, L bbox is the loss function of the text prediction box, λ b is the coefficient, which is generally set to 1. For the loss function L of the heat map s ,

[0078] L s =L s1 +L s2

[0079] L s1 = -Ylog(X)-(1-Y)log(1-X)

[0080]

[0081] Among them, X is the predicted positive sample area, and Y is the text label data corresponding to the positive sample area. For the loss function of the heat map, for the loss function of the text prediction box L bbox ,

[0082]

[0083] Among them, P is the text prediction box, P* is the text label data corresponding to the text prediction box, and λ θ is the angle parameter, and the value of the angle parameter is generally set to 20.

[0084] This embodiment determines the positive sample area corresponding to the original dark-light image with feature completion by outputting the heat map output by the network, and then performs text detection on the original dark-light image with feature completion based on the text prediction box in the positive sample area to obtain the text in the original dark-light image with feature completion, thereby ensuring the accuracy of text detection in a dark-light environment.

[0085] The present embodiment provides a text detection method, which converts an original illuminated image into a corresponding parameter matrix through a feature pyramid, and adjusts the image brightness of the original illuminated image based on the parameter matrix to obtain a dark-light image to be processed corresponding to the original illuminated image; based on a feature extraction network, multi-layer feature extraction is performed on the original dark-light image corresponding to the original illuminated image and the dark-light image to be processed, respectively, to obtain the original features and supplementary features corresponding to the original dark-light image; based on the supplementary features, feature completion is performed on the original features to obtain the feature-completed original dark-light image; based on the output network, a heat map and a text prediction box of the feature-completed original dark-light image are output, and text detection is performed based on the heat map and the text prediction box. It can be seen that the present embodiment directly detects text in a dark-light environment by synthesizing a dark-light image and performing feature completion on the original dark-light image based on the synthesized dark-light data, thereby improving the accuracy of text detection in a dark-light environment.

[0086] Further, refer to Figure 2 , Figure 2 FIG. 2 is a flow chart of the text detection method provided by the present invention, wherein step S30 comprises:

[0087] Step S301, inputting the supplementary features and the first part of the original features into a first feature fusion module, wherein the first feature fusion module includes a first merging function and a first convolutional layer;

[0088] Step S302: based on the first merging function and the first convolutional layer, completing the supplementary features into the first part of the original features;

[0089] Step S303, performing feature fusion on the second part of the original features and the corresponding upper-level features to obtain the original dark-light image with the features completed.

[0090] It should be noted that step S201 describes that there are four layers of supplementary features, and therefore the supplementary features are divided into a first part of original features and a second part of original features. The first part of original features is the highest two layers of original features, and the second part of original features is the lowest two layers of original features.

[0091] The text detection system inputs the top two layers of supplementary features and the first part of original features in the supplementary features into the first feature fusion module, that is, the top two layers of supplementary features in the supplementary features and the top two layers of original features in the original features are input into the first feature fusion module, wherein the first feature fusion module includes a first merging function concat and a first convolution layer, the first convolution layer is a two-layer 1×1 convolution layer, and the top two layers of supplementary features in the supplementary features and the top two layers of original features in the original features are feature-completed through the first merging function concat and the two-layer 1×1 convolution. Next, the text detection system performs feature fusion on the second part of the original features and their corresponding upper-level features, that is, the bottom two layers of original features in the original features and their corresponding upper-level features are feature-fused to obtain the original dark-light image with feature completion, as specifically shown in steps S3031 to S3034.

[0092] Step S3031, obtaining a first upper-level feature corresponding to the first feature, and inputting the first feature and the first upper-level feature into a second feature fusion module, wherein the second feature fusion module includes a second merging function and a second convolutional layer;

[0093] Step S3032: performing a first feature fusion on the first feature and the first previous level feature based on the second merging function and the second convolutional layer;

[0094] Step S3033, obtaining a second upper-level feature corresponding to the second feature, and performing a second feature fusion on the second feature and the second upper-level feature based on the second merging function and the second convolutional layer;

[0095] Step S3034, based on the first feature fusion and the second feature fusion, obtain the original dark light image with feature completion.

[0096] Specifically, the second part of the original features also includes two layers of original features, so the second part of the original features can be divided into the first feature and the second feature. There is no substantial difference between the first feature and the second feature, but the first and second features are distinguished by different levels, that is, the first feature is at the bottom of the second feature, that is, the first feature is the bottom-level original feature in the original features, and the second feature is the second-to-last original feature in the original features.

[0097] The text detection system obtains the first upper-level feature corresponding to the first feature, and inputs the first feature and the first upper-level feature into the second feature fusion module, that is, the bottom-level original feature in the original feature and its corresponding first upper-level feature are input into the second feature fusion module, wherein the second feature fusion module includes a second merging function concat and a second convolutional layer, the second convolutional layer is composed of a 3×3 convolutional layer and a 1×1 convolutional layer, and the bottom-level original feature in the original feature and its corresponding first upper-level feature are subjected to the first feature fusion through the second merging function concat, and the 3×3 convolutional layer and the 1×1 convolutional layer in the second convolutional layer. Next, the text detection system obtains the second-upper-level feature corresponding to the second feature, that is, the second-upper-level feature corresponding to the second-to-last layer of original features in the original features, and inputs the second-to-last layer of original features in the original features and their corresponding second-upper-level features into the second feature fusion module, and performs a second feature fusion on the second-to-last layer of original features in the original features and their corresponding second-upper-level features through the second merging function concat and the 3×3 convolution layer and the 1×1 convolution layer in the second convolution layer. Through the first feature fusion and the second feature fusion, all four layers of original features are completed to obtain the original dark light image with feature completion.

[0098] The present embodiment provides a text detection method, which inputs the supplementary features and the first part of the original features into the first feature fusion module, wherein the first feature fusion module includes a first merging function and a first convolutional layer; based on the first merging function and the first convolutional layer, the supplementary features are completed to the first part of the original features; the second part of the original features and their corresponding upper-level features are feature fused to obtain the feature-completed original dark-light image. It can be seen that the present embodiment first performs feature completion on the highest two layers of original features in the original features, and then performs feature fusion on the lowest two layers of original features in the original features, making full use of global feature information and local feature information, thereby obtaining a feature-completed original dark-light image with clearer and richer detail information, thereby ensuring the accuracy of text detection in a dark-light environment.

[0099] Furthermore, the text detection device provided by the present invention is described below. The text detection device described below and the text detection method described above can be referred to each other.

[0100] like Figure 3 As shown, Figure 3 : is a structural schematic diagram of a text detection device provided by the present invention, the text detection device comprises:

[0101] A synthesis module 301 is used to convert the original light image into a corresponding parameter matrix through a feature pyramid, and adjust the image brightness of the original light image based on the parameter matrix to obtain a dark light image to be processed corresponding to the original light image;

[0102] An extraction module 302 is used to perform multi-layer feature extraction on the original dark-light image corresponding to the original illuminated image and the dark-light image to be processed based on a feature extraction network, and obtain original features and supplementary features corresponding to the original dark-light image respectively;

[0103] A fusion module 303 is used to complete the original feature based on the supplementary feature to obtain a feature-completed original dark-light image;

[0104] The detection module 304 is used to output the heat map and text prediction box of the original dark light image with feature completion based on the output network, and perform text detection based on the heat map and the text prediction box.

[0105] Furthermore, the detection module 304 is also used for:

[0106] Determine, based on the heat map, a pixel score corresponding to each pixel point in the original dark-light image of the feature completion;

[0107] A corresponding positive sample area is determined based on each of the pixel scores, and text detection is performed based on the positive sample area and the text prediction box.

[0108] Furthermore, the fusion module 303 is also used for:

[0109] Inputting the supplementary features and the first part of the original features into a first feature fusion module, wherein the first feature fusion module includes a first merging function and a first convolutional layer;

[0110] Based on the first merging function and the first convolutional layer, completing the supplementary features into the first part of original features;

[0111] The second part of original features and their corresponding upper-level features are subjected to feature fusion to obtain the original dark-light image with feature completion.

[0112] Furthermore, the fusion module 303 is also used for:

[0113] Obtaining a first upper-level feature corresponding to the first feature, and inputting the first feature and the first upper-level feature into a second feature fusion module, wherein the second feature fusion module includes a second merging function and a second convolutional layer;

[0114] Based on the second merging function and the second convolutional layer, performing a first feature fusion on the first feature and the first previous-level feature;

[0115] Obtain a second upper-level feature corresponding to the second feature, and perform a second feature fusion on the second feature and the second upper-level feature based on the second merging function and the second convolutional layer;

[0116] Based on the first feature fusion and the second feature fusion, the original dark-light image with feature completion is obtained.

[0117] Furthermore, the synthesis module 301 is also used for:

[0118] Determine the pixel value of each image point in the original illumination image, and iteratively adjust each pixel value based on the parameter matrix in combination with a preset iterative function to obtain an adjusted pixel value corresponding to each pixel value;

[0119] The image brightness of the original illuminated image is adjusted based on each of the adjusted pixel values ​​and the constraint information of the average brightness to obtain the dark-light image to be processed.

[0120] Furthermore, the extraction module 302 is also used for:

[0121] Based on the feature extraction network, multi-layer feature extraction is performed on the original dark-light image to obtain preset layer features corresponding to the original dark-light image, and the preset layer features are determined as the original features.

[0122] Furthermore, the text detection device also includes:

[0123] An acquisition module is used to capture an image with text data under a preset normal lighting environment by a camera device to obtain a corresponding original lighting image;

[0124] The acquisition module is also used to capture an image corresponding to the original illumination image in a preset dark light environment by using the camera device to obtain an original dark light image corresponding to the original illumination image.

[0125] The specific embodiments of the text detection device provided by the present invention are basically the same as the embodiments of the above-mentioned text detection method, and will not be described in detail here.

[0126] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the text detection method, which includes:

[0127] The original light image is converted into a corresponding parameter matrix through a feature pyramid, and the image brightness of the original light image is adjusted based on the parameter matrix to obtain a dark light image to be processed corresponding to the original light image;

[0128] Based on the feature extraction network, multi-layer feature extraction is performed on the original dark-light image corresponding to the original illuminated image and the dark-light image to be processed, respectively, to obtain original features and supplementary features corresponding to the original dark-light image;

[0129] Performing feature completion on the original feature based on the supplementary feature to obtain a feature-completed original dark-light image;

[0130] The heat map and text prediction box of the original dark-light image with feature completion are output based on the output network, and text detection is performed based on the heat map and the text prediction box.

[0131] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0132] On the other hand, the present invention further provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, when the program instructions are executed by a computer, the computer can execute the text detection method provided by the above methods, the method comprising:

[0133] The original light image is converted into a corresponding parameter matrix through a feature pyramid, and the image brightness of the original light image is adjusted based on the parameter matrix to obtain a dark light image to be processed corresponding to the original light image;

[0134] Based on the feature extraction network, multi-layer feature extraction is performed on the original dark-light image corresponding to the original illuminated image and the dark-light image to be processed, respectively, to obtain original features and supplementary features corresponding to the original dark-light image;

[0135] Performing feature completion on the original feature based on the supplementary feature to obtain a feature-completed original dark-light image;

[0136] The heat map and text prediction box of the original dark-light image with feature completion are output based on the output network, and text detection is performed based on the heat map and the text prediction box.

[0137] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the above-mentioned text detection method, the method comprising:

[0138] The original light image is converted into a corresponding parameter matrix through a feature pyramid, and the image brightness of the original light image is adjusted based on the parameter matrix to obtain a dark light image to be processed corresponding to the original light image;

[0139] Based on the feature extraction network, multi-layer feature extraction is performed on the original dark-light image corresponding to the original illuminated image and the dark-light image to be processed, respectively, to obtain original features and supplementary features corresponding to the original dark-light image;

[0140] Performing feature completion on the original feature based on the supplementary feature to obtain a feature-completed original dark-light image;

[0141] The heat map and text prediction box of the original dark-light image with feature completion are output based on the output network, and text detection is performed based on the heat map and the text prediction box.

[0142] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0143] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0144] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A text detection method, characterized in that: include: The original light image is converted into a corresponding parameter matrix through a feature pyramid, and the image brightness of the original light image is adjusted based on the parameter matrix to obtain a dark light image to be processed corresponding to the original light image; Based on the feature extraction network, multi-layer feature extraction is performed on the original dark-light image corresponding to the original illuminated image and the dark-light image to be processed, respectively, to obtain original features and supplementary features corresponding to the original dark-light image; Performing feature completion on the original feature based on the supplementary feature to obtain a feature-completed original dark-light image; Outputting a heat map and a text prediction box of the original dark-light image with feature completion based on the output network, and performing text detection based on the heat map and the text prediction box; The original features include a first part of original features and a second part of original features, The step of completing the original feature based on the supplementary feature to obtain the feature-completed original dark-light image comprises: Inputting the supplementary features and the first part of the original features into a first feature fusion module, wherein the first feature fusion module includes a first merging function and a first convolutional layer; Based on the first merging function and the first convolutional layer, completing the supplementary features into the first part of original features; The second part of original features and their corresponding upper-level features are subjected to feature fusion to obtain the original dark-light image with feature completion.

2. The text detection method according to claim 1, characterized in that: The step of performing text detection based on the heat map and the text prediction box includes: Determine, based on the heat map, a pixel score corresponding to each pixel point in the original dark-light image of the feature completion; A corresponding positive sample area is determined based on each of the pixel scores, and text detection is performed based on the positive sample area and the text prediction box.

3. The text detection method according to claim 1, characterized in that: The second part of original features includes a first feature and a second feature, the first feature is at the bottom layer of the second feature, The step of fusing the second part of original features and their corresponding upper-level features to obtain the feature-completed original dark-light image comprises: Obtaining a first upper-level feature corresponding to the first feature, and inputting the first feature and the first upper-level feature into a second feature fusion module, wherein the second feature fusion module includes a second merging function and a second convolutional layer; Based on the second merging function and the second convolutional layer, performing a first feature fusion on the first feature and the first previous-level feature; Acquire a second upper-level feature corresponding to the second feature, and perform a second feature fusion on the second feature and the second upper-level feature based on the second merging function and the second convolutional layer; Based on the first feature fusion and the second feature fusion, the original dark-light image with feature completion is obtained.

4. The text detection method according to claim 1, characterized in that: The step of adjusting the image brightness of the original light image based on the parameter matrix to obtain a dark light image to be processed corresponding to the original light image comprises: Determine the pixel value of each image point in the original illumination image, and iteratively adjust each pixel value based on the parameter matrix in combination with a preset iterative function to obtain an adjusted pixel value corresponding to each pixel value; The image brightness of the original illuminated image is adjusted based on each of the adjusted pixel values ​​and the constraint information of the average brightness to obtain the dark-light image to be processed.

5. The text detection method according to claim 1, characterized in that: The step of performing multi-layer feature extraction on the original dark-light image corresponding to the original illuminated image based on the feature extraction network to obtain the original features corresponding to the original dark-light image comprises: Based on the feature extraction network, multi-layer feature extraction is performed on the original dark-light image to obtain preset layer features corresponding to the original dark-light image, and the preset layer features are determined as the original features.

6. The text detection method according to any one of claims 1 to 5, characterized in that: Before the step of converting the original light image into a corresponding parameter matrix through the feature pyramid and adjusting the image brightness of the original light image based on the parameter matrix to obtain a dark light image to be processed corresponding to the original light image, the step further includes: Using a camera to capture an image with text data under a preset normal lighting environment, and obtaining a corresponding original lighting image; An image corresponding to the original illumination image in a preset dark light environment is captured by the camera device to obtain an original dark light image corresponding to the original illumination image.

7. A text detection device, characterized in that: include: A synthesis module, used to convert the original light image into a corresponding parameter matrix through a feature pyramid, and adjust the image brightness of the original light image based on the parameter matrix to obtain a dark light image to be processed corresponding to the original light image; An extraction module, configured to perform multi-layer feature extraction on an original dark-light image corresponding to the original illuminated image and the dark-light image to be processed based on a feature extraction network, respectively, to obtain original features and supplementary features corresponding to the original dark-light image; A fusion module, used for completing the original features based on the supplementary features to obtain an original dark-light image with completed features; A detection module, configured to output a heat map and a text prediction box of the original dark-light image with feature completion based on an output network, and perform text detection based on the heat map and the text prediction box; The original features include a first part of original features and a second part of original features, The step of completing the original feature based on the supplementary feature to obtain the feature-completed original dark-light image comprises: Inputting the supplementary features and the first part of the original features into a first feature fusion module, wherein the first feature fusion module includes a first merging function and a first convolutional layer; Based on the first merging function and the first convolutional layer, completing the supplementary features into the first part of original features; The second part of original features and their corresponding upper-level features are subjected to feature fusion to obtain the original dark-light image with feature completion.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the text detection method according to any one of claims 1 to 6 are implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the text detection method according to any one of claims 1 to 6 are implemented.