Text detection model training method, text detection method, device and electronic equipment

By searching for the appropriate inverting distance in the text detection model for internal and external expansion, the problems of text adhesion and low accuracy in the text detection model are solved, and the text positioning accuracy and the accuracy of training labels are improved.

CN118823787BActive Publication Date: 2024-12-03HANGZHOU HIKVISION DIGITAL TECHNOLOGY CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202411272987.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-11
Publication Date
2024-12-03
Estimated Expiration
2044-09-11

AI Technical Summary

Technical Problem

When the existing text detection model deals with diverse text lines, there is a problem of text obsession, resulting in low detection accuracy and the inflection and expansion method cannot achieve consistency, which affects the text positioning accuracy of the model.

Method used

By searching for an appropriate inverted distance within the inverted distance of the text area, the area of ​​the out-expanded area is the same as the area of ​​the text area, and then the out-expanded are generated to generate segmented map labels, threshold map labels and binary maps Tags, used to train text detection models.

Benefits of technology

The consistency processing of text area area is achieved, text obsession and information loss are avoided, and the positioning accuracy of text detection model and the accuracy of training labels are improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118823787B_ABST
    Figure CN118823787B_ABST
Patent Text Reader

Abstract

The embodiments of the present application provide a text detection model training method, a text detection method, a device and an electronic device. The text detection model training method includes: for each text area in each training image, searching for the corresponding indentation distance of the text area according to the preset search target within the indentation distance range corresponding to the text area; shrinking each text area based on the indentation distance corresponding to each text area in the training image to obtain a shrinkage image of the training image; expanding each text area to obtain an expanded area corresponding to each text area; generating a segmentation map label, a threshold map label and a binary map label for each training image; inputting the training image into the text detection model to be trained, and training based on the differences between the predicted probability map, the predicted threshold map and the predicted binary map output by the model and the segmentation map label, the threshold map label and the binary map label, respectively, to obtain a text detection model. Thereby improving the text positioning accuracy of the text detection model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of text detection technology, and in particular to a text detection model training method, a text detection method, a device and an electronic device. Background Art

[0002] Text detection is increasingly used in various fields, for example, in image search, identity authentication, and visual navigation. Text detection refers to the process of locating and identifying text areas in digitized images or videos during the OCR (Optical Character Recognition) process.

[0003] In actual application scenarios, due to the diversity of text line size, font, color, shape, direction and background, there may be text adhesion, which will affect the accuracy of text detection. In order to overcome this problem, when using the semantic segmentation-based text detection method to detect text on an image, it is generally necessary to shrink the detected text box first and then expand it. Specifically, the vatti_clipping algorithm is used to shrink the text box according to fixed shrinking experience parameters to obtain the shrinking area, and then the shrinking area is expanded according to fixed expansion experience parameters to obtain the expansion area. After that, the text box, the shrinking area and the expansion area can be used for model training to obtain a text detection model.

[0004] Under ideal conditions, the shapes of the regions before and after shrinking and expanding should be as close as possible. However, the current shrinking and expanding methods often fail to achieve consistency. For example, for a long text region, according to the current shrinking and expanding methods, the shapes of the regions before and after shrinking and expanding are quite different. The shrinking box is too small, so that the shrinking box cannot include all the text in the long text region. This will result in low accuracy of text positioning of the text detection model obtained by training the shrinking area and the expanding area obtained by the shrinking and expanding methods. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide a text detection model training method, a text detection method, a device and an electronic device to improve the accuracy of text positioning of the text detection model. The specific technical solution is as follows:

[0006] In a first aspect, an embodiment of the present application provides a text detection model training method, the method comprising:

[0007] For each text region in each training image, searching for the corresponding inward contraction distance of the text region within the inward contraction distance range corresponding to the text region according to a first preset search target, wherein the first preset search target is the inward contraction distance obtained by searching so that the area of ​​the outward expansion region is the same as the area of ​​the text region, and the outward expansion region is the region obtained by inward contracting the text region according to the inward contraction distance and then outward expanding the text region;

[0008] Based on the shrinkage distance corresponding to each text area in each training image, shrinking each text area to obtain a shrinkage image of each training image;

[0009] Expand each text region in each training image to obtain an expanded region corresponding to each text region;

[0010] Based on the inner image of each training image and the outer expansion area corresponding to each text area in the training image, a segmentation image label, a threshold image label and a binary image label corresponding to the training image are generated;

[0011] Each training image is input into the text detection model to be trained, and the text detection model is trained based on the differences between the predicted probability map, the predicted threshold map and the predicted binary map output by the text detection model and the segmentation map label, the threshold map label and the binary map label, respectively, to obtain a trained text detection model.

[0012] Optionally, the step of searching for the indentation distance corresponding to the text area in each training image within the indentation distance range corresponding to the text area according to the first preset search target includes:

[0013] For each text area in each training image, the indentation distance range is determined by taking 0 as the left boundary of the indentation distance range and taking the target value as the right boundary of the indentation distance range, wherein the target value is determined based on the perimeter of the text area;

[0014] When the difference between the left boundary and the right boundary of the retracted distance range is greater than a preset accuracy threshold, determining an alternative retracted distance within the retracted distance range;

[0015] The retraction distance range is updated based on the alternative retraction distance, and the step of determining the alternative retraction distance within the retraction distance range when the difference between the left boundary and the right boundary of the retraction distance range is greater than the preset accuracy threshold is returned to execute, until the difference between the left boundary and the right boundary of the current retraction distance range is not greater than the preset accuracy threshold, and the retraction distance corresponding to the text area is determined based on the current retraction distance range.

[0016] Optionally, the step of updating the retraction distance range based on the candidate retraction distance includes:

[0017] In the case where the alternative inward contraction distance fails to be used for inward contraction, or the area of ​​the alternative outward expansion region obtained based on the alternative inward contraction distance is smaller than the area of ​​the text region, the alternative inward contraction distance is used as the right boundary of the inward contraction distance range to obtain an updated inward contraction distance range, wherein the area of ​​the alternative outward expansion region is the area of ​​the outward expansion region obtained based on the alternative inward contraction distance;

[0018] In the case that the area of ​​the candidate outward expansion region is larger than the area of ​​the text region, the candidate inward contraction distance is used as the left boundary of the inward contraction distance range to obtain an updated inward contraction distance range.

[0019] Optionally, the step of determining the indentation distance corresponding to the text area based on the current indentation distance range includes:

[0020] If the right boundary of the current shrinking distance range fails to shrink, or the absolute value of the first area difference is greater than the absolute value of the second area difference, the left boundary of the current shrinking distance range is returned as the shrinking distance corresponding to the text area, wherein the first area difference is the difference between the area of ​​the expanded area obtained based on the right boundary and the area of ​​the text area, and the second area difference is the difference between the area of ​​the expanded area obtained based on the left boundary and the area of ​​the text area;

[0021] When the absolute value of the first area difference is not greater than the absolute value of the second area difference, the right boundary of the current indentation distance range is returned as the indentation distance corresponding to the text area.

[0022] Optionally, the step of inputting each training image into a text detection model to be trained, and training the text detection model based on the differences between the predicted probability map, the predicted threshold map, and the predicted binary map output by the text detection model and the segmentation map label, the threshold map label, and the binary map label, respectively, to obtain a trained text detection model comprises:

[0023] Input each training image into the text detection model to be trained to obtain a prediction probability map, a prediction threshold map, and a prediction binary map;

[0024] Based on the segmentation map label, the threshold map label, the binary map label, the predicted probability map, the predicted threshold map, the predicted binary map and a preset function, calculating the loss value corresponding to the training image, wherein the preset function includes a cross entropy loss function, a DICE loss function and a pre-constructed intersection-over-union loss function, and the intersection-over-union loss function is only used to calculate the difference between the text area shape in the training image predicted by the text detection model and the text area shape represented by the segmentation map label corresponding to the training image;

[0025] The parameters of the text detection model are adjusted according to the loss value until the text detection model converges to obtain the text detection model.

[0026] Optionally, the step of calculating the loss value corresponding to the training image based on the segmentation map label, the threshold map label, the binary map label, the predicted probability map, the predicted threshold map, the predicted binary map and a preset function includes:

[0027] Based on the segmentation map label, the prediction probability map and the cross entropy loss function, calculating a first cross entropy loss value corresponding to the training image;

[0028] Based on the binary image label, the predicted binary image and the cross entropy loss function, calculating a second cross entropy loss value corresponding to the training image;

[0029] Calculating a DICE loss value corresponding to the training image based on the threshold map label, the predicted threshold map, and the DICE loss function;

[0030] Based on the segmentation map label, the predicted probability map and the pre-constructed intersection-and-union loss function, a first intersection-and-union loss value corresponding to the training image is calculated; and / or, based on the binary map label, the predicted binary map and the pre-constructed intersection-and-union loss function, a second intersection-and-union loss value corresponding to the training image is calculated;

[0031] Calculate the first cross entropy loss value, the second cross entropy loss value, the DICE loss value, the first intersection-over-union loss value, and / or the sum of the second intersection-over-union loss value to obtain a loss value corresponding to the training image.

[0032] Optionally, the step of calculating a first intersection-and-union loss value corresponding to the training image based on the segmentation map label, the predicted probability map, and a pre-constructed intersection-and-union loss function; and / or the step of calculating a second intersection-and-union loss value corresponding to the training image based on the binary map label, the predicted binary map, and a pre-constructed intersection-and-union loss function, comprises:

[0033] The first intersection-over-combination loss value and / or the second intersection-over-combination loss value are calculated according to the following formula:

[0034] ;

[0035] in, is the intersection-and-union loss value; is a pixel point in the prediction image output by the text detection model, is the pixel point in the label corresponding to the prediction image, is the length of the prediction graph, is the width of the prediction graph, For a pixel point in the prediction probability map, is the pixel point in the segmentation map label, For the pixel point in the predicted binary image, is a pixel point in the binary image label; for The weight of the region, If the area is a non-text area, ,exist If the area is a text area, , is the area of ​​the text region to which the pixel belongs.

[0036] In a second aspect, an embodiment of the present application provides a text detection method, the method comprising:

[0037] Acquire the image to be detected;

[0038] Inputting the image to be detected into a pre-trained text detection model to obtain a binary image of the image to be detected output by the text detection model, wherein the text detection model is trained by any one of the methods described in the first aspect;

[0039] A connected area in the binary image is determined to obtain a text prediction box where the text in the image to be detected is located, and the text prediction box is expanded to obtain a region where the text in the image to be detected is located.

[0040] Optionally, the method further includes:

[0041] According to a second preset search target, searching for four target points corresponding to the region where the text is located among the edge points of the region where the text is located, wherein the second preset search target is that the quadrilateral formed by the searched four target points has the largest overlap area with the region where the text is located;

[0042] The quadrilateral formed by the four target points is determined as the area where the text in the image to be detected is located.

[0043] Optionally, the step of searching, according to the second preset search target, four target points corresponding to the region where the text is located among the edge points of the region where the text is located comprises:

[0044] Select four candidate points from the edge points of the region where the text is located;

[0045] Calculating a score of a quadrilateral formed by the four candidate points, wherein the score is determined based on the area and / or side length of the quadrilateral;

[0046] Traversing each candidate point in turn, when traversing to each candidate point, updating the target point in the search range corresponding to the candidate point to the candidate point, wherein the target point is: an edge point of a quadrilateral formed by the other three candidate points except the candidate point, which has the highest score and is higher than the score, and the search range of the candidate point includes each edge point between the candidate point before the candidate point and the candidate point after the candidate point;

[0047] After the traversal is completed, the current four candidate points are used as four target points corresponding to the area where the text is located.

[0048] Optionally, the step of calculating the score of the quadrilateral formed by the four candidate points includes:

[0049] The score is calculated according to the following formula:

[0050] ;

[0051] in, For the score, is the area of ​​the quadrilateral formed by the four candidate points, is the perimeter of the quadrilateral formed by the four candidate points, is the total length of the diagonals of the quadrilateral formed by the four candidate points.

[0052] Optionally, the method further includes:

[0053] The longer midline of the two midlines of the text region is used as the target midline of the rotated rectangle to be constructed, and a rotated rectangle with the same area as the text region is constructed as the text region in the image to be detected.

[0054] Optionally, the step of taking the longer midline of the two midlines of the region where the text is located as the target midline of the rotated rectangle to be constructed and constructing a rotated rectangle having the same area as the area of ​​the region where the text is located comprises:

[0055] The two end points of the longer midline of the two midlines of the region where the text is located are respectively used as the midpoints of the two short sides of the rotated rectangle to be constructed;

[0056] The ratio of the area of ​​the region where the text is located to the length of the longer midline of the two midlines of the region where the text is located is used as the length of the two short sides of the rotated rectangle to be constructed;

[0057] The longer midline of the two midlines of the region where the text is located is used as the target midline of the rotated rectangle to be constructed, and four vertices of the rotated rectangle are determined according to the length of the short side and the midpoints of the two short sides, so that the two short sides of the rotated rectangle are perpendicular to the target midline;

[0058] The quadrilateral formed by the four vertices is used as the rotated rectangle corresponding to the area where the text is located.

[0059] Optionally, the step of taking the longer midline of the two midlines of the region where the text is located as the target midline of the rotated rectangle to be constructed, and determining four vertices of the rotated rectangle according to the length of the short side and the midpoints of the two short sides includes:

[0060] Using a direction vector perpendicular to a centerline vector of the target centerline as a direction vector of the short side, wherein the direction of the centerline vector is one of the extension directions of the target centerline;

[0061] Based on the direction vector and the length of the short side, a target vector is obtained, wherein the direction of the target vector is the same as the direction vector, and the length of the target vector is half of the length of the short side;

[0062] Calculate the sum of the midpoint of each short side and the target vector respectively to obtain two vertices of the rotated rectangle;

[0063] The difference between the midpoint of each short side and the target vector is calculated respectively to obtain the other two vertices of the rotated rectangle.

[0064] In a third aspect, an embodiment of the present application provides a text detection model training device, the device comprising:

[0065] A first search module is used to search for the corresponding indentation distance of the text region within the indentation distance range corresponding to the text region according to a first preset search target for each text region in each training image, wherein the first preset search target is the indentation distance obtained by searching so that the area of ​​the expanded region is the same as the area of ​​the text region, and the expanded region is the region obtained by shrinking the text region according to the indentation distance and then expanding it;

[0066] A shrinking module, used to shrink each text area in each training image based on the shrinking distance corresponding to each text area, so as to obtain a shrinking image of each training image;

[0067] An expansion module is used to expand each text area in each training image to obtain an expansion area corresponding to each text area;

[0068] A label generation module, used to generate a segmentation map label, a threshold map label and a binary map label corresponding to each training image based on the inner image of each training image and the outer expansion area corresponding to each text area in the training image;

[0069] A training module is used to input each training image into a text detection model to be trained, and to train the text detection model based on the differences between the predicted probability map, the predicted threshold map, and the predicted binary map output by the text detection model and the segmentation map label, the threshold map label, and the binary map label, so as to obtain a trained text detection model.

[0070] In a fourth aspect, an embodiment of the present application provides a text detection model, the model comprising:

[0071] An image acquisition module, used for acquiring an image to be detected;

[0072] A binary image acquisition module, used for inputting the image to be detected into a pre-trained text detection model, and acquiring a binary image of the image to be detected output by the text detection model, wherein the text detection model is trained by the method described in any one of the first aspects;

[0073] The region determination module is used to determine the connected region in the binary image, obtain the text prediction box where the text in the image to be detected is located, and expand the text prediction box to obtain the region where the text in the image to be detected is located.

[0074] In a fifth aspect, an embodiment of the present application provides an electronic device, including:

[0075] Memory, used to store computer programs;

[0076] The processor is used to implement any method described in the first aspect or the second aspect when executing the program stored in the memory.

[0077] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in either the first aspect or the second aspect is implemented.

[0078] In a seventh aspect, an embodiment of the present application further provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute any of the methods described in the first or second aspect above.

[0079] Beneficial effects of the embodiments of the present application:

[0080] In the text detection model training method provided by the embodiment of the present application, the electronic device can search for the indentation distance corresponding to each text area in each training image within the indentation distance range corresponding to the text area according to a first preset search target, wherein the first preset search target is the indentation distance obtained by searching so that the area of ​​the expanded area is the same as the area of ​​the text area, and the expanded area is the area obtained by shrinking the text area according to the indentation distance and then expanding it; based on the indentation distance corresponding to each text area in each training image, shrink each text area to obtain a shrinkage image of each training image; expand each text area in each training image to obtain an expanded area corresponding to each text area; based on the shrinkage image of each training image and the expanded area corresponding to each text area in the training image, generate a segmentation map label, a threshold map label and a binary map label corresponding to the training image; input each training image into the text detection model to be trained, and train the text detection model based on the differences between the predicted probability map, the predicted threshold map and the predicted binary map output by the text detection model and the segmentation map label, the threshold map label and the binary map label, respectively, to obtain a trained text detection model.

[0081] Since the inward contraction distance corresponding to each text area is searched within the inward contraction distance range corresponding to the text area, the area of ​​the expanded area obtained after the text area is first contracted and then expanded is the same as the area of ​​the text area. In this way, compared with the method of contracting and expanding according to fixed inward contraction experience parameters, for various types of text areas, the inward contraction distance corresponding to the text area is used to contract and expand the text area, and the consistency of the area before and after contraction and expansion can be maintained. Thus, both adhesion between text areas and loss of text information can be avoided, and the accuracy of text detection results is improved. And by using the inward contraction area of ​​each text area obtained according to the inward contraction distance corresponding to each text area, the segmentation map label, threshold map label and binary map label corresponding to the training image are calculated, which can improve the accuracy of the training label, and then use each training image and the label corresponding to each training image to train the text detection model, which can improve the text positioning accuracy of the trained text detection model. Of course, it is not necessary to achieve all the advantages described above at the same time to implement any product or method of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0083] Figure 1 A flowchart of a text detection model training method provided in an embodiment of the present application;

[0084] Figure 2 A flowchart of a specific implementation method of step S101 is shown;

[0085] FIG3 (a) is a schematic diagram of the area of ​​the expanded area and the area of ​​the text area corresponding to the candidate indentation distance;

[0086] FIG3 (b) is another schematic diagram of the area of ​​the expanded region and the area of ​​the text region corresponding to the alternative indentation distance;

[0087] FIG4 (a) is a schematic diagram of an expanded area corresponding to a text area obtained by shrinking and expanding according to an inward shrinking distance determined by a binary search method;

[0088] FIG4( b ) is a schematic diagram of an expanded area corresponding to a text area obtained by shrinking and expanding the text area according to a shrinking distance calculated according to an empirical shrinking parameter in the prior art;

[0089] Figure 5 A flowchart of a specific implementation method of step S105 is shown;

[0090] Figure 6 A flowchart of a text detection method provided in an embodiment of the present application;

[0091] Figure 7 A schematic diagram of the minimum enclosing rectangle of the area where the text is located;

[0092] Figure 8 A schematic diagram of a process for constructing a rotated rectangle provided in an embodiment of the present application;

[0093] Fig. 9 A schematic diagram of a text detection model training device provided in an embodiment of the present application;

[0094] Fig.10 A schematic diagram of a text detection device provided in an embodiment of the present application;

[0095] Fig.11 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0096] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field based on the present application belong to the scope of protection of the present application.

[0097] In order to improve the accuracy of text positioning of a text detection model, the present application provides a text detection model training method, a text detection method, an apparatus, an electronic device, a computer-readable storage medium, and a computer program product. The following first introduces a text detection model training method provided in an embodiment of the present application.

[0098] like Figure 1 As shown, a text detection model training method provided in an embodiment of the present application includes:

[0099] S101: for each text region in each training image, searching for an indentation distance corresponding to the text region within an indentation distance range corresponding to the text region according to a first preset search target.

[0100] The first preset search target is the inward contraction distance obtained by the search so that the area of ​​the outward expansion region is the same as the area of ​​the text region, and the outward expansion region is the region obtained by contracting the text region according to the inward contraction distance and then expanding it.

[0101] S102: based on the shrinkage distance corresponding to each text area in each training image, shrinkage is performed on each text area to obtain a shrinkage image of each training image.

[0102] S103: Expand each text region in each training image to obtain an expanded region corresponding to each text region.

[0103] S104: Based on the thumbnail image of each training image and the expanded area corresponding to each text area in the training image, a segmentation image label, a threshold image label and a binary image label corresponding to the training image are generated.

[0104] S105: Input each training image into the text detection model to be trained, and train the text detection model based on the differences between the predicted probability map, the predicted threshold map, and the predicted binary map output by the text detection model and the segmentation map label, the threshold map label, and the binary map label, to obtain a trained text detection model.

[0105] In the text detection model training method provided by the embodiment of the present application, the electronic device can search for the indentation distance corresponding to each text area in each training image within the indentation distance range corresponding to the text area according to a first preset search target, wherein the first preset search target is the indentation distance obtained by searching so that the area of ​​the expanded area is the same as the area of ​​the text area, and the expanded area is the area obtained by shrinking the text area according to the indentation distance and then expanding it; based on the indentation distance corresponding to each text area in each training image, shrink each text area to obtain a shrinkage image of each training image; expand each text area in each training image to obtain an expanded area corresponding to each text area; based on the shrinkage image of each training image and the expanded area corresponding to each text area in the training image, generate a segmentation map label, a threshold map label and a binary map label corresponding to the training image; input each training image into the text detection model to be trained, and train the text detection model based on the differences between the predicted probability map, the predicted threshold map and the predicted binary map output by the text detection model and the segmentation map label, the threshold map label and the binary map label, respectively, to obtain a trained text detection model.

[0106] Since the inward contraction distance corresponding to each text region is searched within the inward contraction distance range corresponding to the text region, the area of ​​the expanded area obtained after the text region is first contracted and then expanded is the same as the area of ​​the text region. In this way, compared with the method of contracting and expanding according to fixed inward contraction experience parameters, for various types of text regions, the inward contraction distance corresponding to the text region is used to contract and expand the text region, and the consistency of the area before and after contraction and expansion can be maintained. Therefore, both the adhesion between text regions and the loss of text information can be avoided, and the accuracy of text detection results is improved. In addition, the segmentation map label, threshold map label and binary map label corresponding to the training image are calculated by using the inward contraction area of ​​each text region obtained according to the inward contraction distance corresponding to each text region, so as to improve the accuracy of the training label, and then use each training image and the label corresponding to each training image to train the text detection model, so as to improve the text positioning accuracy of the trained text detection model.

[0107] In order to improve the efficiency of text detection, a text detection model for text detection can be trained, and the trained text detection model can be used for text detection later. When training the text detection model, the electronic device can first obtain a training image set for training the text detection model, and then train the text detection model using each training image in the training image set and the training label corresponding to each training image.

[0108] As an implementation manner, a training image set may be obtained, and each training image in the training image set may be annotated with a text region where text is located, wherein the text region may be any polygon, such as a rotated rectangle.

[0109] In the above step S101, for each text area in each training image, the electronic device can search for the corresponding indentation distance of the text area within the indentation distance range corresponding to the text area according to the first preset search target. The first preset search target is the indentation distance obtained by searching so that the area of ​​the expanded area is the same as the area of ​​the text area, and the expanded area is the area obtained by shrinking the text area according to the indentation distance and then expanding it.

[0110] For each training image, the training image may include multiple texts, and the spacing between adjacent texts may be small or even may be contiguous. When performing text detection in the presence of contiguous texts, the contiguous texts may be mistakenly identified as one text, thereby affecting the accuracy of the text detection result.

[0111] In order to avoid the adhesion between texts affecting the accuracy of text detection results, when performing text detection based on the text detection method based on semantic segmentation, there is usually a process of first shrinking and then expanding. That is, each text area in the image can be shrunk first to avoid adhesion between texts. Since the shape and area of ​​each text area can be different, before shrinking each text area, the electronic device can first determine the shrinking distance corresponding to each text area.

[0112] Generally, when shrinking and expanding a text area, it is hoped that the area and shape of the text area before and after shrinking and expanding should be as close as possible, that is, it is hoped that the area of ​​the expanded area obtained by shrinking the text area and then expanding it is the same as the area of ​​the text area before shrinking.

[0113] Based on this, a first preset search target can be set, which is: the retraction distance obtained by the search makes the area of ​​the expanded area the same as the area of ​​the text area, and the expanded area is the area obtained by shrinking the text area according to the retraction distance and then expanding it.

[0114] That is to say, the search goal to be achieved by performing the shrinking distance search is: the area of ​​the expanded area obtained by shrinking the text area according to the searched shrinking distance and then expanding it is the same as the area of ​​the text area.

[0115] In this way, the electronic device can search for the corresponding inward distance of each text area within the inward distance range corresponding to the text area according to the set first preset search target. For each text area, the text area is inwardly contracted and then expanded using the inward distance corresponding to the text area obtained by searching according to the above first search target, and the area of ​​the expanded area obtained by inwardly contracting and expanding the text area is the same as the area of ​​the text area. In some cases, there may be errors in the process of inward and outward contraction, resulting in the area of ​​the expanded area obtained by inward and outward contraction according to the inward distance obtained by searching according to the first preset search target being not strictly equal to the area of ​​the text area.

[0116] For each text area, the indentation distance range corresponding to the text area may have a preset value as a left boundary and a target value as a right boundary.

[0117] The above preset value is less than the target value. The above target value can be determined based on the perimeter of the text area, for example, the target value can be 1 / 2, 1 / 3, etc. of the perimeter of the text area, which is not specifically limited here. The above preset value can be a preset fixed value, for example, 0, 1, etc.; it can also be determined based on the perimeter of the text area, for example, the preset value can be 1 / 20 of the perimeter of the text area, etc., which is not specifically limited here.

[0118] As an implementation manner, for each text region, the electronic device may perform a binary search within the indentation distance range corresponding to the text region according to a first preset search target to search for the indentation distance corresponding to the text region.

[0119] As an implementation mode, for each text area, the electronic device may search for the indentation distance corresponding to the text area based on a preset step length within the indentation distance range corresponding to the text area according to the first preset search target. The preset step length may be set according to the indentation distance range to be searched, and is not specifically limited here. For example, when the indentation distance range to be searched is [2, 10], the preset step length may be 1; when the indentation distance range to be searched is [10, 100], the preset step length may be 5.

[0120] For the sake of clarity, the implementation method of the electronic device searching for the indentation distance corresponding to each text area will be specifically introduced below.

[0121] Furthermore, in the above step S102, the electronic device may shrink each text area based on the shrinkage distance corresponding to each text area in each training image to obtain a shrinkage image of each training image.

[0122] For each training image, after searching for the indentation distances corresponding to each text area in the training image, the electronic device can indent each text area based on the indentation distances corresponding to each text area in the training image to obtain a shrinkage image of the training image.

[0123] Specifically, for each text area, the electronic device shrinks the text area according to the shrinking distance of the text area. Shrinking the text area means shrinking the outer boundary box of the text area by the shrinking distance inward of the text area to obtain a new outer boundary box of the text area, and the area surrounded by the new outer boundary box is the shrinking area corresponding to the text area. After the shrinking of each text area in the training image is completed, a shrinkage image of the training image can be obtained. In the case where the training image includes multiple text areas, the shrinkage image of the training image also includes the shrinking areas of multiple text areas.

[0124] As an implementation manner, for each text area, the electronic device may shrink the text area according to the shrinking distance of the text area by using a vatti_clipping algorithm to obtain a shrinking area corresponding to the text area.

[0125] Furthermore, in the above step S103, the electronic device may expand each text region in each training image to obtain an expanded region corresponding to each text region.

[0126] For each text region in the training image, the electronic device may expand the text region to obtain an expanded region corresponding to each text region.

[0127] Specifically, for each text region, before expanding the text region, the electronic device may first calculate the expansion distance corresponding to the text region according to the area and perimeter of the text region.

[0128] For each text area, after determining the expansion distance corresponding to the text area, the electronic device can expand the text area according to the expansion distance corresponding to the text area to obtain the expansion area corresponding to the text area. Specifically, the electronic device can expand the outer boundary box of the text area outward by the above expansion distance to obtain a new outer boundary box of the text area, and the area surrounded by the new outer boundary box is the expansion area corresponding to the text area. After completing the expansion of each text area in the training image, the expansion area of ​​each text area in the training image is obtained.

[0129] In the above step S104, the electronic device may generate a segmentation map label, a threshold map label and a binary map label corresponding to the training image based on the thumbnail of each training image and the expanded area corresponding to each text area in the training image.

[0130] For each training image, the electronic device obtains a thumbnail of the training image and an expanded area corresponding to each text area in the training image, and generates a segmentation map label, a threshold map label and a binary map label corresponding to the training image.

[0131] Among them, for each training image, the electronic device can generate a segmentation map label corresponding to the training image based on the retracted area corresponding to each text area in the training image. The segmentation map label corresponding to the training image can be understood as a calibration image or pixel matrix corresponding to the training image, which is marked with the retracted area boundary positions of each text area to be identified in the training image; the electronic device can obtain the threshold map corresponding to the training image based on the retracted area and the expanded area corresponding to each text area in the training image, and generate a threshold map label corresponding to the training image, that is, the electronic device can generate a progressive label of the threshold map based on the area from the boundary of each text area in the training image to the boundary of the retracted area and the boundary of the expanded area; and the electronic device can generate a binary image corresponding to the training image based on the retracted image and the threshold map corresponding to the training image, and generate a binary image label corresponding to the training image.

[0132] In the above step S105, the electronic device can input each training image into the text detection model to be trained, and train the text detection model based on the differences between the predicted probability map, predicted threshold map and predicted binary map output by the text detection model and the segmentation map label, threshold map label and binary map label, so as to obtain a trained text detection model.

[0133] For each training image, the training image and the segmentation map label, threshold map label and binary map label corresponding to the training image are used together to train the text detection model to be trained.

[0134] Among them, the text detection model to be trained may include a feature extraction module, a probability map prediction module, a threshold map prediction module and a binarization module.

[0135] As an implementation method, the feature extraction module of the text detection model to be trained can be based on the ResNet-18 or ResNet-50 deep convolutional neural network, and feature maps of different sizes can be fused in the form of a feature pyramid to extract text features corresponding to the training image from the input training image; the composition structure of the probability map prediction module and the threshold map prediction module of the text detection model to be trained can be consistent, and the probability map prediction module and the threshold map prediction module are respectively used to generate a predicted probability map and a predicted threshold map corresponding to the training image based on the text features corresponding to the training image; the binary map prediction module of the text detection model to be trained is used to generate a predicted binary map of the training image based on the predicted probability map corresponding to the training image.

[0136] When training a text detection model using each training image and the label corresponding to each training image, the electronic device can input each training image into the text detection model to be trained. In this way, the feature extraction module of the text detection model can extract the text features corresponding to the training image, and obtain the text feature map corresponding to the training image. Then, the text feature map is input into the probability map prediction module and the threshold map prediction module respectively to obtain the predicted probability map and predicted threshold map corresponding to the training image, and the predicted probability map and predicted threshold map are input into the binarization module to obtain the predicted binary map corresponding to the training image. The electronic device can obtain the predicted probability map, predicted threshold map and predicted binary map corresponding to each training image output by the text detection model.

[0137] Furthermore, for each training image, the electronic device can determine whether the text detection model has converged based on the difference between the predicted probability map corresponding to the training image output by the text detection model and the segmentation map label, the difference between the predicted threshold map corresponding to the training image output by the text detection model and the threshold map label, and the difference between the predicted binary map corresponding to the training image output by the text detection model and the binary map label. In addition, when it is determined that the text detection model has not converged, the electronic device can adjust the parameters of the text detection model to be trained based on the difference between the predicted probability map and the segmentation map label, the difference between the predicted threshold map and the threshold map label, and the difference between the predicted binary map and the binary map label, until the text detection model is trained, thereby obtaining a trained text detection model.

[0138] Specifically, the electronic device can calculate the loss value corresponding to the training image based on the predicted probability map, predicted threshold map, predicted binary map, segmentation map label, threshold map label, binary map label and preset loss function output by the text detection model. Determine whether the text detection model converges based on the loss value corresponding to the training image. If the model has not converged, the electronic device can adjust the parameters of the text detection model based on the loss value corresponding to the training image. For example, the gradient descent method can be used to adjust the parameters of the text detection model using the loss value. Until the text detection model converges, a trained text detection model is obtained.

[0139] The preset loss function may include at least one of a cross entropy loss function, a DICE loss function, and a pre-constructed intersection-over-union loss function. Moreover, the pre-constructed intersection-over-union loss function is only used to calculate the difference between the text area shape in the training image predicted by the text detection model and the text area shape represented by the segmentation map label corresponding to the training image.

[0140] As an implementation mode, the electronic device may stop training when the number of iterations of each training image reaches a preset number of times, and obtain a trained text detection model. The preset number of times may be set according to actual needs, for example, 20,000 times, 50,000 times, etc., and is not specifically limited here.

[0141] In addition, the trained text detection model can be called DB-Align (differentiable binary segmentation model).

[0142] Since the inward contraction distance corresponding to each text region is searched within the inward contraction distance range corresponding to the text region, the area of ​​the expanded region obtained after the text region is first contracted and then expanded is the same as the area of ​​the text region. In this way, compared with the method of contracting and expanding according to fixed inward contraction experience parameters, for various types of text regions, the inward contraction distance corresponding to the text region is used to contract and expand the text region, and the consistency of the area before and after contraction and expansion can be maintained. Therefore, both adhesion between text regions and loss of text information can be avoided, and the accuracy of text detection results is improved. In addition, the inward contraction area of ​​each text region obtained by the inward contraction distance corresponding to each text region is used to calculate the segmentation map label, threshold map label and binary map label corresponding to the training image, which can improve the accuracy of the training label, and then the text detection model is trained by using each training image and the label corresponding to each training image, which can improve the text positioning accuracy of the trained text detection model.

[0143] As an implementation method of the present application, Figure 2 As shown, the above step S101, i.e., the step of searching for the indentation distance corresponding to the text area within the indentation distance range corresponding to the text area according to the first preset search target for each text area in each training image, may include the following steps S201-S203:

[0144] S201: For each text area in each training image, determine the indentation distance range by taking 0 as the left boundary of the indentation distance range and taking the target value as the right boundary of the indentation distance range.

[0145] The target value is determined based on the perimeter of the text area.

[0146] For each text region in each training image, the electronic device may determine the retracted distance range of the text region by taking 0 as the left boundary of the retracted distance range and taking the target value as the right boundary of the retracted distance range.

[0147] As mentioned above, the target value may be determined based on the perimeter of the text area. For example, the target value may be 1 / 2, 1 / 3, etc. of the perimeter of the text area, which is not specifically limited here.

[0148] For example, the training image includes text area A and text area B, wherein the perimeter of text area A is 16, and the perimeter of text area B is 20. The indentation distance range of text area A can be determined as [0, 8] with 0 as the left boundary of the indentation distance range of text area A and 8 as the right boundary of the indentation distance range of text area A; the indentation distance range of text area B can be determined as [0, 10] with 0 as the left boundary of the indentation distance range of text area B and 10 as the right boundary of the indentation distance range of text area B.

[0149] After determining the retracted distance range of each text region, the electronic device may search for the retracted distance corresponding to the text region according to the first preset search target within the retracted distance range of each text region.

[0150] S202: When the difference between the left boundary and the right boundary of the inward-retracting distance range is greater than a preset accuracy threshold, determine an alternative inward-retracting distance within the inward-retracting distance range.

[0151] For each text region, a preset accuracy threshold can be pre-set, and the difference between the left boundary and the right boundary of the indentation distance range of the text region is not greater than the preset accuracy threshold as the search cutoff condition corresponding to the text region. The preset accuracy threshold can be set according to actual needs, for example, 0.5, 1, etc., and is not specifically limited here. In addition, when a training image includes multiple text regions, the preset accuracy thresholds corresponding to each text region can be the same or different, and are not specifically limited here.

[0152] That is, for each text area, the electronic device may first determine whether the difference between the left boundary and the right boundary of the indentation distance range of the text area is greater than a preset accuracy threshold. When the difference between the left boundary and the right boundary is greater than the preset accuracy threshold, the electronic device may search for the indentation distance corresponding to the text area within the indentation distance range of the text area, and may continuously update the indentation distance range of the text area during the search process until the difference between the left boundary and the right boundary of the indentation distance range of the text area is no greater than the above-mentioned preset accuracy threshold, and then the indentation distance corresponding to the text area may be determined based on the current indentation distance range of the text area.

[0153] Specifically, for each text region, when the difference between the left boundary and the right boundary of the indentation distance range of the text region is greater than a preset accuracy threshold, the electronic device may first determine an alternative indentation distance within the indentation distance range of the text region.

[0154] As an implementation manner, for each text region, the electronic device may perform a binary search within the indentation distance range of each text region, and may determine the middle value of the left boundary and the right boundary as the candidate indentation distance.

[0155] As an implementation mode, for each text region, when searching the indentation distance range of the text region according to a preset step length, the electronic device may increase the left boundary of the indentation distance range by a value obtained by the preset step length, or reduce the right boundary of the indentation distance range by a value obtained by the preset step length, and determine it as an alternative indentation distance, etc. Among them, the above-mentioned preset step length can be set according to actual needs, for example, 0.5, 2, etc., which is not specifically limited here. In addition, when the training image includes multiple text regions, when the electronic device searches each text region separately, the preset step lengths corresponding to each text region may be the same or different, which is not specifically limited here.

[0156] As an implementation mode, for each text area, when the difference between the left boundary and the right boundary of the indentation distance range of the text area is not greater than a preset accuracy threshold, the electronic device can determine the indentation distance corresponding to the text area based on the indentation distance range of the text area.

[0157] S203: Update the retraction distance range based on the alternative retraction distance, and return to execute the step of determining the alternative retraction distance within the retraction distance range when the difference between the left boundary and the right boundary of the retraction distance range is greater than the preset accuracy threshold, until the difference between the left boundary and the right boundary of the current retraction distance range is not greater than the preset accuracy threshold, and determine the retraction distance corresponding to the text area based on the current retraction distance range.

[0158] For each text region, after determining a candidate retraction distance within the retraction distance range of the text region, the electronic device may update the retraction distance range of the text region based on the candidate retraction distance.

[0159] As an implementation manner, the electronic device may use a vatti_clipping algorithm to indent the text area based on the candidate indentation distance, and when the candidate indentation distance fails to indent, use the candidate indentation distance to update the right boundary of the indentation distance range of the text area.

[0160] As an implementation mode, the electronic device may shrink the text area based on the candidate shrink distance, and then expand the shrink area obtained by shrinking to obtain the expanded area corresponding to the candidate shrink distance, and calculate the difference between the area of ​​the expanded area corresponding to the candidate shrink distance and the area of ​​the text area. Furthermore, the electronic device may determine whether to update the left boundary or the right boundary of the shrink distance range of the text area using the candidate shrink distance based on the difference, and update the determined boundary using the candidate shrink distance to obtain a new shrink distance range of the text area.

[0161] For each text area, after updating the indentation distance range of the text area based on the candidate indentation distance, the electronic device may return to execute step S202, that is, determine the candidate indentation distance within the indentation distance range when the difference between the left boundary and the right boundary is greater than a preset accuracy threshold.

[0162] That is, for each text area, after the electronic device updates the indentation distance range of the text area based on the alternative indentation distance, the electronic device can determine whether the difference between the left boundary and the right boundary of the current indentation distance range of the text area is greater than the preset accuracy threshold, and if the difference between the left boundary and the right boundary is greater than the preset accuracy threshold, continue to determine the alternative indentation distance within the current indentation distance range of the text area. Then continue to update the indentation distance range of the text area based on the alternative indentation distance, and again determine whether the difference between the left boundary and the right boundary of the current indentation distance range of the text area is greater than the preset accuracy threshold, and then if the difference between the left boundary and the right boundary is greater than the preset accuracy threshold, continue to determine the alternative indentation distance within the current indentation distance range of the text area, and continue to update the indentation distance range of the text area based on the alternative indentation distance, until the difference between the left boundary and the right boundary of the current indentation distance range of the text area is not greater than the preset accuracy threshold, the electronic device can stop searching, and determine the indentation distance corresponding to the text area based on the current indentation distance range.

[0163] In this embodiment, for each text area in each training image, the retraction distance range can be determined with 0 as the left boundary of the retraction distance range and the target value as the right boundary of the retraction distance range, wherein the target value is determined based on the perimeter of the text area; when the difference between the left boundary and the right boundary of the retraction distance range is greater than a preset accuracy threshold, the electronic device can determine an alternative retraction distance within the retraction distance range; then, the retraction distance range is updated based on the alternative retraction distance, and the step of determining the alternative retraction distance within the retraction distance range when the difference between the left boundary and the right boundary of the retraction distance range is greater than the preset accuracy threshold is returned to execute, until the difference between the left boundary and the right boundary of the current retraction distance range is not greater than the preset accuracy threshold, and the retraction distance corresponding to the text area is determined based on the current retraction distance range. In this way, by continuously updating the indentation distance range of the text area, the indentation distance range finally satisfies the cutoff condition that the difference between the left and right boundaries is not greater than the preset accuracy threshold, and based on the indentation distance range that satisfies the cutoff condition, the indentation distance corresponding to the text area is determined. In this way, the determined indentation distance of the text area satisfies the search goal that the area of ​​the text area before and after indentation and expansion is the same. Therefore, using the indentation distance of the text area for indentation can avoid adhesion between the text areas, thereby improving the accuracy of the text detection result. Using the indentation distance corresponding to each text area to shrink the text area to obtain the indentation image of the training image and the label obtained by training the expanded area of ​​each text area, the text detection model is trained, which can improve the text positioning accuracy of the text detection model.

[0164] As an implementation of an embodiment of the present application, in the above step S203, the step of updating the retracted distance range based on the candidate retracted distance may include:

[0165] In the case where the alternative inward contraction distance fails to be used for inward contraction, or the area of ​​the alternative outward expansion region obtained based on the alternative inward contraction distance is smaller than the area of ​​the text region, the alternative inward contraction distance is used as the right boundary of the inward contraction distance range to obtain an updated inward contraction distance range, wherein the area of ​​the alternative outward expansion region is the area of ​​the outward expansion region obtained based on the alternative inward contraction distance;

[0166] In the case that the area of ​​the candidate outward expansion region is larger than the area of ​​the text region, the candidate inward contraction distance is used as the left boundary of the inward contraction distance range to obtain an updated inward contraction distance range.

[0167] For each text region, after determining a candidate retraction distance within the retraction distance range of the text region, the electronic device may update the retraction distance range based on the candidate retraction distance.

[0168] Specifically, the electronic device may utilize the vatti_clipping algorithm to indent the text area based on the alternative indentation distance.

[0169] Since shrinking the text area means shrinking each bounding box of the text area separately, if the alternative shrinking distance is too large, the alternative shrinking distance may be greater than the length of any bounding box of the text area. In this case, the vatti_clipping algorithm cannot be used to shrink the text area, and the shrinking of the alternative shrinking distance fails. For example, when the text box is a positive direction area with a side length of 3, the alternative shrinking distance is 4. Since the alternative shrinking distance is greater than the side length of the text shrinking distance, the alternative shrinking distance cannot be used to shrink the text area, and the shrinking of the alternative shrinking distance fails.

[0170] Based on this, when the indentation using the alternative indentation distance fails, it can be determined that the alternative indentation distance is too large, and the values ​​between the right boundary and the alternative indentation distance are all too large, so that the indentation distance corresponding to the text area of ​​the above-mentioned first search target is not between the right boundary and the alternative indentation distance. In order to avoid searching an invalid range and improve search efficiency, the electronic device can use the alternative indentation distance as the right boundary of the indentation distance range to obtain an updated indentation distance range. The updated indentation distance range uses 0 as the left boundary and the alternative indentation distance as the right boundary, which is significantly smaller than the indentation distance range before the update.

[0171] When the electronic device can use the vatti_clipping algorithm to shrink the text area based on the alternative shrinking distance to obtain the shrunken area of ​​the text area, that is, when the text area can be shrunk using the alternative shrinking distance, the electronic device can continue to expand the shrunken area to obtain the expanded area of ​​the text area, and use the area of ​​the expanded area obtained based on the alternative shrinking distance as the area of ​​the alternative expanded area.

[0172] When the alternative retraction distance is small, the area and perimeter of the retracted area are larger when the text area is retracted using the alternative retraction distance. After the retracted area is expanded using the expansion distance determined based on the area and perimeter of the retracted area and the fixed expansion empirical parameter, the area of ​​the expanded area is larger than the area of ​​the text area, that is, the area of ​​the alternative expansion area corresponding to the alternative retraction distance is larger than the area of ​​the text area.

[0173] The expansion distance can be calculated according to the following formula:

[0174] ;

[0175] in, is the outward expansion distance; is the area of ​​the indented region; To expand the empirical parameters, is a constant, for example: ; is the perimeter of the indented area.

[0176] Exemplarily, as shown in FIG3 (a), 310 is a text area, and the electronic device determines an alternative retraction distance D1 within the retraction distance range of the text area, and retracts the text area 310 according to the alternative retraction distance D1 to obtain a retracted area 320. An expansion distance D1' is determined based on the area, perimeter, and fixed expansion empirical parameters of the retracted area 320, and the retracted area is expanded according to the expansion distance D1' to obtain an expansion area 330, and the area of ​​the obtained expansion area 330 is larger than the area of ​​the text area 310.

[0177] Based on this, when the area of ​​the candidate outer expansion area is larger than the area of ​​the text area, it can be determined that the candidate inner contraction distance is too small, and the values ​​between the left boundary and the candidate inner contraction distance are all too small, so that the inner contraction distance corresponding to the text area of ​​the first search target is not between the left boundary and the candidate inner contraction distance. In order to avoid searching an invalid range and improve search efficiency, the electronic device can use the candidate inner contraction distance as the left boundary of the inner contraction distance range to obtain an updated inner contraction distance range.

[0178] When the alternative retraction distance is large, the text area is retracted using the alternative retraction distance, and the area and perimeter of the retracted area are both small. After the retracted area is expanded using the expansion distance determined based on the area and perimeter of the retracted area and the fixed expansion empirical parameter, the area of ​​the expanded area is smaller than the area of ​​the text area, that is, the area of ​​the alternative expansion area corresponding to the alternative retraction distance is smaller than the area of ​​the text area.

[0179] Exemplarily, as shown in FIG3( b ), 310 is a text area, and the electronic device determines an alternative retraction distance D2 within the retraction distance range of the text area, and retracts the text area 310 according to the alternative retraction distance D2 to obtain a retracted area 340. An expansion distance D2′ is determined based on the area, perimeter, and fixed expansion empirical parameters of the retracted area 340, and the retracted area is expanded according to the expansion distance D2′ to obtain an expansion area 350, and the area of ​​the obtained expansion area 350 is smaller than the area of ​​the text area 310.

[0180] Based on this, when the area of ​​the alternative expanded area is smaller than the area of ​​the text area, the alternative retracted distance is too large, and then the values ​​between the right boundary and the alternative retracted distance are too large, so that the retracted distance corresponding to the text area of ​​the first search target is not between the right boundary and the alternative retracted distance. In order to avoid searching an invalid range and improve search efficiency, the electronic device can use the alternative retracted distance as the right boundary of the retracted distance range to obtain an updated retracted distance range.

[0181] Exemplarily, when the electronic device performs a binary search on the inward distance range of the text area, if the difference between the left boundary and the right boundary of the inward distance range is greater than a preset accuracy threshold, the electronic device can calculate the middle value between the left boundary and the right boundary of the inward distance range as an alternative inward distance. The electronic device can inward the text area according to the middle value, and if the inward movement fails, the middle value is updated as the right boundary of the inward distance range. If the area of ​​the alternative outward expansion area obtained based on the middle value is smaller than the area of ​​the text area, the middle value is used as the right boundary of the inward distance range; if the area of ​​the alternative outward expansion area is larger than the area of ​​the text area, the middle value is used as the left boundary of the inward distance range to obtain an updated inward distance range.

[0182] After using the middle value to update the indentation distance range, if the difference between the left and right boundaries of the indentation distance range is still greater than the preset accuracy threshold, continue to calculate the middle value of the left and right boundaries of the updated indentation distance range until the difference between the left and right boundaries of the current indentation distance range is no greater than the preset accuracy threshold, and determine the indentation area corresponding to the text area based on the current indentation distance range.

[0183] In this embodiment, when the alternative retraction distance fails to be used for retraction, or the area of ​​the alternative expansion area obtained based on the alternative retraction distance is smaller than the area of ​​the text area, the electronic device can return the alternative retraction distance as the right boundary of the retraction distance range, so that the electronic device can obtain an updated retraction distance range by updating the alternative retraction distance to the right boundary of the retraction distance range; and when the area of ​​the alternative expansion area is larger than the area of ​​the text area, the electronic device can return the alternative retraction distance as the left boundary of the retraction distance range, so that the electronic device can obtain an updated retraction distance range by updating the alternative retraction distance to the left boundary of the retraction distance range. By updating the retraction distance range using the alternative retraction distance, the electronic device can continuously narrow the retraction distance range that the electronic device needs to search, thereby improving the search efficiency.

[0184] As an implementation of the embodiment of the present application, in the above step S203, the step of determining the indentation distance corresponding to the text area based on the current indentation distance range may include:

[0185] If the indentation using the right boundary of the current indentation distance range fails, or the absolute value of the first area difference is greater than the absolute value of the second area difference, the left boundary of the current indentation distance range is returned as the indentation distance corresponding to the text area, wherein the first area difference is the difference between the area of ​​the expanded area obtained based on the right boundary and the area of ​​the text area, and the second area difference is the difference between the area of ​​the expanded area obtained based on the left boundary and the area of ​​the text area; if the absolute value of the first area difference is not greater than the absolute value of the second area difference, the right boundary of the current indentation distance range is returned as the indentation distance corresponding to the text area.

[0186] For each text area, after each update of the indentation distance range of the text area, the electronic device can determine whether the difference between the left boundary and the right boundary of the current indentation distance range is greater than a preset accuracy threshold. If the difference between the left boundary and the right boundary of the current indentation distance range is not greater than the preset accuracy threshold, the electronic device can determine the indentation distance corresponding to the text area based on the current indentation distance range.

[0187] As an implementation mode, the electronic device may utilize the vatti_clipping algorithm to indent the text area based on the left and right boundaries of the current indentation distance range. As previously mentioned, if the right boundary of the current indentation distance range fails to indent, it may be determined that the right boundary is too large, and the left boundary of the current indentation distance range may be returned as the indentation distance corresponding to the text area, that is, the electronic device may use the left boundary of the current indentation distance range as the indentation distance corresponding to the text area. When the text area is indented based on the right boundary of the current indentation distance range to obtain the indentation area corresponding to the right boundary, the indentation area corresponding to the right boundary is further expanded to obtain the expanded area. The difference between the area of ​​the expanded area obtained based on the right boundary and the area of ​​the text area is calculated to obtain a first area difference.

[0188] Furthermore, the text area can be retracted based on the left boundary of the current retracted distance range to obtain the retracted area corresponding to the left boundary, and the retracted area corresponding to the left boundary can be further expanded to obtain the expanded area. The difference between the area of ​​the expanded area obtained based on the left boundary and the area of ​​the text area is calculated to obtain a second area difference.

[0189] As an implementation method, each time the retraction distance range is updated based on an alternative retraction distance, the difference between the area of ​​the expanded area corresponding to the alternative retraction distance and the area of ​​the text area is updated to the area difference corresponding to the boundary updated by the alternative retraction distance. Moreover, in the event that the alternative retraction distance fails to retract, after the right boundary of the retraction distance range of the text area is updated using the alternative retraction distance, the failure of the right boundary of the updated retraction distance range to retract is recorded.

[0190] Since we hope that the area of ​​the expanded area obtained by shrinking the text area using the shrinking distance corresponding to the text area and then expanding it outwards is close to or even the same as the area of ​​the text area, the absolute value of the difference between the area of ​​the expanded area obtained based on the shrinking distance corresponding to the text area and the area of ​​the text area should be as small as possible or even 0.

[0191] Based on this, after obtaining the first area difference corresponding to the right boundary and the second area difference corresponding to the left boundary, when the absolute value of the first area difference is greater than the absolute value of the second area difference, the electronic device can return the left boundary of the current retraction distance range as the retraction distance corresponding to the text area, that is, the electronic device can determine that the left boundary of the current retraction distance range is the retraction distance corresponding to the text area; and when the absolute value of the first area difference is not greater than the absolute value of the second area difference, the electronic device can return the right boundary of the current retraction distance range as the retraction distance corresponding to the text area, that is, the electronic device can determine that the right boundary of the current retraction distance range is the retraction distance corresponding to the text area.

[0192] In this embodiment, when the difference between the left boundary and the right boundary of the current inward retraction distance range is not greater than the preset accuracy threshold, the electronic device can determine the inward retraction distance corresponding to the text area based on the current inward retraction distance range. Wherein, when the inward retraction using the right boundary of the current inward retraction distance range fails, or the absolute value of the first area difference between the area of ​​the outward expansion area obtained based on the right boundary and the area of ​​the text area is greater than the absolute value of the second area difference between the area of ​​the outward expansion area obtained based on the left boundary and the area of ​​the text area, the electronic device can return the left boundary of the current inward retraction distance range as the inward retraction distance corresponding to the text area; and when the absolute value of the first area difference is not greater than the absolute value of the second area difference, the electronic device can return the right boundary of the current inward retraction distance range as the inward retraction distance corresponding to the text area. In this way, when the current inward retraction distance range meets the search cutoff condition, the electronic device can use the boundary value of the left boundary and the right boundary of the current inward retraction distance range, the boundary value of the smaller difference between the area of ​​the outward expansion area and the area of ​​the text area corresponding to the text area as the inward retraction distance corresponding to the text area, so that the determined inward retraction distance is the inward retraction distance that meets the first preset search target. Using the shrinkage distance to shrink the text area can avoid adhesion between the text areas, thereby improving the accuracy of the text detection results. And using the shrinkage distance corresponding to each text area to shrink the text area to obtain the shrinkage image of the training image and the labels obtained by training the expanded area of ​​each text area, the text detection model can be trained to improve the text positioning accuracy of the text detection model.

[0193] Taking binary search as an example, a specific implementation method of the electronic device searching for the inward distance corresponding to the text area according to the inward distance range corresponding to the first preset search target in the text area is specifically described below.

[0194] The algorithm logic to be executed when the electronic device performs a binary search for the indentation distance corresponding to the text area within the indentation distance range corresponding to the text area according to the first preset search target is called the indentation amplitude binary search algorithm. The algorithm implementation process is as follows:

[0195] For GT (Ground Truth, real label), in this solution, GT is the text area. When searching for the indentation distance corresponding to the GT, the input is the polygon P of the text area, and the output is the indentation distance D corresponding to the GT. Pre-set the minimum indentation accuracy (that is, the preset accuracy threshold mentioned above) , external experience parameters .

[0196] Take 0 as the left boundary of the GT's indentation distance range , calculate the perimeter of the GT , and the area of ​​the GT The right boundary of the indentation distance range is half of the perimeter. .

[0197] Left edge of the indent distance range To shrink, shrink the area , and for the indented area Expand outward, where the expansion distance , and get the extended area , and calculate the difference between the area of ​​the expanded area and the area of ​​the text area .

[0198] Right edge of the indent distance range If the indentation fails based on the right boundary, then .

[0199] If you can indent based on the right border, you can get the indent area , and for the indented area Expand outward, where the expansion distance , and get the extended area , and calculate the difference between the area of ​​the expanded area and the area of ​​the text area .

[0200] The electronic device determines the left boundary of the retracted distance range With right border Is the difference greater than the minimum shrinkage precision? . At the left edge of the indent distance range With right border The difference is greater than the minimum shrinkage precision In the case of . Using the vatti_clipping algorithm, based on the median value Indent the text area to get the indented area . Then the expansion distance is calculated based on the shrinking area , and use the expansion distance to the shrinking area Expand outward to obtain the expanded area , then calculate the difference between the area of ​​the expanded area and the area of ​​the text area If the indentation using the intermediate value fails, The value of is updated to None.

[0201] exist or In the case of Updated to the right edge of the indent distance range , and update the difference between the area of ​​the expanded area corresponding to the middle value and the area of ​​the text area to the difference between the area of ​​the expanded area corresponding to the right boundary and the area of ​​the text area .

[0202] exist In the case of Updated to the left edge of the indent distance range , and update the difference between the area of ​​the expanded area corresponding to the intermediate value and the area of ​​the text area to the difference between the area of ​​the expanded area corresponding to the left boundary and the area of ​​the text area .

[0203] After updating the retracted distance range, the electronic device may continue to determine whether the difference between the left boundary and the right boundary of the updated retracted distance range is greater than the minimum retracted precision. , and the difference between the left and right boundaries is greater than the minimum indentation precision , continue to calculate the middle value of the left boundary and the right boundary, and use the middle value to continue to update the indentation distance range until the difference between the left boundary and the right boundary of the current indentation distance range is no greater than the minimum indentation precision .

[0204] The difference between the left and right boundaries of the current retraction distance range is not greater than the minimum retraction precision In the case of , the electronic device can return to the left boundary (return ), that is, the left border is used as the indentation distance corresponding to the text area; In the case of ), that is, the left edge is used as the corresponding indentation distance of the text area. In the case of ), that is, the right border is used as the corresponding indentation distance of the text area.

[0205] The code implementation of the above binary search process is as follows:

[0206] Input: polygon P, consisting of a series of points;

[0207] Output: shrinkage amplitude D;

[0208] Initialization: Minimum indentation precision , external expansion parameters during inference ;

[0209] 1. Set the left boundary of the search ;

[0210] 2. Calculate the circumference of GT , set the right boundary of the search ;

[0211] 3. Calculate the area of ​​GT ;

[0212] 4. Get the left edge and shrink it ;

[0213] 5. Yes To expand outward, , ;

[0214] 6. Calculate the error area on the left side. ;

[0215] 7. Calculate the error area on the right side in the same way as steps 4 to 6. , if the indentation fails, then ;

[0216] 8. While ;

[0217] 9. ;

[0218] 10. Follow the same method as steps 4 to 6 to calculate , if the indentation fails, then ;

[0219] 11. if or ;

[0220] 12. , ;

[0221] 13. else:

[0222] 14. , ;

[0223] 15. if is None;

[0224] 16. return ;

[0225] 17. if ;

[0226] 18. return ;

[0227] 19. else:

[0228] 20. return .

[0229] As shown in FIG4 (a), the image includes text regions "XX", "XX", "XX" and "XXXXXXXXXXXXXXXXXX". The indentation distances corresponding to the text regions are searched respectively. For the text region "XXXXXXXXXXXXXXXXXX", the above binary search method is used to search for the indentation distance corresponding to the text region, and the text region is indented using the indentation distance obtained by the search to obtain the indentation region 410 corresponding to the text region, and the outdentation distance is calculated according to the outdentation empirical parameter, and the indentation region is expanded according to the outdentation distance to obtain the outdentation region 420.

[0230] Compared with the inward shrinking area 430 obtained by shrinking the text area using the inward shrinking distance calculated using the fixed inward shrinking empirical parameter, and the outward expansion area 440 obtained by expanding the inward shrinking area 430 of the text area using the outward expansion distance calculated using the outward expansion empirical parameter as shown in FIG4 (b), the area of ​​the outward expansion area 420 obtained by searching the inward shrinking distance corresponding to the text area using the above-mentioned binary search method is closer to the area of ​​the text area, and the text detection accuracy is higher, which can not only avoid text adhesion, but also better retain text information, and the effect is better.

[0231] As an implementation method of the present application, Figure 5 As shown, the above step S105, i.e., the step of inputting each training image into the text detection model to be trained, training the text detection model based on the differences between the predicted probability map, the predicted threshold map, and the predicted binary map output by the text detection model and the segmentation map label, the threshold map label, and the binary map label, respectively, to obtain a trained text detection model, may include:

[0232] S501: Input each training image into the text detection model to be trained to obtain a prediction probability map, a prediction threshold map, and a prediction binary map.

[0233] When using each training image to train the text detection model to be trained, the electronic device can input each training image into the text detection model to be trained. The feature extraction module of the text detection model is used to extract the text features corresponding to the training image, and then the text feature map is respectively input into the probability map prediction module and the threshold map prediction module to obtain the predicted probability map and the predicted threshold map corresponding to the training image, and the predicted probability map and the predicted threshold map are input into the binarization module to obtain the predicted binary map corresponding to the training image.

[0234] S502: Based on the segmentation map label, the threshold map label, the binary map label, the predicted probability map, the threshold map label, the predicted binary map and a preset function, a loss value corresponding to the training image is calculated. The preset function includes a cross entropy loss function, a DICE loss function and a pre-constructed intersection-over-union loss function, and the intersection-over-union loss function is only used to calculate the difference between the text area shape in the training image predicted by the text detection model and the text area shape represented by the segmentation map label corresponding to the training image.

[0235] During the model training process, a preset function can be used to calculate the loss value corresponding to each training image, so as to adjust the model parameters according to the loss value corresponding to each training image. The preset function includes a cross entropy loss function, a DICE loss function, and a pre-built intersection-over-union loss function.

[0236] The above-mentioned cross entropy loss function (BCE loss) can provide a more direct gradient to help the model converge quickly; while the DICE loss function (DICE loss) is more reliable in measuring segmentation quality. However, from the perspective of text detection, the loss function should not only pay attention to the shape of the text area but also pay attention to the suppression of false positives. Since both the cross entropy loss function and the DICE loss function are pixel-level loss functions, the weight of paying attention to small objects in the loss is very small.

[0237] To this end, the Intersection over Union loss (IoUloss) function is pre-constructed. This IoU loss function is used to calculate the difference between the shape of each text region in the training image predicted by the text detection model and the shape of each text region represented by the segmentation map label corresponding to the training image. This loss function only considers the text region when calculating, so it can eliminate the interference of a large number of negative samples and focus on the shape constraints of the text region; and the loss function is inversely weighted according to the area of ​​the text region, so as to ensure that each text region has the same weight in the loss function, so that small targets can be valued. In addition, the IoU loss limited to the text region is introduced to improve the consistency of the shape of the semantic segmentation truth value and the model prediction.

[0238] After obtaining the predicted probability map, the predicted threshold map, and the predicted binary map, the electronic device can calculate the loss value corresponding to the training image based on the segmentation map label, the threshold map label, the binary map label, the predicted probability map, the threshold map label, the predicted binary map, and the preset function. Specifically, the electronic device can calculate the sum of the cross entropy loss value, the DICE loss value, and the intersection-and-union ratio loss value, and use the sum as the loss value corresponding to the training image.

[0239] S503: Adjust the parameters of the text detection model according to the loss value until the text detection model converges to obtain the text detection model.

[0240] After calculating the loss value corresponding to the training image, the electronic device can adjust the parameters of the text detection model according to the loss value until the text detection model converges to obtain a trained text detection model.

[0241] When the loss value corresponding to the training image indicates that the text detection model has converged, the training of the text detection model is completed; when the loss value corresponding to the training image indicates that the text detection model has not converged, the electronic device can calculate the back propagation gradient of the loss value, use stochastic gradient descent optimization to update the model parameters, and obtain a trained text detection model.

[0242] In this embodiment, the electronic device can use each training image to train the text detection module. Specifically, the electronic device can input each training image into the text detection model to be trained to obtain the predicted probability map, predicted threshold map and predicted binary map of the training image. Afterwards, based on the segmentation map label, threshold map label, binary map label, predicted probability map, predicted threshold map, predicted binary map and preset function, the loss value corresponding to the training image is calculated, and the parameters of the text detection model are adjusted according to the loss value until the text detection model converges to obtain the text detection model. In this way, the model parameters of the model are continuously updated and optimized using the loss value of each training image, thereby achieving the purpose of training the text detection model.

[0243] As an implementation manner of an embodiment of the present application, the step of calculating the loss value corresponding to the training image based on the segmentation map label, the threshold map label, the binary map label, the predicted probability map, the predicted threshold map, the predicted binary map and the preset function may include:

[0244] Based on the segmentation map label, the prediction probability map and the cross entropy loss function, calculating a first cross entropy loss value corresponding to the training image;

[0245] Based on the binary image label, the predicted binary image and the cross entropy loss function, calculating a second cross entropy loss value corresponding to the training image;

[0246] Calculating a DICE loss value corresponding to the training image based on the threshold map label, the predicted threshold map, and the DICE loss function;

[0247] Based on the segmentation map label, the predicted probability map and the pre-constructed intersection-and-union loss function, a first intersection-and-union loss value corresponding to the training image is calculated; and / or, based on the binary map label, the predicted binary map and the pre-constructed intersection-and-union loss function, a second intersection-and-union loss value corresponding to the training image is calculated;

[0248] Calculate the first cross entropy loss value, the second cross entropy loss value, the DICE loss value, the first intersection-over-union loss value, and / or the sum of the second intersection-over-union loss value to obtain a loss value corresponding to the training image.

[0249] After obtaining the predicted probability map, predicted binary map and predicted threshold map of the training image output by the model, the electronic device can calculate the loss value of the training image based on the segmentation map label, the predicted probability map, the binary map label, the predicted binary map, the threshold map label, the predicted threshold map and the preset function. Among them, since the preset function can include a cross entropy loss function, a DICE loss function and a pre-constructed intersection-over-union loss function, the loss value can be the sum of the cross entropy loss value, the DICE loss value and the intersection-over-union loss value.

[0250] Based on this, the electronic device can calculate the first cross entropy loss value corresponding to the training image based on the segmentation map label, the predicted probability map and the cross entropy loss function; and calculate the second cross entropy loss value corresponding to the training image based on the binary map label, the predicted binary map and the cross entropy loss function.

[0251] The first cross entropy loss value and the second cross entropy loss value corresponding to the training image can be calculated respectively according to the following cross entropy loss function.

[0252] ;

[0253] in, is the cross entropy loss value; Sample collection for text region; is the prediction graph output by the model, is the label corresponding to the preset graph, in For the predicted probability map, is the segmentation map label; In the case of predicting a binary image, is a binary image label.

[0254] The electronic device can calculate the DICE loss value corresponding to the training image based on the threshold map label, the predicted threshold map and the DICE loss function.

[0255] The DICE loss value corresponding to the training image can be calculated according to the following DICE loss function.

[0256] ;

[0257] in, is the DICE loss value; is the prediction threshold map, is the threshold map label.

[0258] In addition, the electronic device can also calculate a first intersection-and-union loss value corresponding to the training image based on the segmentation map label, the predicted probability map, and a pre-constructed intersection-and-union loss function; and / or calculate a second intersection-and-union loss value corresponding to the training image based on the binary map label, the predicted binary map, and a pre-constructed intersection-and-union loss function.

[0259] As an implementation of the embodiment of the present application, the first intersection-over-combination loss value and / or the second intersection-over-combination loss value may be calculated according to the following formula:

[0260] ;

[0261] in, is the intersection-and-union loss value; is a pixel point in the prediction image output by the text detection model, is the pixel point in the label corresponding to the prediction image, is the length of the prediction graph, is the width of the prediction graph, When is a pixel point in the predicted probability map, is the pixel point in the segmentation map label, When is a pixel point in the predicted binary image, is a pixel point in the binary image label; for The weight of the region, if If the area is a non-text area, ,like belongs to the text area, , is the area of ​​the text region to which the pixel belongs.

[0262] Due to the weight It is the inverse of the area A of the text area to which it belongs, 1 / A. That is to say, the larger the area of ​​the text area to which it belongs, the smaller the weight of each pixel in the text area. In this way, the total weight of each text area is always the same. This avoids ignoring smaller targets, makes smaller targets more important, and makes the detection results more accurate.

[0263] Furthermore, the electronic device can calculate the sum of the first cross entropy loss value, the second cross entropy loss value, the DICE loss value, the first intersection-over-union loss value and / or the second intersection-over-union loss value to obtain the loss value corresponding to the training image.

[0264] As an implementation method, the electronic device can calculate the weighted sum of the first cross entropy loss value, the second cross entropy loss value, the DICE loss value, the first intersection-over-union loss value, and the second intersection-over-union loss value to obtain the loss value corresponding to the training image. The weight of each loss value can be set according to actual needs and is not specifically limited here.

[0265] The loss value corresponding to the training image can be calculated according to the following formula.

[0266] ;

[0267] in, is the loss value corresponding to the training image; is the first cross entropy loss value, is the weight of the first cross entropy loss value; is the second cross entropy loss value, is the weight of the second cross entropy loss value; is the DICE loss value; is the weight of the DICE loss value; is the first intersection-over-union loss value, is the weight of the first intersection loss value; is the second intersection-over-union loss value, is the weight of the second intersection-over-union loss value.

[0268] In this embodiment, the electronic device can calculate the first cross entropy loss value corresponding to the training image based on the segmentation map label, the predicted probability map and the cross entropy loss function; calculate the second cross entropy loss value corresponding to the training image based on the binary map label, the predicted binary map and the cross entropy loss function; calculate the DICE loss value corresponding to the training image based on the threshold map label, the predicted threshold map and the DICE loss function; calculate the first intersection and union loss value corresponding to the training image based on the segmentation map label, the predicted probability map and the pre-constructed intersection and union loss function; and / or, calculate the second intersection and union loss value corresponding to the training image based on the binary map label, the predicted binary map and the pre-constructed intersection and union loss function; and then calculate the sum of the first cross entropy loss value, the second cross entropy loss value, the DICE loss value, the first intersection and union loss value, and / or the second intersection and union loss value to obtain the loss value corresponding to the training image. The loss value is used to update and optimize the model parameters of the model, thereby achieving the purpose of training the text detection model. Moreover, since the loss value includes the intersection-over-union loss value, and the intersection-over-union loss function only considers the text area when calculating, it can eliminate the interference of a large number of negative samples and focus on the shape constraints of the text area; and the loss function is inversely weighted according to the area of ​​the text area, thereby ensuring that each text area has the same weight in the loss function, so that small targets can be taken seriously. In this way, the model optimized using the above loss value has a higher positioning accuracy.

[0269] Next, the construction process of the intersection-over-union loss function is introduced.

[0270] The original IoU loss function is:

[0271] ;

[0272] in, is the predicted probability map output by the model, is the segmentation map label. The above prediction probability map and segmentation map label are both collections of text regions. is the ratio of the intersection and union of two sets.

[0273] The IoU loss function is expressed in pixel-level form:

[0274] ;

[0275] ;

[0276] ;

[0277] In order to improve the positioning accuracy of the model, a weight map is introduced For each pixel belonging to the non-text area, is 0. For each pixel belonging to the text area, It is the inverse of the area A of the text area to which it belongs, 1 / A. That is to say, the larger the area of ​​the text area, the smaller the weight of each pixel in the text area. In this way, the total weight of each text area is always the same. The weighted IoU loss function can be expressed as:

[0278] ;

[0279] ;

[0280] ;

[0281] Corresponding to the above-mentioned text detection model training method, an embodiment of the present application also provides a text detection method.

[0282] like Figure 6 As shown, a text detection method, the method comprising:

[0283] S601: Acquire an image to be detected;

[0284] S602: Input the image to be detected into a pre-trained text detection model to obtain a binary image of the image to be detected output by the text detection model. The text detection model is trained by a text detection model training method provided in the present application;

[0285] S603: Determine a connected area in the binary image, obtain a text prediction box where the text in the image to be detected is located, and expand the text prediction box to obtain a region where the text in the image to be detected is located.

[0286] After the text detection model is trained, when it is desired to perform text detection on an image to be detected, the electronic device can obtain the image to be detected. For example, the electronic device can collect the image to be detected; for another example, the electronic device can obtain the image to be detected carried in a text detection instruction sent by a user, etc.

[0287] The image to be detected is an image containing text information to be recognized. The image to be detected may be a static image acquired by an image acquisition device or an image frame in a video, and there is no limitation on this.

[0288] It should be noted that the electronic device used to execute the text detection method can be the same electronic device as the electronic device used to execute the above-mentioned text detection model training method, or can be a different electronic device, and this is not limited here.

[0289] The electronic device can input the image to be detected into a text detection model pre-trained using a text detection model training method provided in the present application, and obtain a binary image of the image to be detected output by the text detection model. Then, each connected area in the binary image is determined, and each text prediction box where the text in the image to be detected is located is obtained, and each text prediction box is expanded as the area where each text in the image to be detected is located.

[0290] As an implementation method, the expansion distance of the text prediction box may be calculated according to the following formula:

[0291] ;

[0292] in, is the outward expansion distance; is the area of ​​the text prediction box; is the externalized empirical parameter; The perimeter of the text prediction box.

[0293] The above-mentioned process of expanding the text prediction box, the process of expanding each text area to obtain the expanded area when calculating the label corresponding to the training image during the model training process, and the process of expanding the retracted area when searching for the retracted distance during the model training process are all different, and the three target different expansion objects and expansion distances.

[0294] As can be seen from the above, since the inward distance corresponding to each text area is searched within the inward distance range corresponding to the text area, the area of ​​the expanded area obtained after the text area is first inwardly shrunk and then expanded is the same as the area of ​​the text area. In this way, compared with the method of inwardly shrunk and expanded according to fixed inward empirical parameters, for various types of text areas, the inward and outward expansion of the text area using the inward distance corresponding to the text area can maintain the consistency of the area before and after the inward and outward expansion. Therefore, it can avoid adhesion between text areas and the loss of text information, and improve the accuracy of text detection results. And the inward area of ​​each text area obtained by the inward distance corresponding to each text area is used to calculate the segmentation map label, threshold map label and binary map label corresponding to the training image, which can improve the accuracy of the training label, and then use each training image and the label corresponding to each training image to train the text detection model, which can improve the text positioning accuracy of the trained text detection model. In this way, the accuracy of the detection result obtained by using the text detection model to perform text detection on the image to be detected is high.

[0295] As an implementation of an embodiment of the present application, a text detection method provided by the present application may also include:

[0296] According to the second preset search target, four target points corresponding to the area where the text is located are searched among the edge points of the area where the text is located, wherein the second preset search target is a quadrilateral formed by the four target points obtained by the search and has the largest overlapping area with the area where the text is located; the quadrilateral formed by the four target points is determined as the area where the text is located in the image to be detected.

[0297] Since the image to be detected may include a certain proportion of curved texts, and the detection results of the text detection model often have higher evaluation indicators, based on this, the area where each text is located predicted by the text detection model can be an arbitrary polygon.

[0298] However, after obtaining the prediction map output by the text detection model, the electronic device also needs to cut out the prediction map to obtain the area where each text is located, and input the obtained area where each text is located into the preset text recognition model for recognition. When the area where each text is located is an arbitrary polygon, it is not convenient for the text recognition model to recognize the area where the text is located. When the area where the text is located is an arbitrary quadrilateral or a rotated rectangle, the electronic device can convert the text area into a rectangle through transformation methods such as perspective projection, which is convenient for the text recognition model to recognize the area where the text is located. The arbitrary quadrilateral will correct some deformation information during the projection transformation. The rotated rectangle will preserve a relatively original shape.

[0299] like Figure 7 As shown, after the image to be detected is input into the text detection model, for each text area in the image to be detected predicted by the model, the area of ​​the text area 720 in the segmentation diagram predicted by the text detection model is consistent with the area of ​​the indented area 710 after the text area is indented using the indented distance corresponding to the text area. However, since the text area 720 is an irregular polygon, there is a certain amount of noise at the edge of the text area 720. In order to facilitate subsequent character recognition, the minimum bounding rectangle 730 can be taken for the text area 720. However, the minimum bounding rectangle 730 of the text area 720 will cause the prediction box corresponding to the text area to become larger, especially for long texts. The larger size of some text boxes will cause the duty cycle of the data of the recognition model to fluctuate, which may further lead to a decrease in the performance of the text recognition model.

[0300] Based on this, in order to have a higher detection intersection-merge ratio in the area where the text is located, and to provide the recognition model with a text area with a more stable duty cycle, thereby improving the end-to-end recognition rate, when the area where the text is located is an arbitrary polygon, the electronic device can convert the area where the text is located into a quadrilateral with the largest overlapping area with the text area.

[0301] The electronic device can search for four vertices of a quadrilateral among the edge points of the region where the text is located according to the second preset search target, and then determine the quadrilateral composed of the four searched vertices as the region where the text is located in the image to be detected. The second preset search target is that the overlap area between the quadrilateral composed of the four searched target points and the region where the text is located is the largest.

[0302] As an implementation method, the electronic device can extract each edge point in the area where the text is located in OPENCV (Open Source Computer Vision Library), and select four candidate vertices from multiple edge points, and then walk on the edge of the polygon in the area where the text is located, and search for the four vertices with the largest score of the quadrilateral formed as the four vertices of the quadrilateral corresponding to the area where the text is located.

[0303] In this embodiment, the electronic device can search for four target points corresponding to the area where the text is located in each edge point of the area where the text is located according to the second preset search target, wherein the second preset search target is that the quadrilateral formed by the four target points obtained by the search has the largest overlap area with the area where the text is located. And the quadrilateral formed by the four target points obtained by the search is determined as the area where the text is located in the image to be detected. In this way, the area where the text is located is converted from a polygon to a quadrilateral, which facilitates the subsequent recognition model to perform text recognition on the area where the text is located. In addition, the obtained quadrilateral has a large overlap area with the area where the text is located, which avoids the loss of text information as much as possible.

[0304] As an implementation of an embodiment of the present application, the step of searching for four target points corresponding to the region where the text is located among the edge points of the region where the text is located according to the second preset search target may include:

[0305] Select four candidate points from the edge points of the area where the text is located; calculate the score of the quadrilateral formed by the four candidate points, wherein the score is determined based on the area and / or side length of the quadrilateral; traverse the candidate points in turn, and when traversing to each candidate point, update the target point within the search range corresponding to the candidate point to the candidate point, wherein the target point is: the edge point with the highest score and higher than the score of the quadrilateral formed by the other three candidate points except the candidate point, and the search range of the candidate point includes the edge points between the previous candidate point and the next candidate point of the candidate point; after the traversal is completed, use the current four candidate points as the four target points corresponding to the area where the text is located.

[0306] The electronic device may select four candidate points from the edge points of the region where the text is located according to the number of edge points of the region where the text is located.

[0307] As an implementation method, the electronic device can randomly select four candidate points from each edge point in the text area according to the number of edge points in the text area, so that the number of edge points between each two adjacent candidate points in the selected four candidate points is different.

[0308] As an implementation method, the electronic device can select four candidate points from each edge point of the text area according to the number of edge points of the text area, in accordance with the progressive step length, so that the number of edge points between each two adjacent candidate points in the selected four candidate points is different. Among them, the progressive step length is that when selecting each edge point in turn, the step length taken to select the next edge point is greater than the step length taken when selecting the previous edge point. And the progressive step length is determined based on the number of edge points. For example, the electronic device can randomly select point A, and then select point B with a step length of 30 edge points, that is, there are 30 edge points between point A and point B; then select point C with a step length of 80 edge points, that is, there are 80 edge points between point B and point C; finally select point D with a step length of 100 edge points, that is, there are 100 edge points between point C and point D.

[0309] As an implementation mode, the electronic device may select four candidate points from each edge point of the text area according to the number of edge points of the text area in an equal step size manner, so that the number of edge points between each two adjacent candidate points among the four candidate points is the same, for example, the selected four points are point A, point B, point C and point D, point A is adjacent to point B and point D respectively, point B is adjacent to point C and point A respectively, point C is adjacent to point B and point D respectively, and point D is adjacent to point C and point A respectively. The number of edge points between points AB, the number of edge points between points BC, the number of edge points between points CD, and the number of edge points between points DA are the same.

[0310] The electronic device may calculate the score of the quadrilateral formed by the four candidate points based on the area and / or side length of the quadrilateral.

[0311] As an implementation mode, the electronic device may calculate the area of ​​a quadrilateral formed by four candidate points, and calculate the score of the quadrilateral based on the area of ​​the quadrilateral and a preset score calculation formula.

[0312] As an implementation mode, the electronic device may respectively calculate the area and perimeter of a quadrilateral formed by four candidate points, and calculate the score of the quadrilateral based on the area, perimeter and a preset score calculation formula of the quadrilateral.

[0313] Afterwards, the electronic device can sequentially traverse each candidate point, and when traversing to each candidate point, update the target point in the search range corresponding to the candidate point to the candidate point. The target point is: the edge point with the highest score and higher than the score of the quadrilateral formed by the other three candidate points except the candidate point, and the search range of the candidate point includes the edge points between the candidate point before the candidate point and the candidate point after the candidate point.

[0314] When each candidate point is updated, the score of the quadrilateral formed by the candidate point and the other three candidate points is also updated accordingly, and when the next candidate point is traversed subsequently, the score used is also the updated score.

[0315] Exemplarily, the electronic device can traverse the four points ABCD in sequence. When traversing to point A, the electronic device can respectively calculate the scores of the quadrilateral formed by the edge points between point D and point B and the three points BCD, and finally update point A using the edge point with the highest score of the quadrilateral formed by the three points BCD and higher than the score of the quadrilateral formed by ABCD, and update the score of the quadrilateral.

[0316] When traversing to point B, the electronic device can calculate the scores of each edge point between point A and point C and the quadrilateral formed by the three points ACD, and finally use the edge point with the highest score of the quadrilateral formed by the three points ACD and higher than the score of the quadrilateral formed by ABCD to update point B and update the score of the quadrilateral.

[0317] When traversing to point C, the electronic device can calculate the scores of each edge point between point B and point D and the quadrilateral formed by the three points ABD, and finally use the edge point with the highest score of the quadrilateral formed by the three points ABD and higher than the score of the quadrilateral formed by ABCD to update point C and update the score of the quadrilateral.

[0318] When traversing to point D, the electronic device can respectively calculate the scores of the quadrilateral formed by each edge point between point C and point A and the three points ABC, and finally use the edge point with the highest score of the quadrilateral formed by the three points ABC and higher than the score of the quadrilateral formed by the three points ABC to update point D and update the score of the quadrilateral.

[0319] As an implementation method, the electronic device may sequentially traverse each candidate point, and when traversing to each candidate point, update the target point in the search range corresponding to the candidate point to the candidate point. The target point is: the edge point with the highest score and higher than the score of the quadrilateral formed by the other three candidate points except the candidate point, and the search range of the candidate point includes the edge points between the previous candidate point and the candidate point, and the edge points between the candidate point and the next candidate point.

[0320] Exemplarily, the electronic device can sequentially traverse the four points ABCD. When traversing to point A, the electronic device can respectively calculate the scores of the quadrilaterals formed by the edge points between point D and point A, and the edge points between point A and point B, respectively, and the three points BCD, and finally use the edge points with the highest score of the quadrilateral formed by the three points BCD and higher than the score of the quadrilateral formed by ABCD to update point A and update the score of the quadrilateral. When traversing to point B, the electronic device can respectively calculate the scores of the quadrilaterals formed by the edge points between point A and point B, and the edge points between point B and point C, respectively, and the three points ACD, and finally use the edge points with the highest score of the quadrilateral formed by the three points ACD and higher than the score of the quadrilateral formed by ABCD to update point B and update the score of the quadrilateral. Continue to traverse points C and D according to the above method, and after traversing the four points ABCD, the four points ABCD are obtained.

[0321] After the traversal is completed, the quadrilateral formed by the four points finally selected has the highest score, and the electronic device can use the current four candidate points as the four target points corresponding to the area where the text is located.

[0322] In this embodiment, the electronic device can first select four candidate points from each edge point of the area where the text is located, and then calculate the score of the quadrilateral formed by the four candidate points; then traverse each candidate point in turn, and when traversing to each candidate point, update the target point in the search range corresponding to the candidate point to the candidate point, and after the traversal is completed, use the current four candidate points as the four target points corresponding to the area where the text is located. In this way, by searching for four vertices whose area of ​​the constructed quadrilateral overlaps with the area of ​​the text area among the edge points of the area where the text is located, a quadrilateral with a higher IoU with the area where the text is located can be searched, thereby reducing text information loss and facilitating text recognition processing. By converting the polygon into a quadrilateral, it has a higher IoU than the minimum circumscribed rotated rectangle.

[0323] As an implementation manner of an embodiment of the present application, the step of calculating the score of the quadrilateral formed by the four candidate points includes:

[0324] The score is calculated according to the following formula:

[0325] ;

[0326] in, For the score, is the area of ​​the quadrilateral formed by the four candidate points, is the perimeter of the quadrilateral formed by the four candidate points, is the total length of the diagonals of the quadrilateral formed by the four candidate points.

[0327] For every four candidate points, the electronic device can calculate the score of the quadrilateral formed by the four candidate points according to the above formula, and use the scores of each quadrilateral to search among the edge points of the area where the text is located to obtain four vertices whose area overlaps more with the area where the text is located.

[0328] As an implementation method, the score of the quadrilateral composed of four candidate points can also be calculated according to other formulas related to the area and / or side length of the quadrilateral. For example, ,or etc. to calculate the score of the quadrilateral.

[0329] In this embodiment, the electronic device uses the above formula to calculate the score of the quadrilateral formed by each candidate vertex, which can improve the vertex search efficiency. Moreover, since the formula is related to the area and perimeter of the quadrilateral, the quadrilateral with the highest score calculated by the formula has a higher degree of overlap with the area where the text is located.

[0330] The following specifically describes the process of converting a polygon into an arbitrary quadrilateral by the electronic device searching for each edge point of the region where the text is located according to the second preset search target in conjunction with the code.

[0331] For each text region, when converting the text region into a quadrilateral, the input is the polygon P of the text region, which is composed of a series of edge points and has a length greater than or equal to 4, for example: [(50, 59), (70, 68), ...].

[0332] The final output of this process is a quadrilateral composed of four points. Before performing vertex search, the number of edge points can be initialized to , and set the untraversed edge points After traversing the edge point, update the edge point of the edge point , that is, after traversing each edge point, the edge point .

[0333] Determine the number of edge points in the area where the text is located, and based on the number of edge points , determine the step length Then determine the candidate vertices among the edge points in the area where the text is located. , based on the step size, determine the candidate vertices , Alternative Vertex and candidate vertices , where the candidate vertices Located at the candidate vertex and candidate vertices Between, alternative vertices Located at the candidate vertex and candidate vertices Between, alternative vertices Located at the candidate vertex and candidate vertices Between, and, alternative vertices Located at the candidate vertex and candidate vertices between.

[0334] Using preset formulas Right now , calculate the candidate vertices , Alternative Vertex , Alternative Vertex And the alternative vertex The fraction of the quadrilateral formed by , as the current maximum score .

[0335] Traverse each candidate vertex in turn, and when you traverse to the candidate vertex When The point between The points between are considered as candidate points, and the correlation between each candidate point and point is calculated separately. Fractions of quadrilaterals .if Greater than , then the Updated to , and update the candidate point to the candidate vertex When traversing all the candidate vertices After the candidate points are determined, the candidate point with the highest score in the quadrilateral formed by the other three candidate points is determined. When using Update candidate vertices . Using the same method, traverse the candidate vertices , Alternative Vertex And the alternative vertex , and update the candidate vertices , Alternative Vertex And the alternative vertex .

[0336] After traversing the four candidate vertices, the searched candidate vertex is returned , Alternative Vertex , Alternative Vertex And the alternative vertex The quadrilateral formed by the text is the quadrilateral corresponding to the area where the text is located.

[0337] The code to implement the above search process is as follows:

[0338] Input: A polygon P consists of a series of coordinate points with a length greater than or equal to 4, such as [(50, 59), (70, 68), …];

[0339] Output: a quadrilateral consisting of four points;

[0340] initialization: , ;

[0341] , the number of vertices;

[0342] , , , , initialize four candidate points;

[0343] , calculate the score of the current quadrilateral, the score is defined as: ;

[0344] While :

[0345] ;

[0346] , point D is on the left side of point A, point B is on the right side of point A, the points between AD and AB in polygon P are all considered as candidate points, and the other three points are similar;

[0347] for in new_a_list:

[0348] ;

[0349] if > :

[0350] ;

[0351] ;

[0352] ;

[0353] ;

[0354] for in new_b_list:

[0355] ;

[0356] if > :

[0357] ;

[0358] ;

[0359] ;

[0360] ;

[0361] for in new_c_list:

[0362] ;

[0363] if > :

[0364] ;

[0365] ;

[0366] ;

[0367] ;

[0368] for in new_d_list:

[0369] ;

[0370] if > :

[0371] ;

[0372] ;

[0373] ;

[0374] return .

[0375] As an implementation of an embodiment of the present application, a text detection method provided by the present application may also include:

[0376] The longer midline of the two midlines of the text region is used as the target midline of the rotated rectangle to be constructed, and a rotated rectangle with the same area as the text region is constructed as the text region in the image to be detected.

[0377] During the text recognition process, the text area will be corrected through projection transformation, and any quadrilateral may lose shape information during the projection transformation, while the rotated rectangle will preserve the initial shape. Based on this, in order to facilitate subsequent text recognition, when the text area is a quadrilateral, the text area can be converted into a rotated rectangle.

[0378] Based on this, in order to improve the accuracy of subsequent text recognition results, the electronic device may post-process the text region in the binary image output by the text detection model to convert the text region into a rotated rectangle with the same area.

[0379] Specifically, the electronic device can use the longer midline of the two midlines of the area where the text is located as the target midline of the rotated rectangle to be constructed, and construct a rotated rectangle with the same area as the area where the text is located as the area where the text is located in the image to be detected. In this way, the obtained rotated rectangle has the same area as the area where the text is located, thereby avoiding fluctuations in the duty cycle of the data of the recognition model caused by the large text box, and eliminating the influence of edge noise of irregular graphics on the detection results.

[0380] In this embodiment, the electronic device can use the longer midline of the two midlines of the area where the text is located as the target midline of the rotated rectangle to be constructed, and construct a rotated rectangle with the same area as the area where the text is located. In this way, by post-processing the area where the text is located and converting the area where the text is located into a rotated rectangle with the same area, the information loss caused by the subsequent projection transformation can be reduced, and the impact of a high duty cycle on the performance of the text recognition model can be avoided. In addition, by converting a polygon or quadrilateral into a rotated rectangle, a higher IoU is achieved than the minimum circumscribed rotated rectangle.

[0381] As an implementation of an embodiment of the present application, the step of taking the longer midline of the two midlines of the region where the text is located as the target midline of the rotated rectangle to be constructed and constructing a rotated rectangle having the same area as the area of ​​the region where the text is located may include:

[0382] The two end points of the longer midline of the two midlines of the region where the text is located are respectively used as the midpoints of the two short sides of the rotated rectangle to be constructed; the ratio of the area of ​​the region where the text is located to the length of the longer midline of the two midlines of the region where the text is located is used as the length of the two short sides of the rotated rectangle to be constructed; the longer midline of the two midlines of the region where the text is located is used as the target midline of the rotated rectangle to be constructed, and the four vertices of the rotated rectangle are determined according to the length of the short sides and the midpoints of the two short sides so that the two short sides of the rotated rectangle are perpendicular to the target midline; the quadrilateral formed by the four vertices is used as the rotated rectangle corresponding to the region where the text is located.

[0383] like Figure 8 As shown, the electronic device can determine the lengths of the two center lines AC and BD of the text area respectively, and use the two endpoints A and C of the longer center line AC of the two center lines of the text area as the midpoints of the two short sides of the rotated rectangle to be constructed.

[0384] Afterwards, the electronic device may use the ratio of the area of ​​the region where the text is located to the length of the longer midline of the two midlines of the region where the text is located as the lengths of the two short sides of the rotated rectangle to be constructed.

[0385] Next, the electronic device can use the longer midline of the two midlines of the area where the text is located as the target midline AC of the rotated rectangle to be constructed, and determine the two short sides of the rotated rectangle perpendicular to the target midline AC based on the length of the short side and the midpoints A and C of the two short sides, and then use the vertices of the two short sides as the vertices of the rotated rectangle to obtain the four vertices of the rotated rectangle. And the quadrilateral formed by the four vertices is used as the rotated rectangle corresponding to the area where the text is located.

[0386] In this embodiment, the electronic device can use the two endpoints of the longer midline of the two midlines of the area where the text is located as the midpoints of the two short sides of the rotated rectangle to be constructed. Afterwards, the electronic device can use the ratio of the area of ​​the area where the text is located to the length of the longer midline of the two midlines of the area where the text is located as the length of the two short sides of the rotated rectangle to be constructed. Then, the longer midline of the two midlines of the area where the text is located is used as the target midline of the rotated rectangle to be constructed, and the four vertices of the rotated rectangle are determined according to the length of the short side and the midpoint of the two short sides, so that the two short sides of the rotated rectangle are perpendicular to the target midline; the quadrilateral formed by the four vertices is used as the rotated rectangle corresponding to the area where the text is located. In this way, the constructed rotated rectangle has the same midline and the same area as the area where the text is located, and the text information of the area where the text is located is retained as much as possible, reducing information loss.

[0387] As an implementation of an embodiment of the present application, the step of taking the longer midline of the two midlines of the area where the text is located as the target midline of the rotated rectangle to be constructed, and determining the four vertices of the rotated rectangle according to the length of the short side and the midpoints of the two short sides may include:

[0388] A direction vector perpendicular to the centerline vector of the target centerline is used as the direction vector of the short side, wherein the direction of the centerline vector is one of the extension directions of the target centerline; a target vector is obtained based on the direction vector and the length of the short side, wherein the direction of the target vector is the same as the direction vector, and the length of the target vector is half of the length of the short side; the sum of the midpoint of each short side and the target vector is calculated respectively to obtain two vertices of the rotated rectangle; the difference between the midpoint of each short side and the target vector is calculated respectively to obtain the other two vertices of the rotated rectangle.

[0389] For a rectangle, each median is parallel to two sides of the rectangle and perpendicular to the other two sides. For example, the median between the two short sides is parallel to the two long sides of the rectangle and perpendicular to the two short sides of the rectangle.

[0390] Based on this, Figure 8 As shown, after determining the target center line AC of the two short sides of the rotated rectangle, the electronic device can determine that the direction vector A of the target center line is perpendicular to the direction vector of the short side. For example, the electronic device can rotate the direction vector A of the center line by 90 degrees to obtain a direction vector perpendicular to the short side.

[0391] Furthermore, the electronic device may use a direction vector perpendicular to the direction vector A of the target center line AC as the direction vector B of the short side, wherein the direction of the direction vector A is one of the extension directions of the target center line AC.

[0392] Next, the electronic device can obtain a target vector C based on the direction vector B of the short side and the length of the short side, wherein the direction of the target vector C is the same as the direction vector B of the short side, and the length of the target vector C is half of the length of the short side.

[0393] Furthermore, the electronic device can calculate the sum of the midpoint of each short side and the target vector C respectively, and obtain the two vertices A1 and C1 of the rotated rectangle. Among them, the sum of the midpoint and the target vector C is the length of translating the midpoint along the direction of the target vector C. And calculate the difference between the midpoint of each short side and the target vector C respectively, and obtain the other two vertices A2 and C2 of the rotated rectangle. Among them, the difference between the midpoint and the target vector C is the length of translating the midpoint along the opposite direction of the target vector C. Furthermore, the quadrilateral formed by vertices A1, A2, C1 and C2 is a rotated rectangle with the same area as the area where the text is located.

[0394] In this embodiment, the electronic device can use the direction vector perpendicular to the midline vector of the target midline as the direction vector of the short side; obtain the target vector based on the direction vector and the length of the short side; calculate the sum of the midpoint of each short side and the target vector to obtain two vertices of the rotated rectangle; calculate the difference between the midpoint of each short side and the target vector to obtain the other two vertices of the rotated rectangle. In this way, the obtained rotated rectangle has the same midline and the same area as the area where the text is located, and the text information of the area where the text is located is retained as much as possible, which can reduce information loss. In addition, by converting a polygon or quadrilateral into a rotated rectangle, it has a higher IoU than the minimum circumscribed rotated rectangle.

[0395] The following is a detailed description of the process of converting the area where the text is located into a rotated rectangle by an electronic device, combined with the code.

[0396] Electronic recognition can first calculate the area where the text is located The area of ​​the quadrilateral . Then, according to the coordinates of the four vertices , determine the midpoint of each edge separately , , as well as The coordinates of the text area are obtained by and .

[0397] Calculate the lengths of the two medians respectively. and , and use the longer of the two midlines as the target midline between the two short sides of the rotated rectangle.

[0398] The electronic device can then use the area of ​​the text area to and the length of the target midline, calculate the length L of the short side of the rotated rectangle .

[0399] The electronic device can then determine the direction vector of the target centerline, i.e., the long side, and rotate the direction vector of the long side 90° clockwise to obtain the unit direction vector of the short side. The product of the unit direction vector of the short side and half the length of the short side is calculated to obtain the target vector , respectively calculate the sum of the two endpoints of the target midline and the target vector, and calculate the difference between the two endpoints of the target midline and the target vector to obtain the four vertices of the rotated rectangle.

[0400] The code to implement the above vertex determination process is as follows:

[0401] Input: Any quadrilateral ;

[0402] Output: a rotated rectangle consisting of four points;

[0403] , calculate the area of ​​any quadrilateral;

[0404] ;

[0405] ;

[0406] ;

[0407] ;

[0408] , , calculate the length of the two medians;

[0409] > :

[0410] , calculate the length of the short side;

[0411] , the vector of the long side;

[0412] , rotate 90° clockwise;

[0413] , the unit vector of the short side is obtained;

[0414] ;

[0415] ;

[0416] ;

[0417] ;

[0418] ;

[0419] else:

[0420] , calculate the length of the short side;

[0421] , the vector of the long side;

[0422] , rotate 90° clockwise;

[0423] , the unit vector of the short side is obtained;

[0424] ;

[0425] ;

[0426] ;

[0427] ;

[0428] .

[0429] Corresponding to the text detection model training method, the present embodiment also provides a text detection model training device.

[0430] like Fig. 9 As shown, a text detection model training device includes:

[0431] A first search module 910 is used to search for the corresponding indentation distance of the text region within the indentation distance range corresponding to the text region according to a first preset search target for each text region in each training image, wherein the first preset search target is the indentation distance obtained by searching so that the area of ​​the expanded region is the same as the area of ​​the text region, and the expanded region is the region obtained by shrinking the text region according to the indentation distance and then expanding it;

[0432] A shrinking module 920, configured to shrink each text area in each training image based on the shrinking distance corresponding to each text area in the training image, to obtain a shrinking image of each training image;

[0433] An expansion module 930 is used to expand each text region in each training image to obtain an expansion region corresponding to each text region;

[0434] The label generation module 940 is used to generate a segmentation map label, a threshold map label and a binary map label corresponding to each training image based on the inner image of each training image and the outer expansion area corresponding to each text area in the training image;

[0435] The training module 950 is used to input each training image into the text detection model to be trained, and train the text detection model based on the differences between the predicted probability map, the predicted threshold map and the predicted binary map output by the text detection model and the segmentation map label, the threshold map label and the binary map label, so as to obtain a trained text detection model.

[0436] As can be seen from the above, since the inward contraction distance corresponding to each text area is searched within the inward contraction distance range corresponding to the text area, the area of ​​the expanded area obtained after the text area is first contracted and then expanded is the same as the area of ​​the text area. In this way, compared with the method of contracting and expanding according to fixed inward contraction experience parameters, for various types of text areas, the inward contraction distance corresponding to the text area is used to contract and expand the text area, and the consistency of the area before and after contraction and expansion can be maintained. Therefore, it can avoid adhesion between text areas and the loss of text information, and improve the accuracy of text detection results. And by using the contraction area of ​​each text area obtained by the inward contraction distance corresponding to each text area, the segmentation map label, threshold map label and binary map label corresponding to the training image can be calculated, which can improve the accuracy of the training label, and then use each training image and the label corresponding to each training image to train the text detection model, which can improve the text positioning accuracy of the trained text detection model.

[0437] As an implementation of the embodiment of the present application, the search module 910 includes:

[0438] A range determination submodule is used to determine the indentation distance range for each text area in each training image, with 0 as the left boundary of the indentation distance range and a target value as the right boundary of the indentation distance range, wherein the target value is determined based on the perimeter of the text area;

[0439] A distance determination submodule, configured to determine an alternative retracted distance within the retracted distance range when a difference between a left boundary and a right boundary of the retracted distance range is greater than a preset accuracy threshold;

[0440] An updating submodule is used to update the retraction distance range based on the alternative retraction distance, and return to execute the step of determining the alternative retraction distance within the retraction distance range when the difference between the left boundary and the right boundary of the retraction distance range is greater than a preset accuracy threshold, until the difference between the left boundary and the right boundary of the current retraction distance range is not greater than the preset accuracy threshold, and the retraction distance corresponding to the text area is determined based on the current retraction distance range.

[0441] As an implementation of an embodiment of the present application, the updating submodule includes:

[0442] a first updating unit, configured to obtain an updated inward distance range by using the alternative inward distance as the right boundary of the inward distance range, when inward contraction using the alternative inward contraction distance fails or the area of ​​the alternative outward expansion region obtained based on the alternative inward contraction distance is smaller than the area of ​​the text region, wherein the area of ​​the alternative outward expansion region is the area of ​​the outward expansion region obtained based on the alternative inward contraction distance;

[0443] The second updating unit is configured to obtain an updated retracted distance range by using the candidate retracted distance as the left boundary of the retracted distance range when the area of ​​the candidate outward expansion region is larger than the area of ​​the text region.

[0444] As an implementation of an embodiment of the present application, the updating submodule includes:

[0445] A first determining unit is used to return the left boundary of the current inward shrinking distance range as the inward shrinking distance corresponding to the text area when the inward shrinking using the right boundary of the current inward shrinking distance range fails or the absolute value of the first area difference is greater than the absolute value of the second area difference, wherein the first area difference is the difference between the area of ​​the outward expansion area obtained based on the right boundary and the area of ​​the text area, and the second area difference is the difference between the area of ​​the outward expansion area obtained based on the left boundary and the area of ​​the text area;

[0446] The second determining unit is used to return the right boundary of the current indentation distance range as the indentation distance corresponding to the text area when the absolute value of the first area difference is not greater than the absolute value of the second area difference.

[0447] As an implementation of the embodiment of the present application, the training module 950 includes:

[0448] An image input submodule is used to input each training image into the text detection model to be trained to obtain a prediction probability map, a prediction threshold map, and a prediction binary map;

[0449] A loss value calculation submodule, used to calculate the loss value corresponding to the training image based on the segmentation map label, the threshold map label, the binary map label, the predicted probability map, the threshold map label, the predicted binary map and a preset function, wherein the preset function includes a cross entropy loss function, a DICE loss function and a pre-constructed intersection-over-union loss function, and the intersection-over-union loss function is only used to calculate the difference between the text area shape in the training image predicted by the text detection model and the text area shape represented by the segmentation map label corresponding to the training image;

[0450] A parameter adjustment submodule is used to adjust the parameters of the text detection model according to the loss value until the text detection model converges to obtain the text detection model.

[0451] As an implementation of an embodiment of the present application, the loss value calculation submodule includes:

[0452] A first calculation unit, configured to calculate a first cross entropy loss value corresponding to the training image based on the segmentation map label, the prediction probability map, and the cross entropy loss function;

[0453] A second calculation unit, configured to calculate a second cross entropy loss value corresponding to the training image based on the binary image label, the predicted binary image, and the cross entropy loss function;

[0454] A third calculation unit, configured to calculate a DICE loss value corresponding to the training image based on the threshold map label, the predicted threshold map, and the DICE loss function;

[0455] A fourth calculation unit is used to calculate a first intersection-and-union loss value corresponding to the training image based on the segmentation map label, the predicted probability map and a pre-constructed intersection-and-union loss function; and / or, based on the binary map label, the predicted binary map and a pre-constructed intersection-and-union loss function, calculate a second intersection-and-union loss value corresponding to the training image;

[0456] The fifth calculation unit is used to calculate the first cross entropy loss value, the second cross entropy loss value, the DICE loss value, the first intersection-and-union loss value, and / or the sum of the second intersection-and-union loss value to obtain the loss value corresponding to the training image.

[0457] As an implementation manner of the embodiment of the present application, the fifth computing unit includes:

[0458] The calculation subunit is used to calculate the first intersection-over-combination loss value and / or the second intersection-over-combination loss value according to the following formula:

[0459] ;

[0460] in, is the intersection-and-union loss value; is a pixel point in the prediction image output by the text detection model, is the pixel point in the label corresponding to the prediction image, is the length of the prediction graph, is the width of the prediction graph, When is a pixel point in the predicted probability map, is the pixel point in the segmentation map label, When is a pixel point in the predicted binary image, is a pixel point in the binary image label; for The weight of the region, if If the area is a non-text area, ,like belongs to the text area, , is the area of ​​the text region to which the pixel belongs.

[0461] Corresponding to the text detection method, the present embodiment also provides a text detection device.

[0462] like Fig.10 As shown, a text detection device comprises:

[0463] An image acquisition module 1010 is used to acquire an image to be detected;

[0464] A binary image acquisition module 1020 is used to input the image to be detected into a pre-trained text detection model to obtain a binary image of the image to be detected output by the text detection model, wherein the text detection model is trained by any text detection module training method provided in the embodiments of the present application;

[0465] The region determination module 1030 is used to determine the connected region in the binary image, obtain the text prediction box where the text in the image to be detected is located, and expand the text prediction box to obtain the region where the text in the image to be detected is located.

[0466] As can be seen from the above, since the inward distance corresponding to each text area is searched within the inward distance range corresponding to the text area, the area of ​​the expanded area obtained after the text area is first inwardly shrunk and then expanded is the same as the area of ​​the text area. In this way, compared with the method of inwardly shrunk and expanded according to fixed inward empirical parameters, for various types of text areas, the inward and outward expansion of the text area using the inward distance corresponding to the text area can maintain the consistency of the area before and after the inward and outward expansion. Therefore, it can avoid adhesion between text areas and the loss of text information, and improve the accuracy of text detection results. And the inward area of ​​each text area obtained by the inward distance corresponding to each text area is used to calculate the segmentation map label, threshold map label and binary map label corresponding to the training image, which can improve the accuracy of the training label, and then use each training image and the label corresponding to each training image to train the text detection model, which can improve the text positioning accuracy of the trained text detection model. In this way, the accuracy of the detection result obtained by using the text detection model to perform text detection on the image to be detected is high.

[0467] As an implementation of the embodiment of the present application, the device further includes:

[0468] A second search module searches, according to a second preset search target, for four candidate vertices corresponding to the region where the text is located among the edge points of the region where the text is located, wherein the second preset search target is that the quadrilateral formed by the searched four candidate vertices has the largest overlap area with the region where the text is located;

[0469] The region determination module is used to determine the quadrilateral formed by the four candidate vertices as the region where the text in the image to be detected is located.

[0470] As an implementation manner of the embodiment of the present application, the second search module includes:

[0471] A candidate point selection submodule is used to select four candidate points from the edge points of the area where the text is located;

[0472] A score calculation submodule, used to calculate the score of the quadrilateral formed by the four candidate points, wherein the score is determined based on the area and / or side length of the quadrilateral;

[0473] The candidate point updating submodule is used to traverse the candidate points in sequence, and when traversing to each candidate point, update the candidate vertex in the search range corresponding to the candidate point to the candidate point, wherein the candidate vertex is: the edge point of the quadrilateral formed by the other three candidate points except the candidate point, which has the highest score and is higher than the score, and the search range of the candidate point includes the edge points between the previous candidate point and the next candidate point of the candidate point;

[0474] The candidate vertex determination submodule is used to use the current four candidate points as the four candidate vertices corresponding to the area where the text is located after the traversal is completed.

[0475] As an implementation of an embodiment of the present application, the score calculation submodule includes:

[0476] The sixth calculation unit is used to calculate the score according to the following formula:

[0477] ;

[0478] in, For the score, is the area of ​​the quadrilateral formed by the four candidate points, is the perimeter of the quadrilateral formed by the four candidate points, is the total length of the diagonals of the quadrilateral formed by the four candidate points.

[0479] As an implementation of the embodiment of the present application, the device further includes:

[0480] The rectangle construction module is used to use the longer midline of the two midlines of the text area as the target midline of the rotated rectangle to be constructed, and to construct a rotated rectangle with the same area as the text area as the text area in the image to be detected.

[0481] As an implementation of the embodiment of the present application, the rectangular building module includes:

[0482] A midpoint determination submodule, used to use the two end points of the longer midline of the two midlines of the region where the text is located as the midpoints of the two short sides of the rotated rectangle to be constructed;

[0483] A length determination submodule, used to use the ratio of the area of ​​the region where the text is located to the length of the longer midline of the two midlines of the region where the text is located as the length of the two short sides of the rotated rectangle to be constructed;

[0484] A vertex determination submodule, used to use the longer midline of the two midlines of the region where the text is located as the target midline of the rotated rectangle to be constructed, and determine four vertices of the rotated rectangle according to the length of the short side and the midpoints of the two short sides, so that the two short sides of the rotated rectangle are perpendicular to the target midline;

[0485] The rectangle determination submodule is used to use the quadrilateral formed by the four vertices as the rotated rectangle corresponding to the area where the text is located.

[0486] As an implementation of an embodiment of the present application, the vertex determination submodule includes:

[0487] a direction vector determining unit, configured to use a direction vector perpendicular to a centerline vector of the target centerline as a direction vector of the short side, wherein the direction of the centerline vector is one of the extending directions of the target centerline;

[0488] a target vector determining unit, configured to obtain a target vector based on the direction vector and the length of the short side, wherein the direction of the target vector is the same as the direction vector, and the length of the target vector is half the length of the short side;

[0489] A first vertex determination unit, used for respectively calculating the sum of the midpoint of each short side and the target vector to obtain two vertices of the rotated rectangle;

[0490] The second vertex determination unit is used to calculate the difference between the midpoint of each short side and the target vector to obtain the other two vertices of the rotated rectangle.

[0491] The present application also provides an electronic device, such as Fig.11 As shown, including:

[0492] Memory 1101, used for storing computer programs;

[0493] The processor 1102 is used to implement the text detection module training method and / or the text detection method provided in the embodiment of the present application when executing the program stored in the memory 1101.

[0494] In addition, the electronic device may further include a communication bus and / or a communication interface, and the processor 1102, the communication interface, and the memory 1101 communicate with each other via the communication bus.

[0495] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0496] The communication interface is used for communication between the above electronic device and other devices.

[0497] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0498] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0499] In another embodiment provided in the present application, a computer-readable storage medium is also provided, in which a computer program is stored. When the computer program is executed by a processor, it implements any of the above-mentioned text detection module training methods and / or the steps of the text detection method.

[0500] In another embodiment provided in the present application, a computer program product comprising instructions is also provided, which, when executed on a computer, enables the computer to execute any text detection module training method and / or text detection method in the above-mentioned embodiments.

[0501] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a solid-state hard disk (SSD), etc.

[0502] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0503] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, electronic device, computer-readable storage medium, and computer program product embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0504] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A text detection model training method, characterized in that: The method comprises: For each text region in each training image, searching for the corresponding inward contraction distance of the text region within the inward contraction distance range corresponding to the text region according to a first preset search target, wherein the first preset search target is the inward contraction distance obtained by searching so that the area of ​​the outward expansion region is the same as the area of ​​the text region, and the outward expansion region is the region obtained by inward contracting the text region according to the inward contraction distance and then outward expanding the text region; Based on the shrinkage distance corresponding to each text area in each training image, shrinking each text area to obtain a shrinkage image of each training image; Expand each text region in each training image to obtain an expanded region corresponding to each text region; Based on the inner image of each training image and the outer expansion area corresponding to each text area in the training image, a segmentation image label, a threshold image label and a binary image label corresponding to the training image are generated; Each training image is input into the text detection model to be trained, and the text detection model is trained based on the differences between the predicted probability map, the predicted threshold map and the predicted binary map output by the text detection model and the segmentation map label, the threshold map label and the binary map label, respectively, to obtain a trained text detection model.

2. The method according to claim 1, characterized in that: The step of searching for the indentation distance corresponding to the text area in each training image according to the first preset search target within the indentation distance range corresponding to the text area comprises: For each text area in each training image, the indentation distance range is determined by taking 0 as the left boundary of the indentation distance range and taking the target value as the right boundary of the indentation distance range, wherein the target value is determined based on the perimeter of the text area; When the difference between the left boundary and the right boundary of the retracted distance range is greater than a preset accuracy threshold, determining an alternative retracted distance within the retracted distance range; The retraction distance range is updated based on the alternative retraction distance, and the step of determining the alternative retraction distance within the retraction distance range when the difference between the left boundary and the right boundary of the retraction distance range is greater than the preset accuracy threshold is returned to execute, until the difference between the left boundary and the right boundary of the current retraction distance range is not greater than the preset accuracy threshold, and the retraction distance corresponding to the text area is determined based on the current retraction distance range.

3. The method according to claim 2, characterized in that The step of updating the retraction distance range based on the candidate retraction distance comprises: In the case where the alternative inward contraction distance fails to be used for inward contraction, or the area of ​​the alternative outward expansion region obtained based on the alternative inward contraction distance is smaller than the area of ​​the text region, the alternative inward contraction distance is used as the right boundary of the inward contraction distance range to obtain an updated inward contraction distance range, wherein the area of ​​the alternative outward expansion region is the area of ​​the outward expansion region obtained based on the alternative inward contraction distance; In the case that the area of ​​the candidate outward expansion region is larger than the area of ​​the text region, the candidate inward contraction distance is used as the left boundary of the inward contraction distance range to obtain an updated inward contraction distance range.

4. The method according to claim 3, characterized in that The step of determining the indentation distance corresponding to the text area based on the current indentation distance range includes: If the right boundary of the current shrinking distance range fails to shrink, or the absolute value of the first area difference is greater than the absolute value of the second area difference, the left boundary of the current shrinking distance range is returned as the shrinking distance corresponding to the text area, wherein the first area difference is the difference between the area of ​​the expanded area obtained based on the right boundary and the area of ​​the text area, and the second area difference is the difference between the area of ​​the expanded area obtained based on the left boundary and the area of ​​the text area; When the absolute value of the first area difference is not greater than the absolute value of the second area difference, the right boundary of the current indentation distance range is returned as the indentation distance corresponding to the text area.

5. The method according to any one of claims 1 to 4, characterized in that: The step of inputting each training image into a text detection model to be trained, and training the text detection model based on the differences between the predicted probability map, the predicted threshold map, and the predicted binary map output by the text detection model and the segmentation map label, the threshold map label, and the binary map label, to obtain a trained text detection model comprises: Input each training image into the text detection model to be trained to obtain a prediction probability map, a prediction threshold map, and a prediction binary map; Based on the segmentation map label, the threshold map label, the binary map label, the predicted probability map, the predicted threshold map, the predicted binary map and a preset function, calculating the loss value corresponding to the training image, wherein the preset function includes a cross entropy loss function, a DICE loss function and a pre-constructed intersection-over-union loss function, and the intersection-over-union loss function is only used to calculate the difference between the text area shape in the training image predicted by the text detection model and the text area shape represented by the segmentation map label corresponding to the training image; The parameters of the text detection model are adjusted according to the loss value until the text detection model converges to obtain the text detection model.

6. The method according to claim 5, characterized in that The step of calculating the loss value corresponding to the training image based on the segmentation map label, the threshold map label, the binary map label, the predicted probability map, the predicted threshold map, the predicted binary map and the preset function comprises: Based on the segmentation map label, the prediction probability map and the cross entropy loss function, calculating a first cross entropy loss value corresponding to the training image; Based on the binary image label, the predicted binary image and the cross entropy loss function, calculating a second cross entropy loss value corresponding to the training image; Calculating a DICE loss value corresponding to the training image based on the threshold map label, the predicted threshold map, and the DICE loss function; Based on the segmentation map label, the predicted probability map and the pre-constructed intersection-and-union loss function, a first intersection-and-union loss value corresponding to the training image is calculated; and / or, based on the binary map label, the predicted binary map and the pre-constructed intersection-and-union loss function, a second intersection-and-union loss value corresponding to the training image is calculated; Calculate the first cross entropy loss value, the second cross entropy loss value, the DICE loss value, the first intersection-over-union loss value, and / or the sum of the second intersection-over-union loss value to obtain a loss value corresponding to the training image.

7. The method according to claim 6, characterized in that The step of calculating a first intersection-and-union loss value corresponding to the training image based on the segmentation map label, the predicted probability map, and a pre-constructed intersection-and-union loss function; and / or the step of calculating a second intersection-and-union loss value corresponding to the training image based on the binary map label, the predicted binary map, and a pre-constructed intersection-and-union loss function, comprises: The first intersection-over-combination loss value and / or the second intersection-over-combination loss value are calculated according to the following formula: ; in, is the intersection-and-union loss value; is a pixel point in the prediction image output by the text detection model, is the pixel point in the label corresponding to the prediction image, is the length of the prediction graph, is the width of the prediction graph, For a pixel point in the prediction probability map, is the pixel point in the segmentation map label, For the pixel point in the predicted binary image, is a pixel point in the binary image label; for The weight of the region, If the area is a non-text area, ,exist If the area is a text area, , is the area of ​​the text region to which the pixel belongs.

8. A text detection method, characterized in that: The method comprises: Acquire the image to be detected; Inputting the image to be detected into a pre-trained text detection model to obtain a binary image of the image to be detected output by the text detection model, wherein the text detection model is trained by the method according to any one of claims 1 to 7; A connected area in the binary image is determined to obtain a text prediction box where the text in the image to be detected is located, and the text prediction box is expanded to obtain a region where the text in the image to be detected is located.

9. The method according to claim 8, characterized in that The method further comprises: According to a second preset search target, searching for four target points corresponding to the region where the text is located among the edge points of the region where the text is located, wherein the second preset search target is that the quadrilateral formed by the searched four target points has the largest overlap area with the region where the text is located; The quadrilateral formed by the four target points is determined as the area where the text in the image to be detected is located.

10. The method according to claim 9, characterized in that The step of searching, according to the second preset search target, four target points corresponding to the region where the text is located among the edge points of the region where the text is located comprises: Select four candidate points from the edge points of the region where the text is located; Calculating a score of a quadrilateral formed by the four candidate points, wherein the score is determined based on the area and / or side length of the quadrilateral; Traversing each candidate point in turn, when traversing to each candidate point, updating the target point in the search range corresponding to the candidate point to the candidate point, wherein the target point is: an edge point of a quadrilateral formed by the other three candidate points except the candidate point, which has the highest score and is higher than the score, and the search range of the candidate point includes each edge point between the candidate point before the candidate point and the candidate point after the candidate point; After the traversal is completed, the current four candidate points are used as four target points corresponding to the area where the text is located.

11. The method according to claim 10, characterized in that The step of calculating the score of the quadrilateral formed by the four candidate points comprises: The score is calculated according to the following formula: ; in, For the score, is the area of ​​the quadrilateral formed by the four candidate points, is the perimeter of the quadrilateral formed by the four candidate points, is the total length of the diagonals of the quadrilateral formed by the four candidate points.

12. The method according to any one of claims 8 to 11, characterized in that: The method further comprises: The longer midline of the two midlines of the text region is used as the target midline of the rotated rectangle to be constructed, and a rotated rectangle with the same area as the text region is constructed as the text region in the image to be detected.

13. The method according to claim 12, characterized in that The step of taking the longer midline of the two midlines of the region where the text is located as the target midline of the rotated rectangle to be constructed and constructing a rotated rectangle with the same area as the region where the text is located comprises: The two end points of the longer midline of the two midlines of the region where the text is located are respectively used as the midpoints of the two short sides of the rotated rectangle to be constructed; The ratio of the area of ​​the region where the text is located to the length of the longer midline of the two midlines of the region where the text is located is used as the length of the two short sides of the rotated rectangle to be constructed; The longer midline of the two midlines of the region where the text is located is used as the target midline of the rotated rectangle to be constructed, and four vertices of the rotated rectangle are determined according to the length of the short side and the midpoints of the two short sides, so that the two short sides of the rotated rectangle are perpendicular to the target midline; The quadrilateral formed by the four vertices is used as the rotated rectangle corresponding to the area where the text is located.

14. The method according to claim 13, characterized in that The step of taking the longer midline of the two midlines of the region where the text is located as the target midline of the rotated rectangle to be constructed, and determining the four vertices of the rotated rectangle according to the length of the short side and the midpoints of the two short sides, comprises: Using a direction vector perpendicular to a centerline vector of the target centerline as a direction vector of the short side, wherein the direction of the centerline vector is one of the extension directions of the target centerline; Based on the direction vector and the length of the short side, a target vector is obtained, wherein the direction of the target vector is the same as the direction vector, and the length of the target vector is half of the length of the short side; Calculate the sum of the midpoint of each short side and the target vector respectively to obtain two vertices of the rotated rectangle; The difference between the midpoint of each short side and the target vector is calculated respectively to obtain the other two vertices of the rotated rectangle.

15. A text detection model training device, characterized in that: The device comprises: A first search module is used to search for the corresponding indentation distance of the text region within the indentation distance range corresponding to the text region according to a first preset search target for each text region in each training image, wherein the first preset search target is the indentation distance obtained by searching so that the area of ​​the expanded region is the same as the area of ​​the text region, and the expanded region is the region obtained by shrinking the text region according to the indentation distance and then expanding it; A shrinking module, used to shrink each text area in each training image based on the shrinking distance corresponding to each text area, so as to obtain a shrinking image of each training image; An expansion module is used to expand each text area in each training image to obtain an expansion area corresponding to each text area; A label generation module, used to generate a segmentation map label, a threshold map label and a binary map label corresponding to each training image based on the inner image of each training image and the outer expansion area corresponding to each text area in the training image; A training module is used to input each training image into a text detection model to be trained, and to train the text detection model based on the differences between the predicted probability map, the predicted threshold map, and the predicted binary map output by the text detection model and the segmentation map label, the threshold map label, and the binary map label, so as to obtain a trained text detection model.

16. A text detection model, characterized in that: The model includes: An image acquisition module, used for acquiring an image to be detected; A binary image acquisition module, used for inputting the image to be detected into a pre-trained text detection model, and acquiring a binary image of the image to be detected output by the text detection model, wherein the text detection model is trained by the method according to any one of claims 1 to 7; The region determination module is used to determine the connected region in the binary image, obtain the text prediction box where the text in the image to be detected is located, and expand the text prediction box to obtain the region where the text in the image to be detected is located.

17. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, for implementing the method described in any one of claims 1-7 or any one of claims 8-14 when executing a program stored in a memory.

18. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any one of claims 1 to 7 or any one of claims 8 to 14 is implemented.

Citation Information

Patent Citations

  • Method for detecting texts in any shape in natural scene

    CN111444919A

  • Method for Recognizing Text, Apparatus and Terminal Device

    US20230154217A1

  • End-to-end recognition method for scene text in any shape

    WO2019192397A1

  • Text detection method and apparatus in scene image, computer device, and storage medium

    WO2021189889A1