Method and apparatus for detecting text

Abstract

The embodiment of the present application provides a kind of method and device for detecting text, belong to computer vision field.The method comprises: the image of text to be detected is input into preset convolutional neural network structure, to obtain the candidate frame information of each candidate frame in multiple candidate frames that image is split into;For any candidate frame in multiple candidate frames, based on the left probability, right probability and other probability of the candidate frame, determine whether the candidate frame is located at left boundary, right boundary or other position, to determine whether the candidate frame is left frame, right frame or other frame;And for any left frame, determine the matching right frame according to the basic parameter of left frame;And based on the basic parameter of left frame and the basic parameter of matching right frame, determine a line text region detection frame.Thereby, realize to reduce time-consuming phenomenon, improve the calculation speed.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision, in particular to a method and device for detecting text. BACKGROUND

[0002] Natural scene text detection is an important direction in the field of computer vision. The goal of text detection model is to find out the region of text as accurately as possible from the picture. However, the effect of common target detection method (SSD, YOLO, Faster-RCNN, etc.) directly used for text detection task is not ideal, because compared with the conventional object, the length of text line and the range of length-width ratio change greatly. In order to solve this problem, the common method is to regard the text line as a sequence, rather than a single complete independent target; by splitting the text sequence into many small text boxes for detection respectively, and then merging the small text boxes belonging to the same text line, a complete and large text box is obtained. The previous method (CTPN, Detecting Text in Natural Image with Connectionist Text Proposal Network, etc.) usually needs to predict many candidate boxes, and then merge the overlapping candidate boxes one by one, which is very time-consuming and has a large amount of parameters. Moreover, in natural scene, the text line may not be horizontal, that is, it has a certain angle, and the conventional detection box bbox(x, y, w, h) cannot accurately detect the position of the text line. SUMMARY

[0003] The purpose of the embodiments of the present application is to provide a method and device for detecting text, which can realize solving or at least partially solving the above problems.

[0004] To achieve the above object, one aspect of an embodiment of the present application provides a method for detecting text, the method comprising: inputting an image of text to be detected into a preset convolutional neural network structure to obtain candidate box information of each candidate box in which the image is split, wherein for any candidate box in the candidate boxes, the candidate box information comprises a left probability of the candidate box, a right probability of the candidate box, an other probability of the candidate box, and a basic parameter of the candidate box, wherein the left probability is a probability that the candidate box is located at a left boundary of a text region to be detected in the image, the right probability is a probability that the candidate box is located at a right boundary of the text region to be detected, and the other probability is a probability that the candidate box is located at a position other than the left boundary and the right boundary in the text region to be detected; for any candidate box in the candidate boxes, determining whether the candidate box is located at the left boundary, the right boundary or the other position based on the left probability, the right probability and the other probability of the candidate box to determine whether the candidate box is a left edge box, a right edge box or an other edge box; for any left edge box, determining a paired right edge box according to the basic parameter of the left edge box; and determining a text region detection box for a line of text based on the basic parameter of the left edge box and the basic parameter of the paired right edge box, the text region detection box being used to frame all the text in a line from the left edge box to the paired right edge box in the image.

[0005] Optionally, the basic parameter of the candidate box comprises a coordinate position of a center point of the candidate box, and a length and a width of the candidate box, and the determining of the text region detection box for a line of text based on the basic parameter of the left edge box and the basic parameter of the paired right edge box comprises: determining the text region detection box based on a left center point, a right center point, the length and the width of the left edge box, and the length and the width of the paired right edge box, wherein the left center point is the center point of the left edge box, and the right center point is the center point of the paired right edge box.

[0006] Optionally, for any left edge box, a first line segment is determined based on the left center point and the right center point, a second line segment is obtained by extending a half of the length of the left edge box to the left side of the first line segment and extending a half of the length of the paired right edge box to the right side of the first line segment, the second line segment is taken as a center line of the text region detection box for a line of text, the length of the text region detection box is the length of the second line segment, and the width of the text region detection box is an average of the width of the left edge box and the width of the paired right edge box.

[0007] Optionally, the base parameters of the candidate boxes include coordinate positions of center points of the candidate boxes, and determining the paired right box for any left box includes: calculating distances between the coordinate positions of the center points of each right box in all the right boxes and the coordinate position of the center point of the left box, respectively; and determining a minimum distance value among all the distance values by comparison, wherein the right box corresponding to the minimum distance value is the paired right box.

[0008] Optionally, for any candidate box in the plurality of candidate boxes, the base parameters further include an angle of the candidate box, the angle being an included angle between an upper boundary of the candidate box and a horizontal line of the image and the positive or negative of the angle depending on a rotation direction of the upper boundary of the candidate box relative to the horizontal line of the image; and the paired right box for any left box further satisfies a condition that an absolute value of a difference between the angle of the paired right box and the angle of the left box is less than a preset angle value.

[0009] Optionally, feature maps output by a preset number of convolution layers adjacent to each other in the preset convolutional neural network structure are fused in a channel dimension, there is no pooling layer between any two adjacent convolution layers among the preset number of convolution layers being fused, an output layer in the preset convolutional neural network structure is a convolution layer, and the feature map output by the output layer corresponds to the left probability, the right probability, the other probability and the base parameters.

[0010] Optionally, for any candidate box in the plurality of candidate boxes, the base parameters include a coordinate position of a center point of the candidate box, a length, a width and an angle of the candidate box, the angle being an included angle between an upper boundary of the candidate box and a horizontal line of the image and the positive or negative of the angle depending on a rotation direction of the upper boundary of the candidate box relative to the horizontal line of the image, and an output layer of the preset convolutional neural network structure includes a first convolutional layer and a second convolutional layer in parallel, wherein the feature map output by the first convolutional layer corresponds to the left probability, the right probability and the other probability, and the feature map output by the second convolutional layer corresponds to the coordinate position, the length, the width and the angle.

[0011] Optionally, a loss function used when the preset convolutional neural network structure is trained is: wherein L1 is a loss function of the first convolutional layer, L2 is a loss function of the second convolutional layer, f corresponds to the feature map output by the first convolutional layer, f ijk represents a probability of a kth dimension of a candidate box in an i th row and a j th column in the feature map output by the first convolutional layer, P ijk represents a preset probability of the kth dimension of the candidate box in the i th row and the j th column in the feature map output by the first convolutional layer, x ijThe x-coordinate of the center point of the candidate box in the i-th row and j-th column of the feature map output by the second convolutional layer is equal in size to i. ij This represents the ordinate of the center point of the candidate box in the i-th row and j-th column of the feature map output by the second convolutional layer, and its size is equal to j. This represents the preset x-coordinate of the center point of the candidate box in the i-th row and j-th column of the feature map output by the second convolutional layer. Δx represents the preset ordinate of the center point of the candidate box in the i-th row and j-th column of the feature map output by the second convolutional layer. ij Δy represents the correction value for the x-coordinate of the center point of the candidate box in the i-th row and j-th column of the feature map output by the second convolutional layer. ij w represents the correction value of the ordinate of the center point of the candidate box in the i-th row and j-th column of the feature map output by the second convolutional layer. ij This represents the length of the candidate box in the i-th row and j-th column of the feature map output by the second convolutional layer. Δw represents the preset length of the candidate box in the i-th row and j-th column of the feature map output by the second convolutional layer. ij h represents the correction value for the length of the candidate box in the i-th row and j-th column of the feature map output by the second convolutional layer. ij This represents the width of the candidate box in the i-th row and j-th column of the feature map output by the second convolutional layer. Δh represents the preset width of the candidate box in the i-th row and j-th column of the feature map output by the second convolutional layer. ij θ represents the correction value for the width of the candidate box in the i-th row and j-th column of the feature map output by the second convolutional layer. ij This represents the angle of the candidate box in the i-th row and j-th column of the feature map output by the second convolutional layer. The angle represents the preset angle of the candidate box in the i-th row and j-th column of the feature map output by the second convolutional layer. The feature maps output by the first convolutional layer and the second convolutional layer each have M*N candidate boxes.

[0012] Correspondingly, another aspect of the embodiment of the present application provides a device for detecting text, the device comprising: a candidate box information obtaining module configured to input an image of text to be detected into a preset convolutional neural network structure to obtain candidate box information of each candidate box in a plurality of candidate boxes into which the image is split, wherein for any candidate box in the plurality of candidate boxes, the candidate box information comprises a left side probability of the candidate box, a right side probability of the candidate box, an other probability of the candidate box, and a basic parameter of the candidate box, wherein the left side probability is a probability that the candidate box is located at a left boundary of a text region to be detected in the image, the right side probability is a probability that the candidate box is located at a right boundary of the text region to be detected, and the other probability is a probability that the candidate box is located at a position other than the left boundary and the right boundary in the text region to be detected; a side frame determining module configured to, for any candidate box in the plurality of candidate boxes, determine, based on the left side probability, the right side probability and the other probability of the candidate box, whether the candidate box is located at the left boundary, the right boundary or the position other than the left boundary and the right boundary, to determine whether the candidate box is a left side frame, a right side frame or an other side frame; and a line text region detection frame determining module configured to, for any left side frame, determine a matching right side frame according to the basic parameter of the left side frame, and determine a line text region detection frame based on the basic parameter of the left side frame and the basic parameter of the matching right side frame, the line text region detection frame being used to frame all characters in a line from the left side frame to the matching right side frame in the image.

[0013] Optionally, the basic parameter of the candidate box comprises coordinate positions of a center point of the candidate box, and a length and a width of the candidate box, and the line text region detection frame determining module, for any left side frame, determines the line text region detection frame based on the basic parameter of the left side frame and the basic parameter of the matching right side frame comprises: determining the line text region detection frame based on a left side center point, a right side center point, the length and the width of the left side frame, and the length and the width of the matching right side frame, wherein the left side center point is a center point of the left side frame, and the right side center point is a center point of the matching right side frame.

[0014] Optionally, for any left side frame, a first line segment is determined based on the left side center point and the right side center point, a second line segment is obtained by extending a half of the length of the left side frame to the left side of the first line segment and extending a half of the length of the matching right side frame to the right side of the first line segment, the second line segment is taken as a center line of the line text region detection frame, the length of the line text region detection frame is the length of the second line segment, and the width of the line text region detection frame is an average of the width of the left side frame and the width of the matching right side frame.

[0015] Optionally, the base parameters of the candidate boxes include coordinate positions of center points of the candidate boxes, and determining the matching right box for any left box includes: calculating distances between the coordinate positions of the center points of each right box and the coordinate position of the center point of the left box, respectively; and determining a minimum distance value among all the distance values by comparison, wherein the right box corresponding to the minimum distance value is the matching right box.

[0016] Optionally, for any candidate box in the plurality of candidate boxes, the base parameters further include an angle of the candidate box, the angle being an angle between an upper boundary of the candidate box and a horizontal line of the image and the positive or negative of the angle depending on a rotation direction of the upper boundary of the candidate box relative to the horizontal line of the image; and for any left box, the matching right box further satisfies a condition that an absolute value of a difference between the angle of the matching right box and the angle of the left box is less than a preset angle value.

[0017] Optionally, feature maps output by a preset number of convolution layers adjacent to each other in the preset convolutional neural network structure are fused in a channel dimension, there is no pooling layer between any two adjacent convolution layers among the preset number of convolution layers being fused, an output layer in the preset convolutional neural network structure is a convolution layer, and the feature map output by the output layer corresponds to the left probability, the right probability, the other probability and the base parameters.

[0018] Optionally, for any candidate box in the plurality of candidate boxes, the base parameters include a coordinate position of a center point of the candidate box, a length, a width and an angle of the candidate box, the angle being an angle between an upper boundary of the candidate box and a horizontal line of the image and the positive or negative of the angle depending on a rotation direction of the upper boundary of the candidate box relative to the horizontal line of the image, and an output layer of the preset convolutional neural network structure includes a first convolution layer and a second convolution layer in parallel, wherein the feature map output by the first convolution layer corresponds to the left probability, the right probability and the other probability, and the feature map output by the second convolution layer corresponds to the coordinate position, the length, the width and the angle.

[0019] Optionally, a loss function used when the preset convolutional neural network structure is trained is:

[0020]

[0021] wherein L1 is a loss function of the first convolution layer, L2 is a loss function of the second convolution layer, f corresponds to the feature map output by the first convolution layer, f corresponds to the feature map output by the second convolution layer, and the loss function is: ijka probability of a kth dimension of a candidate box in an ith row and jth column of a feature map output by the first convolutional layer, P ijk a preset probability of a kth dimension of a candidate box in an ith row and jth column of a feature map output by the first convolutional layer, x ij a horizontal coordinate of a center point of a candidate box in an ith row and jth column of a feature map output by the second convolutional layer, and the size is equal to i, y ij a vertical coordinate of a center point of a candidate box in an ith row and jth column of a feature map output by the second convolutional layer, and the size is equal to j, a preset horizontal coordinate of a center point of a candidate box in an ith row and jth column of a feature map output by the second convolutional layer, a preset vertical coordinate of a center point of a candidate box in an ith row and jth column of a feature map output by the second convolutional layer, Δx ij a correction value of a horizontal coordinate of a center point of a candidate box in an ith row and jth column of a feature map output by the second convolutional layer, Δy ij a correction value of a vertical coordinate of a center point of a candidate box in an ith row and jth column of a feature map output by the second convolutional layer, w ij a length of a candidate box in an ith row and jth column of a feature map output by the second convolutional layer, a preset length of a candidate box in an ith row and jth column of a feature map output by the second convolutional layer, Δw ij a correction value of a length of a candidate box in an ith row and jth column of a feature map output by the second convolutional layer, h ij a width of a candidate box in an ith row and jth column of a feature map output by the second convolutional layer, a preset width of a candidate box in an ith row and jth column of a feature map output by the second convolutional layer, Δh ij a correction value of a width of a candidate box in an ith row and jth column of a feature map output by the second convolutional layer, θ ij an angle of a candidate box in an ith row and jth column of a feature map output by the second convolutional layer, a preset angle of a candidate box in an ith row and jth column of a feature map output by the second convolutional layer, the feature maps output by the first convolutional layer and the second convolutional layer respectively have M*N candidate boxes.

[0022] In addition, another aspect of the embodiment of the present application further provides a machine readable storage medium, which stores instructions for causing a machine to execute the above method.

[0023] In addition, another aspect of the embodiment of the present application further provides a computer program product, which comprises computer programs / instructions, and the computer programs / instructions are executed by a processor to implement the above method.

[0024] By the technical solution, the text region detection frame can be determined based on the determined left frame and right frame, without other frames of the text region in the image of the text to be detected except the left frame and the right frame, so that the parameters to be calculated are reduced, the calculation amount is reduced, the time is saved, the time-consuming phenomenon is reduced, and the calculation speed is improved.

[0025] Other features and advantages of the embodiments of the present application will be described in detail in the following detailed description. BRIEF DESCRIPTION OF DRAWINGS

[0026] The accompanying drawings are included to provide a further understanding of the embodiments of the present application, and constitute a part of the specification, and are used to explain the embodiments of the present application together with the following detailed description, but do not constitute a limitation of the embodiments of the present application. In the drawings:

[0027] Figure 1 is a flowchart of a method for detecting text provided by an embodiment of the present application;

[0028] Figure 2 is a structural diagram of a preset convolutional neural network structure provided by another embodiment of the present application; and

[0029] Figure 3 is a structural block diagram of an apparatus for detecting text provided by another embodiment of the present application.

[0030] REFERENCE NUMERALS

[0031] 1 candidate frame information obtaining module 2 frame determining module

[0032] 3 line text region detection frame determining module DETAILED DESCRIPTION

[0033] The specific embodiments of the embodiments of the present application are described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to illustrate and explain the embodiments of the present application, and are not used to limit the embodiments of the present application.

[0034] In the present application, unless otherwise stated, the orientation words such as "up, down, left, right" generally refer to the division based on the center point of the image or the candidate frame, the left boundary refers to the boundary on the left side of the center point, the right boundary refers to the boundary on the right side of the center point, the upper boundary refers to the boundary on the upper side of the center point, the left frame refers to the candidate frame located at the left boundary of the text region to be detected in the image, and the right frame refers to the candidate frame located at the right boundary of the text region to be detected in the image.

[0035] One aspect of the embodiments of the present application provides a method for detecting text.

[0036] Figure 1 is a flowchart of a method for detecting text provided by an embodiment of the present application. As shown in Figure 1 , the method comprises the following.

[0037] In step S10, an image of text to be detected is input into a preset convolutional neural network structure to obtain candidate box information of each candidate box in a plurality of candidate boxes into which the image is split, wherein for any candidate box in the plurality of candidate boxes, the candidate box information comprises a left probability of the candidate box, a right probability of the candidate box, an other probability of the candidate box, and a basic parameter of the candidate box, wherein the left probability is a probability that the candidate box is located at a left boundary of a text-to-be-detected region in the image, the right probability is a probability that the candidate box is located at a right boundary of the text-to-be-detected region, the other probability is a probability that the candidate box is located at a position other than the left boundary and the right boundary in the text-to-be-detected region, and the text-to-be-detected region is a region covered by the text to be detected in the image.

[0038] In step S11, for any candidate box in the plurality of candidate boxes, based on the left probability, the right probability and the other probability of the candidate box, it is determined whether the candidate box is located at the left boundary, the right boundary or the other position, so as to determine whether the candidate box is a left frame, a right frame or an other frame. For example, for any candidate box, the left frame, the right frame or the other frame is determined according to the maximum of the left probability, the right probability and the other probability. For example, for a certain candidate box, the maximum of the left probability, the right probability and the other probability is the left probability, and it is determined that the candidate box is located at the left boundary of the text-to-be-detected region, and the candidate box is a left frame.

[0039] In step S12, for any left bounding box, a paired right bounding box is determined according to the basic parameters of the left bounding box, and the paired right bounding box is the right bounding box closest to the left bounding box. For example, the paired right bounding box can be determined by calculating the distance between the coordinate positions. Alternatively, for any candidate bounding box in the plurality of candidate bounding boxes, the basic parameters include the coordinate position of the center point of the candidate bounding box, i.e., the coordinate position of the center point of the candidate bounding box in the image. For any left bounding box, the paired right bounding box can be determined according to the following. For any left bounding box, the distance value between the coordinate position of the center point of each right bounding box in all right bounding boxes and the coordinate position of the center point of the left bounding box is calculated respectively. Wherein, all right bounding boxes are all right bounding boxes determined in step S11. The minimum distance value in all distance values is determined by comparison, and the right bounding box corresponding to the minimum distance value is the paired right bounding box. By comparing the sizes of all distance values, the smallest distance value is the minimum distance value, and the right bounding box corresponding to the minimum distance value is the paired right bounding box. Alternatively, in the embodiment of the present application, a coordinate system is established based on the image, and the coordinate position of the center point of the candidate bounding box is the coordinate position of the center point of the candidate bounding box in the image.

[0040] In step S13, for any left bounding box, a text line region bounding box is determined based on the basic parameters of the left bounding box and the basic parameters of the paired right bounding box, and the text line region bounding box is used to frame all the text in a line from the left bounding box to the paired right bounding box in the image. Alternatively, for any left bounding box, the text line region bounding box can be determined based on the left center point and the right center point. Wherein, the left center point is the center point of the left bounding box, and the right center point is the center point of the paired right bounding box. In addition, other reference points of the left bounding box and other reference points of the paired right bounding box can also be determined, for example, the center point of the upper boundary of the left bounding box or the center point of the lower boundary of the left bounding box, the center point of the upper boundary of the paired right bounding box or the center point of the lower boundary of the paired right bounding box, etc., and the text line region bounding box is determined based on the determined other reference points of the left bounding box and the other reference points of the paired right bounding box.

[0041] Through the above technical solution, the text line region bounding box can be determined based on the determined left bounding box and right bounding box, without relying on other bounding boxes of the text region to be detected in the image of the text to be detected, reducing the parameters to be calculated, reducing the calculation amount, saving time, reducing the time-consuming phenomenon, and improving the calculation speed.

[0042] Optionally, in the embodiment of the present application, the basic parameters of the candidate box include the coordinate position of the center point of the candidate box and the length and width of the candidate box, and the determining of the line text region detection box based on the basic parameters of the left box and the basic parameters of the matched right box includes: determining the line text region detection box based on the left center point, the right center point, the length and width of the left box and the length and width of the matched right box, wherein the left center point is the center point of the left box and the right center point is the center point of the matched right box. Optionally, in the embodiment of the present application, for any left box, a first line segment is determined based on the left center point and the right center point, the left half of the length of the left box is extended to the left side of the first line segment, and the left half of the length of the matched right box is extended to the right side of the first line segment to obtain a second line segment, the second line segment is taken as the center line of the line text region detection box, the length of the line text region detection box is the length of the second line segment, and the width of the line text region detection box is the average of the width of the left box and the width of the matched right box. Specifically, for any left box, the left center point and the right center point are connected to form a line to obtain a center point connecting line, that is, the first line segment is determined; the center point connecting line is extended outward according to the length of the left box and the length of the matched right box, the left half of the length of the left box is extended outward from the left center point, and the left half of the length of the matched right box is extended outward from the right center point to obtain the second line segment; then the average of the width of the left box and the width of the matched right box is taken as the width of the text region, and the complete line text region detection box is obtained. In addition, in the embodiment of the present application, for any left box, the length of the line text region detection box can also be the length of the left center point and the right center point connecting line plus the length of the left box or the length of the matched right box. In addition, in the embodiment of the present application, for any left box, the width of the line text region detection box can also be the width of the left box or the width of the right box.

[0043] Optionally, in the embodiment of the present application, the angle problem is also considered when determining the right bounding box matched with the left bounding box, so as to accurately determine the inclined text line. Specifically, for any candidate bounding box in the plurality of candidate bounding boxes, the basic parameters further include an angle of the candidate bounding box, the angle being an angle between an upper boundary of the candidate bounding box and a horizontal line of the image, and the positive or negative of the angle depending on a rotation direction of the upper boundary of the candidate bounding box relative to the horizontal line of the image. For example, it can be set that, for any candidate bounding box, when the rotation direction of the upper boundary of the candidate bounding box relative to the horizontal line of the image is counterclockwise rotation, the angle is positive; and when the rotation direction of the upper boundary of the candidate bounding box relative to the horizontal line of the image is clockwise rotation, the angle is negative. Alternatively, for any candidate bounding box, when the rotation direction of the upper boundary of the candidate bounding box relative to the horizontal line of the image is clockwise rotation, the angle is positive; and when the rotation direction of the upper boundary of the candidate bounding box relative to the horizontal line of the image is counterclockwise rotation, the angle is negative. For any left bounding box, the right bounding box matched with the left bounding box further satisfies the following condition: an absolute value of a difference between the angle of the right bounding box and the angle of the left bounding box is less than a preset angle value. By adding the feature of the angle when detecting the text, the text line detection at any angle in a natural scene is realized.

[0044] Optionally, in the embodiment of the present application, the feature maps output by the preset number of convolution layers adjacent to each other in the preset convolutional neural network structure are fused in the channel dimension, there is no pooling layer between any two adjacent convolution layers in the preset number of convolution layers being fused, the output layer in the preset convolutional neural network structure is a convolution layer, and the feature map output by the output layer corresponds to the left probability, the right probability, the other probability and the basic parameters. By fusing the convolution layers, the preset convolutional neural network structure considers different receptive fields when learning the left probability, the right probability and the other probability, so that the left probability, the right probability and the other probability determined by the preset convolutional neural network structure are more accurate. Optionally, in the embodiment of the present application, the preset number and / or the positions of the preset number of convolution layers adjacent to each other in the preset convolutional neural network structure being fused can be set according to specific circumstances.

[0045] Optionally, in the embodiment of the present application, for any candidate bounding box in the plurality of candidate bounding boxes, the basic parameters include a coordinate position of a center point of the candidate bounding box, a length and a width of the candidate bounding box and an angle of the candidate bounding box, the angle being an angle between an upper boundary of the candidate bounding box and a horizontal line of the image, and the positive or negative of the angle depending on a rotation direction of the upper boundary of the candidate bounding box relative to the horizontal line of the image, and the output layer of the preset convolutional neural network structure includes a first convolution layer and a second convolution layer in parallel, wherein the feature map output by the first convolution layer corresponds to the left probability, the right probability and the other probability, and the feature map output by the second convolution layer corresponds to the coordinate position, the length, the width and the angle.

[0046] Optionally, in the embodiment of the present application, the loss function used when the preset convolutional neural network structure is trained is: L1 is a loss function of the first convolutional layer, L2 is a loss function of the second convolutional layer, f corresponds to a feature map output by the first convolutional layer, f ijk Pij k represents a probability of the kth dimension of a candidate frame in the ith row and jth column of a feature map output by the first convolutional layer, P ijk xij k represents a preset probability of the kth dimension of a candidate frame in the ith row and jth column of a feature map output by the first convolutional layer, x ij xi j represents a horizontal coordinate of a center point of a candidate frame in the ith row and jth column of a feature map output by the second convolutional layer and has a size equal to i, y ij yij represents a vertical coordinate of the center point of the candidate frame in the ith row and jth column of the feature map output by the second convolutional layer and has a size equal to j, xi j represents a preset horizontal coordinate of the center point of the candidate frame in the ith row and jth column of the feature map output by the second convolutional layer, x yij represents a preset vertical coordinate of the center point of the candidate frame in the ith row and jth column of the feature map output by the second convolutional layer, y ij Δxi j represents a correction value of the horizontal coordinate of the center point of the candidate frame in the ith row and jth column of the feature map output by the second convolutional layer, Δx ij Δyij represents a correction value of the vertical coordinate of the center point of the candidate frame in the ith row and jth column of the feature map output by the second convolutional layer, y ij wi j represents a length of a candidate frame in the ith row and jth column of a feature map output by the second convolutional layer, w wi j represents a preset length of the candidate frame in the ith row and jth column of the feature map output by the second convolutional layer, w ij Δwi j represents a correction value of the length of the candidate frame in the ith row and jth column of the feature map output by the second convolutional layer, w ij hi j represents a width of a candidate frame in the ith row and jth column of a feature map output by the second convolutional layer, h hi j represents a preset width of the candidate frame in the ith row and jth column of the feature map output by the second convolutional layer, h ij Δhi j represents a correction value of the width of the candidate frame in the ith row and jth column of the feature map output by the second convolutional layer, h ij θij represents an angle of a candidate frame in the ith row and jth column of a feature map output by the second convolutional layer, θ θij represents a preset angle of the candidate frame in the ith row and jth column of the feature map output by the second convolutional layer, the feature map output by the first convolutional layer and the feature map output by the second convolutional layer each have M*N candidate frames. In the embodiment of the present application, the feature map output by the convolutional layer has M*N candidate frames, that is, the candidate frames in the feature map include M rows and N columns; a coordinate system is established based on an image, a row and a column determine a specific candidate frame, thus, in the embodiment of the present application, a row corresponds to a horizontal coordinate of a center point of a candidate frame, a column corresponds to a vertical coordinate of the center point of the candidate frame, and a row and a column represent a coordinate position of the center point of the candidate frame.

[0047] Optionally, in this embodiment of the invention, the preset convolutional neural network structure can be any convolutional neural network structure that satisfies the limitations of the preset convolutional neural network structure in this embodiment of the invention. The following uses the ResNet 50 convolutional neural network as an example to illustrate this embodiment. Compared to the traditional ResNet 50 convolutional neural network, the ResNet 50 convolutional neural network used in this embodiment of the invention has undergone some improvements, see [link to relevant documentation]. Figure 2 The structural diagram shown is as follows. Figure 2 As shown, the image of the text to be detected is input into the ResNet 50 convolutional neural network. The image is input from convolutional layer 1 into the ResNet 50 convolutional neural network. Fusion is performed at the second set of convolutional layers. The feature maps output from convolutional layers 2-1, 2-2, and 2-3 are concatenated along the channel dimension. There are no pooling layers between any two adjacent convolutional layers in convolutional layers 2-1, 2-2, and 2-3. The fused feature map is then input into subsequent convolutional layers and other layers of the ResNet 50 convolutional neural network. Convolutional layer 5-3 is the last convolutional layer of the ResNet 50 convolutional neural network before any improvements, and the feature map output by convolutional layer 5-3 is denoted as f(x,y)∈R. M *N*2048 Here, x and y represent the coordinates of the center point of the candidate box in the feature map. Before the improvement, in the ResNet50 convolutional neural network, convolutional layer 5-3 was followed by a fully connected layer. After the improvement, the fully connected layer of the ResNet50 convolutional neural network is replaced by convolutional layers 6-1 and 6-2. The feature map output by convolutional layer 6-1 is an M*N*3 feature map, and the feature map output by convolutional layer 6-2 is an M*N*5 feature map. The three channels of the M*N*3 feature map represent the probabilities that the corresponding small candidate box is located at the left and right boundaries of the text region to be detected in the input image, and other locations in the text region excluding the left and right boundaries. The M*N*5 feature map is used to regress the coordinates of the center point of the small candidate box, the length and width (x, y, w, h) of the small candidate box, and the angle θ. The feature map output by convolutional layer 6-1 is processed by softmax regression, using the cross-entropy loss function. The feature map output by convolutional layer 6-2 uses a regression loss function, and the loss function is calculated as follows: Where L1 is the loss function of convolutional layer 6-1, L2 is the loss function of convolutional layer 6-2, and f corresponds to the feature map output by convolutional layer 6-1. ijk P represents the probability of the k-th dimension of the candidate box in the i-th row and j-th column of the feature map output by convolutional layer 6-1. ijka preset probability of a kth dimension of a candidate box in an ith row and jth column of a feature map output by the convolutional layer 6-1, x ij an x-coordinate of a center point of a candidate box in an ith row and jth column of a feature map output by the convolutional layer 6-2 and equal in size to i, y ij an y-coordinate of a center point of a candidate box in an ith row and jth column of a feature map output by the convolutional layer 6-2 and equal in size to j, a preset x-coordinate of a center point of a candidate box in an ith row and jth column of a feature map output by the convolutional layer 6-2, a preset y-coordinate of a center point of a candidate box in an ith row and jth column of a feature map output by the convolutional layer 6-2, Δx ij a correction value of an x-coordinate of a center point of a candidate box in an ith row and jth column of a feature map output by the convolutional layer 6-2, Δy ij a correction value of a y-coordinate of a center point of a candidate box in an ith row and jth column of a feature map output by the convolutional layer 6-2, w ij a length of a candidate box in an ith row and jth column of a feature map output by the convolutional layer 6-2, a preset length of a candidate box in an ith row and jth column of a feature map output by the convolutional layer 6-2, Δw ij a correction value of a length of a candidate box in an ith row and jth column of a feature map output by the convolutional layer 6-2, h ij a width of a candidate box in an ith row and jth column of a feature map output by the convolutional layer 6-2, a preset width of a candidate box in an ith row and jth column of a feature map output by the convolutional layer 6-2, Δh ij a correction value of a width of a candidate box in an ith row and jth column of a feature map output by the convolutional layer 6-2, θ ij an angle of a candidate box in an ith row and jth column of a feature map output by the convolutional layer 6-2, The preset angle of the candidate frame of the i-th row and the j-th column in the feature map representing the output of the convolutional layer 6-2, the feature maps output by the convolutional layer 6-1 and the convolutional layer 6-2 have M*N candidate frames. The coordinates of the center points of the left frame and the right frame in all text regions on the image, the length and the width (x, y, h, w) of the left frame and the right frame and the angle θ can be obtained by using the output result of the ResNet 50 convolutional neural network. For each left frame, find the right frame with the same or similar angle (the error is within 15 degrees) closest to the right frame, that is, the paired right frame. For any left frame and the right frame closest to the left frame, take the center points of the left frame and the right frame, connect the two center points into a line, according to the length of the left frame and the right frame, expand the center point connecting line outward by the corresponding length (half of the width of the frame), and then take the average of the width of the left frame and the right frame as the height of the text region, that is, the complete text region detection frame can be obtained. In addition, in the embodiment of the present application, when fusion is performed, low convolutional layers are preferably used for fusion, because the edge, texture and color information of the feature image extracted by the low layer convolutional layer is more suitable for learning the relative position and angle, and the features extracted by the high layer convolutional layer are more abstract. Moreover, there cannot be a pooling layer between the convolutional layers for fusion, because the feature dimensions are not the same and cannot be fused. In addition, the feature map output by the convolutional layer 6-1 is processed by the softmax regression because the cross entropy describes the distance between two probability distributions, but the output of the neural network is not necessarily a probability distribution, and the Softmax changes the output of the neural network into a probability distribution.

[0048] In the embodiment of the present application, for the text line detection, only the leftmost and rightmost candidate boxes need to be located, and the whole text line can be framed by the text on the left and right ends. In the technical solution provided by the embodiment of the present application, the text line is split into small candidate boxes for detection, but different from the previous method, a natural scene text detection system based on relative position and angle information is proposed in the embodiment of the present application, the relative position information and angle information of each point on the feature map corresponding to the candidate box in the image are learned by the loss function and the hidden layer of different receptive fields (convolution layer fusion), because the receptive field represents the area size of the candidate box on the feature map of each layer output of the convolutional neural network mapped on the input picture, and the receptive field of the next layer is usually larger than that of the previous layer, that is, the area of the candidate box on the feature map of the next layer mapped on the input picture is larger than that of the previous layer. Assuming that the area of the input picture corresponding to a certain candidate box on the previous layer feature map is a small candidate box on the text line, the area of the input picture corresponding to the candidate box on the next layer feature map is the small candidate box and the surrounding area, so the position and confidence of the small candidate box are learned by the previous layer feature map, and the relative position (leftmost, middle or rightmost) of the small candidate box in the text line and the angle information of the text line are learned by the next layer feature map, and the complete text line detection box can be obtained by the detection box and the angle information of the two ends of the text line. The method is suitable for text line detection at any angle in a natural scene.

[0049] In summary, the technical solution provided by the embodiment of the present application considers that only the left and right candidate boxes need to be obtained for actual text line position prediction, the relative position information of the small candidate box in the text line is learned by the loss function and the hidden layer of different receptive fields, the problem of too many parameters and time consumption caused by predicting multiple candidate boxes is solved, the speed of text detection is greatly improved, and the angle feature learning of the text line is added, and the text line detection at any angle in a natural scene is realized.

[0050] Correspondingly, another aspect of the embodiment of the present application also provides a device for detecting text.

[0051] Figure 3 is the structure block diagram of the device for detecting text provided by another embodiment of the present application. As Figure 3As shown, the device comprises a candidate frame information obtaining module 1, a side frame determining module 2 and a line text region detection frame determining module 3. The candidate frame information obtaining module 1 is configured to input an image of text to be detected into a preset convolutional neural network structure to obtain candidate frame information of each candidate frame in a plurality of candidate frames into which the image is split, wherein the candidate frame information comprises a left side probability of the candidate frame, a right side probability of the candidate frame, other probabilities of the candidate frame and basic parameters of the candidate frame, wherein the left side probability is a probability that the candidate frame is located at a left boundary of a text region to be detected in the image, the right side probability is a probability that the candidate frame is located at a right boundary of the text region to be detected, and the other probabilities are probabilities that the candidate frame is located at other positions in the text region to be detected except the left boundary and the right boundary; the side frame determining module 2 is configured to determine, for any candidate frame in the plurality of candidate frames, whether the candidate frame is located at the left boundary, the right boundary or the other positions based on the left side probability, the right side probability and the other probabilities of the candidate frame to determine whether the candidate frame is a left side frame, a right side frame or other side frame; and the line text region detection frame determining module 3 is configured to determine, for any left side frame, a matching right side frame according to the basic parameters of the left side frame, and determine a line text region detection frame based on the basic parameters of the left side frame and the basic parameters of the matching right side frame, wherein the line text region detection frame is used to frame all the characters in a line from the left side frame to the matching right side frame in the image.

[0052] Optionally, in the embodiment of the present application, the basic parameters of the candidate frame comprise coordinate positions of a center point of the candidate frame and lengths and widths of the candidate frame, and the line text region detection frame determining module determines the line text region detection frame based on the basic parameters of the left side frame and the basic parameters of the matching right side frame for any left side frame, which comprises determining the line text region detection frame based on a left center point, a right center point, lengths and widths of the left side frame and lengths and widths of the matching right side frame, wherein the left center point is a center point of the left side frame and the right center point is a center point of the matching right side frame.

[0053] Optionally, in the embodiment of the present application, a first line segment is determined by the left center point and the right center point for any left side frame, a second line segment is obtained by extending a half of the length of the left side frame to the left side of the first line segment and extending a half of the length of the matching right side frame to the right side of the first line segment, the second line segment is taken as a center line of the line text region detection frame, the length of the line text region detection frame is the length of the second line segment, and the width of the line text region detection frame is an average of the width of the left side frame and the width of the matching right side frame.

[0054] Optionally, in the embodiment of the present application, the basic parameters of the candidate frame include the coordinate position of the center point of the candidate frame, and the determining of the matched right frame for any left frame includes: calculating the distance value between the coordinate position of the center point of each right frame in all right frames and the coordinate position of the center point of the left frame respectively; and determining the minimum distance value in all distance values by comparison, wherein the right frame corresponding to the minimum distance value is the matched right frame.

[0055] Optionally, in the embodiment of the present application, for any candidate frame in the plurality of candidate frames, the basic parameters further include the angle of the candidate frame, the angle being the included angle between the upper boundary of the candidate frame and the horizontal line of the image and the positive or negative of the angle depending on the rotation direction of the upper boundary of the candidate frame relative to the horizontal line of the image; and the matched right frame for any left frame further satisfies the following condition: the absolute value of the difference between the angle of the matched right frame and the angle of the left frame is less than a preset angle value.

[0056] Optionally, in the embodiment of the present application, the feature maps outputted by the preset value of convolution layers adjacent to each other in the preset convolutional neural network structure are fused in the channel dimension, there is no pooling layer between any two adjacent convolution layers in the preset value of convolution layers being fused, the output layer in the preset convolutional neural network structure is a convolution layer, and the feature map outputted by the output layer corresponds to the left probability, the right probability, the other probability and the basic parameters.

[0057] Optionally, in the embodiment of the present application, for any candidate frame in the plurality of candidate frames, the basic parameters include the coordinate position of the center point of the candidate frame, the length, the width and the angle of the candidate frame, the angle being the included angle between the upper boundary of the candidate frame and the horizontal line of the image and the positive or negative of the angle depending on the rotation direction of the upper boundary of the candidate frame relative to the horizontal line of the image, and the output layer of the preset convolutional neural network structure includes the first convolutional layer and the second convolutional layer in parallel, wherein the feature map outputted by the first convolutional layer corresponds to the left probability, the right probability and the other probability, and the feature map outputted by the second convolutional layer corresponds to the coordinate position, the length, the width and the angle.

[0058] Optionally, in the embodiment of the present application, the loss function adopted when the preset convolutional neural network structure is trained is: wherein L1 is the loss function of the first convolutional layer, L2 is the loss function of the second convolutional layer, f corresponds to the feature map outputted by the first convolutional layer, f ijk represents the probability of the kth dimension of the candidate frame in the i th row and the j th column of the feature map outputted by the first convolutional layer, P ijk represents the preset probability of the kth dimension of the candidate frame in the i th row and the j th column of the feature map outputted by the first convolutional layer, x ij represents the horizontal coordinate of the center point of the candidate frame in the i th row and the j th column of the feature map outputted by the second convolutional layer and the size is equal to i, y ija vertical coordinate of a center point of a candidate box in an i-th row and a j-th column of a feature map output by the second convolutional layer, and the size is equal to j, a preset vertical coordinate of a center point of a candidate box in an i-th row and a j-th column of a feature map output by the second convolutional layer, a preset vertical coordinate of a center point of a candidate box in an i-th row and a j-th column of a feature map output by the second convolutional layer, ij a correction value of a vertical coordinate of a center point of a candidate box in an i-th row and a j-th column of a feature map output by the second convolutional layer, ij a correction value of a vertical coordinate of a center point of a candidate box in an i-th row and a j-th column of a feature map output by the second convolutional layer, ij a length of a candidate box in an i-th row and a j-th column of a feature map output by the second convolutional layer, a preset length of a candidate box in an i-th row and a j-th column of a feature map output by the second convolutional layer, ij a correction value of a length of a candidate box in an i-th row and a j-th column of a feature map output by the second convolutional layer, ij a width of a candidate box in an i-th row and a j-th column of a feature map output by the second convolutional layer, a preset width of a candidate box in an i-th row and a j-th column of a feature map output by the second convolutional layer, ij a correction value of a width of a candidate box in an i-th row and a j-th column of a feature map output by the second convolutional layer, ij an angle of a candidate box in an i-th row and a j-th column of a feature map output by the second convolutional layer, a preset angle of a candidate box in an i-th row and a j-th column of a feature map output by the second convolutional layer, the feature maps output by the first convolutional layer and the second convolutional layer each have M*N candidate boxes.

[0059] The specific working principle and benefits of the device for detecting text provided by the embodiments of the present application are similar to those of the method for detecting text provided by the embodiments of the present application, and will not be repeated here.

[0060] The device for detecting text includes a processor and a memory, and the above-mentioned candidate box information obtaining module, the frame determining module and the line text region detection box determining module are all stored in the memory as program units, and the corresponding functions are realized by the processor executing the above-mentioned program units stored in the memory.

[0061] The processor contains a core, and the corresponding program units are called from the memory by the core. One or more cores can be set, and the calculation parameters, the calculation amount, the time, the time-consuming phenomenon and the calculation speed can be reduced by adjusting the core parameters.

[0062] The memory can include non-persistent memory in the form of computer-readable media, random access memory (RAM), and / or non-volatile memory such as read only memory (ROM) or flash memory, including at least one memory chip.

[0063] An embodiment of the present application provides a machine readable storage medium having stored thereon instructions for causing a machine to perform the method described in the above embodiments.

[0064] Another aspect of an embodiment of the present application further provides a processor for running a program, wherein the program performs the method described in the above embodiments when running.

[0065] Another aspect of an embodiment of the present application further provides a device including a processor, a memory, and a program stored in the memory and capable of running on the processor, and the processor implements the method described in the above embodiments when running the program. The device herein can be a server, a PC, a PAD, a mobile phone, etc.

[0066] Another aspect of an embodiment of the present application further provides a computer program product including a computer program / instructions, which, when executed by a processor, implements the method described in the above embodiments.

[0067] Those skilled in the art should understand that embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage media, etc.) containing computer usable program code.

[0068] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and combinations of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in a flow or multiple flows and / or blocks Figure 1 The functions specified in a flow or multiple flows and / or blocks

[0069] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the block or blocks.

[0070] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions that are executed on the computer or other programmable apparatus provide steps for implementing the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the block or blocks.

[0071] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0072] The memory can include non-persistent memory and / or volatile memory, such as a random access memory (RAM) including a cache area for the temporary storage of data. The memory can also include non-volatile memory, such as read only memory (ROM), electrically programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), flash memory, or non-volatile random access memory (NVRAM), for the storage of information, such as data files or program

[0073] Computer readable media includes permanent and non-permanent, removable and non-removable media implemented in any method or technology for the storage of information. The information can be computer readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital versatile discs (DVDs) or other optical storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer readable media does not include transitory media, such as modulated data signals and carrier waves.

[0074] It should also be noted that the terms "comprising", "comprises" or other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0075] The above embodiments are only used to illustrate the present application, but not to limit it. Instead of the above, various modifications and changes can be made to the application by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the application shall fall into the scope of the claims of the application.

Claims

1. A method for detecting text, characterized by, The method comprises: inputting an image of a text to be detected into a preset convolutional neural network structure to obtain candidate frame information of each candidate frame in a plurality of candidate frames into which the image is split, wherein for any candidate frame in the plurality of candidate frames, the candidate frame information comprises a left side probability of the candidate frame, a right side probability of the candidate frame, an other probability of the candidate frame, and a basic parameter of the candidate frame, wherein the left side probability is a probability that the candidate frame is located at a left boundary of a text region to be detected in the image, the right side probability is a probability that the candidate frame is located at a right boundary of the text region to be detected, and the other probability is a probability that the candidate frame is located at a position other than the left boundary and the right boundary in the text region to be detected; for any candidate frame in the plurality of candidate frames, determining whether the candidate frame is located at the left boundary, the right boundary or the other position based on a maximum value among the left side probability, the right side probability and the other probability of the candidate frame to determine whether the candidate frame is a left side frame, a right side frame or an other side frame; and for any left side frame, determining a paired right side frame according to the basic parameter of the left side frame; and based on the basic parameter of the left side frame and the basic parameter of the paired right side frame, determining a text region detection frame for a line of text, the text region detection frame being used for framing all characters in a line from the left side frame to the paired right side frame in the image.

2. The method of claim 1, wherein, The basic parameter of the candidate frame comprises a coordinate position of a center point of the candidate frame, a length and a width of the candidate frame, and the determination of the text region detection frame for any left side frame based on the basic parameter of the left side frame and the basic parameter of the paired right side frame comprises: determining the text region detection frame based on a left side center point, a right side center point, the length and the width of the left side frame, and the length and the width of the paired right side frame, wherein the left side center point is a center point of the left side frame, and the right side center point is a center point of the paired right side frame.

3. The method of claim 2, wherein, For any left side frame, a first line segment is determined based on the left side center point and the right side center point, a second line segment is obtained by extending a half of the length of the left side frame to the left side of the first line segment and extending a half of the length of the paired right side frame to the right side of the first line segment, the second line segment is taken as a center line of the text region detection frame, the length of the text region detection frame is the length of the second line segment, and the width of the text region detection frame is an average of the width of the left side frame and the width of the paired right side frame.

4. The method according to any one of claims 1-3, characterized in that, The basic parameter of the candidate frame comprises a coordinate position of a center point of the candidate frame, For any left side frame, the determination of the paired right side frame comprises: respectively calculating distance values between the coordinate position of the center point of each right side frame in all the right side frames and the coordinate position of the center point of the left side frame; and A minimum distance value is determined by comparison among all the distance values, wherein the minimum distance value corresponds to the matching right bounding box.

5. The method of claim 4, wherein, For any candidate box in the plurality of candidate boxes, the basic parameter further comprises an angle of the candidate box, the angle being an angle between an upper boundary of the candidate box and a horizontal line of the image and a positive or negative of the angle depending on a rotation direction of the upper boundary of the candidate box relative to the horizontal line of the image. For any left bounding box, the matching right bounding box further satisfies a condition that an absolute value of a difference between the angle of the matching right bounding box and the angle of the left bounding box is less than a preset angle value.

6. The method of claim 1, wherein, The feature maps outputted by preset values of convolution layers adjacent to each other in the preset convolutional neural network structure are fused in a channel dimension, there is no pooling layer between any two adjacent convolution layers among the preset values of convolution layers being fused, an output layer in the preset convolutional neural network structure is a convolution layer, and the feature maps outputted by the output layer correspond to the left edge probability, the right edge probability, the other probability, and the basic parameter.

7. The method of claim 6, wherein, For any candidate box in the plurality of candidate boxes, the basic parameter comprises a coordinate position of a center point of the candidate box, a length, a width, and an angle of the candidate box, the angle being an angle between an upper boundary of the candidate box and a horizontal line of the image and a positive or negative of the angle depending on a rotation direction of the upper boundary of the candidate box relative to the horizontal line of the image, and an output layer of the preset convolutional neural network structure comprises a first convolutional layer and a second convolutional layer in parallel, wherein the feature maps outputted by the first convolutional layer correspond to the left edge probability, the right edge probability, and the other probability, and the feature maps outputted by the second convolutional layer correspond to the coordinate position, the length, the width, and the angle.

8. The method of claim 7, wherein, A loss function used when the preset convolutional neural network structure is trained is: wherein, a loss function of the first convolutional layer, a loss function of the second convolutional layer, a feature map corresponding to the first convolutional layer output, a probability of a kth dimension of a candidate box in an ith row and jth column of the feature map output by the first convolutional layer, a preset probability of a kth dimension of a candidate box in an ith row and jth column of the feature map output by the first convolutional layer, a horizontal coordinate of a center point of a candidate box in an ith row and jth column of the feature map output by the second convolutional layer, and a size equal to i, a vertical coordinate of a center point of a candidate box in an ith row and jth column of the feature map output by the second convolutional layer, and a size equal to j, a preset horizontal coordinate of a center point of a candidate box in an ith row and jth column of the feature map output by the second convolutional layer, a preset vertical coordinate of a center point of a candidate box in an ith row and jth column of the feature map output by the second convolutional layer, a correction value of a horizontal coordinate of a center point of a candidate box in an ith row and jth column of the feature map output by the second convolutional layer, a correction value of a vertical coordinate of a center point of a candidate box in an ith row and jth column of the feature map output by the second convolutional layer, a length of a candidate box in an ith row and jth column of the feature map output by the second convolutional layer, a preset length of a candidate box in an ith row and jth column of the feature map output by the second convolutional layer, a correction value of a length of a candidate box in an ith row and jth column of the feature map output by the second convolutional layer, a width of a candidate box in an ith row and jth column of the feature map output by the second convolutional layer, a preset width of a candidate box in an ith row and jth column of the feature map output by the second convolutional layer, a correction value of a width of a candidate box in an ith row and jth column of the feature map output by the second convolutional layer, an angle of a candidate box in an ith row and jth column of the feature map output by the second convolutional layer, a preset angle of a candidate box in an ith row and jth column of the feature map output by the second convolutional layer, the feature maps output by the first convolutional layer and the second convolutional layer respectively have candidate boxes.

9. A machine-readable storage medium, characterized in that, The machine readable storage medium has instructions stored thereon for causing a machine to perform the method of any one of claims 1-8.

10. A computer program product comprising computer programs / instructions, characterized in that, The computer program / instructions, when executed by a processor, implement the method of any one of claims 1-8. The computer program / instructions, when executed by a processor, implement the method of any one of claims 1-8.

Images