Scene text detection method based on polynomial fitting

Abstract

Embodiments of the present application relate to the technical field of image processing, and disclose a scene text detection method based on polynomial fitting, which comprises the following steps: performing affine transformation on a text region frame marked on a sample image, so that the center of the text region frame after transformation is located at the origin and the line connecting the first and last points of the upper long side is in a horizontal state; performing polynomial fitting on the text region frame after transformation to obtain polynomial coefficients, and marking the polynomial coefficients as training labels; inputting the sample image into an initial network to obtain predicted polynomial coefficients, and iteratively training the initial network based on the predicted polynomial coefficients and the training labels until convergence, so as to obtain a trained detection model; and inputting an image to be detected into the detection model to output a text region in the image to be detected. The method needs to learn fewer parameters and all the parameters conform to visual rules, the pressure of model training is relatively small, and the trained detection model can accurately recognize text regions of any shape.

Description

Technical Field

[0001] The embodiments of this application relate to the field of image processing technology, and in particular to a scene text detection method based on polynomial fitting. Background Technology

[0002] Scene text detection refers to the process of automatically detecting text regions from images or videos in natural scenes using computer vision technology. Scene text detection is the first step in scene text recognition, and its main task is to determine the location and size of text regions, providing accurate data for subsequent character segmentation and recognition. Currently, the more mature end-to-end text detection methods in the industry mainly include two types: segmentation-based scene text detection and regression-based scene text detection.

[0003] Segmentation-based scene text detection methods primarily involve segmenting the input image and then detecting the scene text based on the segmented text regions. For example, the FCN (Fully Convolutional Networks) model proposed by Jonathan Long et al. in their paper "Fully Convolutional Networks for Semantic Segmentation" uses this approach. While the FCN model extracts image features through multiple downsampling operations, this leads to reduced resolution and loss of spatial information, making it difficult to locate and segment small text regions.

[0004] Regression-based scene text detection methods primarily extract features through regression, thereby enabling scene text detection. For example, the ABCNet model proposed by Yuliang Liu et al. in their paper "ABCNet: Real-time Scene Text Spotting with Adaptive Bezier-Curve Network" addresses this issue. While the ABCNet model avoids information loss, it requires learning a large number of parameters, some of which do not conform to visual rules, making training the ABCNet model quite challenging. Summary of the Invention

[0005] The purpose of this application is to provide a scene text detection method based on polynomial fitting. It requires fewer parameters to be learned, all of which conform to visual laws. The training pressure of the model is small, and the trained detection model can accurately identify text regions of any shape. It has strong generalization ability, strong universality, and wide application scenarios.

[0006] To address the aforementioned technical problems, embodiments of this application provide a scene text detection method based on polynomial fitting, comprising the following steps: performing an affine transformation on the text region bounding boxes labeled on the sample image, such that the center of the transformed text region bounding box is located at the origin and the line connecting the first and last points of the upper long side is horizontal; performing polynomial fitting on the transformed text region bounding box to obtain polynomial coefficients, and labeling the polynomial coefficients as training labels for the sample image; inputting the sample image into an initial network to obtain the predicted polynomial coefficients output by the initial network, and iteratively training the initial network based on the predicted polynomial coefficients output by the initial network and the training labels until the initial network converges to obtain a trained detection model; inputting the image to be detected into the detection model to output the text regions in the image to be detected.

[0007] Embodiments of this application also provide an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the above-described scene text detection method based on polynomial fitting.

[0008] Embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described scene text detection method based on polynomial fitting.

[0009] The scene text detection method based on polynomial fitting provided in this application requires less information for scene text detection. A cubic polynomial is sufficient for effective fitting, and only a dozen or so parameters are needed to represent a text region. These parameters characterize edge curve features and the first and last points of the edge. Compared to the Bezier control nodes required by the ABCnet model, this method better conforms to visual patterns, reduces the pressure on model learning and training, and results in faster and more efficient model training. In the labeling stage before model training, an affine transformation is performed on the original text region bounding box. The transformed text region bounding box is centered at the origin, and the line connecting the first and last points of the upper long side is horizontal. This solves the problem of learning text region bounding boxes with excessively steep slopes, making the data easier to fit. The detection model trained in this way can accurately detect and recognize text regions of any shape, exhibiting strong generalization ability, wide applicability, and broad application scenarios.

[0010] In some optional embodiments, the step of performing an affine transformation on the text region bounding box annotated on the sample image, such that the center of the transformed text region bounding box is located at the origin and the line connecting the first and last points of the upper long side is horizontal, includes: obtaining the coordinates of the first and last points of the upper long side of the text region bounding box; determining the rotation angle and scaling size in the affine transformation based on the coordinates of the first and last points of the upper long side; rotating the text region bounding box clockwise by the rotation angle, and dividing the rotated text region bounding box by the scaling size to perform scaling; calculating the mean of the horizontal coordinates and the mean of the vertical coordinates of each annotation point on the upper long side based on the coordinates of each annotation point on the upper long side; wherein the annotation points include the first and last points; and translating the scaled text region bounding box based on the mean of the horizontal coordinates and the mean of the vertical coordinates to obtain the transformed text region bounding box. Affine transformations, including rotation, scaling, and translation, eliminate excessively steep slopes on the long side of text region boxes. This results in standardized and regular training labels, and the text region boxes detected by the trained detection model also exhibit standardized and regular characteristics, facilitating subsequent text recognition.

[0011] In some optional embodiments, the rotation angle and scaling dimension in the affine transformation are determined based on the coordinates of the first and last points of the upper long side using the following formula:

[0012]

[0013]

[0014] in, This represents the coordinates of the first point of the upper longer side. The coordinates of the tail point of the upper long side are represented by θ, the rotation angle is represented by l, and the scaling size is represented by l.

[0015] In some optional embodiments, the mean abscissa and mean ordinate of each annotation point on the upper long side are calculated based on the coordinates of each annotation point on the upper long side using the following formula:

[0016]

[0017] in, This represents the x-coordinate of the i-th gaze point on the upper longer side. x represents the ordinate of the i-th gaze point on the upper longer side. median The mean of the x-axis, y median Let represent the mean of the vertical coordinates, and n represent the total number of annotation points; the transformed text region box is obtained by translating the scaled text region box based on the mean of the horizontal coordinates and the mean of the vertical coordinates, using the following formula:

[0018]

[0019]

[0020] in, This represents the coordinates of the k-th point on the scaled text region box. This represents the coordinates of the k-th point on the transformed text region box.

[0021] In some optional embodiments, the step of inputting the image to be detected into the detection model and outputting the text region in the image to be detected includes: inputting the image to be detected into the detection model; using the fitting module of the detection model to determine the fitting polynomial coefficients, rotation coefficients, scaling coefficients, and translation coefficients corresponding to the text region in the image to be detected; using the edge reproduction module of the detection model to generate a visualized edge function curve based on the fitting polynomial coefficients; and performing an inverse affine transformation on the edge function curve based on the rotation coefficients, the scaling coefficients, and the translation coefficients to output the text region in the image to be detected. Since the training labels of the sample images are obtained by performing an affine transformation on the original text region boxes, the polynomial coefficients predicted by the trained model are also after the affine transformation. The visualized edge function curve drawn based on the polynomial coefficients is not the real text region box. Therefore, the detection model also has the function of predicting rotation coefficients, scaling coefficients, and translation coefficients. Only by performing an inverse affine transformation on the edge function curve based on the rotation coefficients, scaling coefficients, and translation coefficients can the text region in the image to be detected be output.

[0022] In some optional embodiments, the translation coefficient includes a horizontal coordinate translation coefficient and a vertical coordinate translation coefficient. The step of performing an inverse affine transformation on the edge function curve based on the rotation coefficient, the scaling coefficient, and the translation coefficient to output the text region in the image to be detected includes: rotating the edge function curve counterclockwise by the rotation angle corresponding to the rotation coefficient; multiplying the inversely rotated edge function curve by the scaling coefficient to perform inverse scaling; performing inverse translation on the inversely scaled edge function curve based on the horizontal coordinate translation coefficient and the vertical coordinate translation coefficient; and outputting the region corresponding to the inversely translated edge function curve as the text region detected from the image to be detected. The inverse affine transformation corresponds to the reverse operation of rotation, scaling, and translation in the label generation process. After the inverse affine transformation, the large slope of the long side is restored, and the edge function curve is reproduced in the true position of the text region.

[0023] In some optional embodiments, the inverse translation of the inversely scaled edge function curve based on the horizontal coordinate translation coefficient and the vertical coordinate translation coefficient is achieved by the following formula:

[0024]

[0025]

[0026] Where, x * Let y be the horizontal coordinate translation coefficient. * Let x be the translation coefficient of the ordinate. m ,y m () represents the coordinates of the m-th point on the inversely scaled edge function curve. This represents the coordinates of the m-th point on the edge function curve after the inverse translation.

[0027] In some optional embodiments, the step of performing polynomial fitting on the transformed text region bounding box to obtain polynomial coefficients includes: performing linear interpolation between two adjacent points on the transformed text region bounding box based on their coordinates to obtain a linearly interpolated text region bounding box; and performing polynomial fitting on the linearly interpolated text region bounding box to obtain polynomial coefficients. Linear interpolation can enrich the data of the points in the text region bounding box, thus making the training label information more accurate, the trained detection model has stronger detection and fitting capabilities, and the output text region is more accurate. Attached Figure Description

[0028] One or more embodiments are illustrated by way of example with reference to the accompanying drawings, and these illustrative descriptions do not constitute a limitation on the embodiments.

[0029] Figure 1 This is a flowchart of a scene text detection method based on polynomial fitting provided in one embodiment of this application;

[0030] Figure 2 This is a flowchart provided by an embodiment of the present application, which describes the affine transformation of a text region box marked on a sample image, such that the center of the transformed text region box is located at the origin and the line connecting the first and last points of the upper long side is horizontal.

[0031] Figure 3 This is a flowchart provided by one embodiment of the present application, which describes how to input an image to be detected into a detection model and output the text region in the image to be detected;

[0032] Figure 4 This is a schematic diagram of model fitting ability visualization analysis provided in one embodiment of this application;

[0033] Figure 5 This is a schematic diagram of statistical analysis of model fitting ability provided in one embodiment of this application;

[0034] Figure 6This is a fitting effect diagram provided in one embodiment of this application;

[0035] Figure 7 This is another fitting effect diagram provided by one embodiment of this application;

[0036] Figure 8 This is a schematic diagram of the structure of an electronic device provided in another embodiment of this application. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the various embodiments of this application will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details have been provided in the various embodiments of this application to help readers better understand this application. However, the technical solutions claimed in this application can be implemented even without these technical details and various changes and modifications based on the following embodiments. The division of the various embodiments below is for the convenience of description and should not constitute any limitation on the specific implementation of this application. The various embodiments can be combined with and referenced by each other without contradiction.

[0038] One embodiment of this application provides a scene text detection method based on polynomial fitting, applied to an electronic device, wherein the electronic device can be a terminal or a server. In this embodiment and the following embodiments, the electronic device is described using a server as an example. The implementation details of the scene text detection method based on polynomial fitting in this embodiment are described below. The following content is only for the convenience of understanding the implementation details provided and is not necessary for implementing this solution.

[0039] The specific process of the scene text detection method based on polynomial fitting in this embodiment can be described as follows: Figure 1 As shown, it includes:

[0040] Step 101: Perform an affine transformation on the text region bounding box marked on the sample image so that the center of the transformed text region bounding box is located at the origin and the line connecting the first and last points of the upper long side is horizontal.

[0041] In the specific implementation, when training the initial network, the first step is to obtain a training sample set. The server can generate the training sample set based on existing open-source datasets on the Internet. The training sample set contains several sample images, each labeled with a text region bounding box (which can be considered as the original label). The slope of the long side of the unprocessed text region bounding box may be very large, which is not conducive to polynomial fitting. Therefore, the server performs an affine transformation on the labeled text region bounding box of each sample image, so that the center of the transformed text region bounding box is located at the origin and the line connecting the first and last points of the long side is horizontal. In this way, the slope of the long side of the transformed text region bounding box is not very large, which is conducive to polynomial fitting. At the same time, the center of the transformed text region bounding box is located at the origin, making the labeling process more standardized and the newly generated labels more regular.

[0042] It is worth noting that a text region box usually has two long sides, namely the upper long side and the lower long side. The upper long side and the lower long side are usually parallel. Even if they are not parallel, the difference in their slopes will not be too large. Therefore, the affine transformation only requires that the line connecting the first and last points of the upper long side of the transformed text region box be horizontal.

[0043] In one example, the server performs an affine transformation on the text region bounding boxes labeled on the sample image, so that the center of the transformed text region bounding box is located at the origin and the line connecting the first and last points of the upper longer side is horizontal. This can be achieved by, for example... Figure 2 The steps shown are implemented as follows:

[0044] Step 1011: Obtain the coordinates of the first and last points of the upper long side of the text region box. Based on the coordinates of the first and last points of the upper long side, determine the rotation angle and scaling size in the affine transformation.

[0045] In its implementation, affine transformation includes three operations: rotation, scaling, and translation. The text region bounding box on the sample image is actually composed of several annotation points. The annotation points on the upper long side of the text region bounding box can be represented as follows: in, Let be the coordinates of the first point of the upper longer side. These are the coordinates of the last point of the upper longer side. The server can determine the rotation angle and scaling size in the affine transformation based on the coordinates of the first and last points of the upper longer side.

[0046] In one example, the server can determine the rotation angle and scaling size in an affine transformation based on the coordinates of the first and last points of the upper longer side using the following formula:

[0047]

[0048]

[0049] In the formula, θ represents the rotation angle in the affine transformation, and l represents the scaling dimension in the affine transformation. As can be seen from the formula, the scaling dimension l is actually the distance between the first and last points of the upper long side.

[0050] Step 1012: Rotate the text area frame clockwise by the rotation angle, and divide the rotated text area by the scaling size to perform scaling.

[0051] In the specific implementation, after the server determines the rotation angle θ and scaling size l in the affine transformation, it rotates the text region box clockwise by the rotation angle θ to complete the rotation transformation, and then divides the rotated text region by the scaling size l to complete the scaling transformation.

[0052] In one example, scaling transformation can be represented as: This represents the coordinates of the k-th point on the rotated text area box. This represents the coordinates of the k-th point on the scaled text area box.

[0053] Step 1013: Calculate the mean x-coordinate and mean y-coordinate of each annotation point on the upper long side based on their coordinates.

[0054] In the specific implementation, after completing the rotation and scaling transformations, the server needs to continue with the translation transformation to ensure that the center of the transformed text area box is located at the origin. The server calculates the average horizontal and vertical coordinates of each annotation point on the upper long side based on the coordinates of each annotation point on the upper long side (the annotation points on the upper long side include the first and last points of the upper long side).

[0055] In one example, the server calculates the mean x-coordinate and mean y-coordinate of all annotation points on the upper long side using the following formula:

[0056]

[0057]

[0058] In the formula, This represents the x-coordinate of the i-th gaze point on the upper longer side. Let x represent the ordinate of the i-th gaze point on the upper longer side. median This represents the calculated mean of the x-axis, y. median This represents the calculated mean of the ordinate, and n represents the total number of annotation points.

[0059] Step 1014: Based on the mean of the horizontal and vertical coordinates, translate the scaled text region box to obtain the transformed text region box.

[0060] In the implementation, after the server calculates the mean values ​​of the x-coordinate and y-coordinate, it can translate the scaled text region box based on these mean values ​​to obtain the transformed text region box. Affine transformations, including rotation, scaling, and translation, eliminate excessively steep slopes on the long side of the text region box. This results in standardized and regular training labels, and the text region boxes detected by the trained detection model also exhibit standardized and regular characteristics, facilitating subsequent text recognition.

[0061] In one example, the server translates the scaled text region box based on the mean of the horizontal and vertical coordinates to obtain the transformed text region box, which can be achieved using the following formula:

[0062]

[0063]

[0064] In the formula, This represents the coordinates of the k-th point on the scaled text area box. This represents the coordinates of the k-th point on the text region box after the affine transformation.

[0065] Step 102: Perform polynomial fitting on the transformed text region bounding box to obtain polynomial coefficients, and label the polynomial coefficients as training labels for the sample image.

[0066] In the specific implementation, after the server completes the affine transformation of the text region, it can perform polynomial fitting on the transformed text region box to obtain the polynomial coefficients of the fitted polynomial. The polynomial coefficients of the fitted polynomial are then labeled as the training labels of the sample image, and the original labels are no longer used during subsequent training.

[0067] In one example, when performing polynomial fitting, the server first performs linear interpolation between two adjacent points on the transformed text region bounding box, obtaining the linearly interpolated text region bounding box. Then, it performs polynomial fitting on the linearly interpolated text region bounding box to obtain the polynomial coefficients. Linear interpolation enriches the data of the points in the text region bounding box, making the training label information more accurate. This results in a more powerful detection and fitting model, leading to a more accurate output text region.

[0068] Step 103: Input the sample image into the initial network to obtain the predicted polynomial coefficients of the initial network output. Based on the predicted polynomial coefficients of the initial network output and the training labels, iteratively train the initial network until the initial network converges to obtain the trained detection model.

[0069] In the specific implementation, after the server completes the labeling of all sample images in the training sample set, it can enter the model training phase. The sample images are input into the initial network to obtain the prediction polynomial coefficients output by the initial network. The initial network is the untrained detection model. Based on the prediction polynomial coefficients output by the initial network and the training labels annotated on the sample images, the server iteratively trains the initial network until the initial network converges, resulting in the trained detection model.

[0070] Step 104: Input the image to be detected into the detection model and output the text region in the image to be detected.

[0071] In practice, once the server obtains the trained detection model, it can put the detection model into the image retrieval task. The image to be detected is input into the detection model, and the detection model outputs the text region in the image to be detected.

[0072] In one example, the server inputs the image to be detected into the detection model and outputs the text regions in the image to be detected, which can be done through methods such as... Figure 3 The steps shown are implemented as follows:

[0073] Step 1041: Input the image to be detected into the detection model, and use the fitting module of the detection model to determine the fitting polynomial coefficients, rotation coefficients, scaling coefficients and translation coefficients corresponding to the text region in the image to be detected.

[0074] In the specific implementation, since the training labels of the sample images are obtained by affine transformation of the original text region boxes, the polynomial coefficients predicted by the trained model are also those of the affine transformation. Therefore, the visualized edge function curves drawn based on the polynomial coefficients are not the actual text region boxes. Thus, in addition to predicting the fitted polynomial coefficients, the detection model also has the function of predicting rotation coefficients, scaling coefficients, and translation coefficients. The server inputs the image to be detected into the detection model, i.e., into the fitting module of the detection model. The fitting module outputs the fitted polynomial coefficients, rotation coefficients, scaling coefficients, and translation coefficients corresponding to the text regions in the image to be detected, and provides them to the edge reproduction module of the detection model.

[0075] In one example, the rotation coefficients output by the fitting module are denoted as θ. * The scaling factor is denoted as l. * The translation coefficients include the horizontal axis translation coefficient and the vertical axis translation coefficient. The horizontal axis translation coefficient is denoted as x. * The vertical axis translation coefficient is denoted as y. * .

[0076] Step 1042: Using the edge reproduction module of the detection model, a visualized edge function curve is generated based on the fitted polynomial coefficients. The edge function curve is then subjected to an inverse affine transformation based on the rotation coefficient, scaling coefficient, and translation coefficient to output the text region in the image to be detected.

[0077] In the specific implementation, the edge reconstruction module first generates (draws) a visualized edge function curve based on the coefficients of the fitted polynomial. Then, it performs an inverse affine transformation on the edge function curve based on the rotation coefficient, scaling coefficient, and translation coefficient. The inverse affine transformation corresponds to the reverse operation of rotation, scaling, and translation in the training label generation process. After the inverse affine transformation, the large slope of the long side is restored, and the edge function curve is reproduced in the real position of the text region, thus obtaining the text region in the image to be detected.

[0078] In one example, the inverse affine transformation includes three inverse transformation operations: inverse rotation, inverse scaling, and inverse translation. The server first rotates the edge function curve counterclockwise by the rotation angle θ corresponding to the rotation coefficient. * Then multiply the inversely rotated edge function curve by the scaling factor l * To perform the inverse scaling transformation, the inverse scaling transformation can be expressed as: This represents the coordinates of the m-th point on the inversely rotated edge function curve. This represents the coordinates of the m-th point on the inversely scaled edge function curve. Finally, the edge reconstruction module performs an inverse translation on the inversely scaled edge function curve based on the horizontal and vertical coordinate translation coefficients, and outputs the region corresponding to the inversely translated edge function curve as the text region detected from the image to be detected.

[0079] In one example, the server performs an inverse translation on the inversely scaled edge function curve based on the horizontal and vertical coordinate translation coefficients, which can be achieved using the following formula:

[0080]

[0081]

[0082] In the formula, x * The horizontal axis translation coefficient is y. * The vertical axis translation coefficient is... This represents the coordinates of the m-th point on the edge function curve after inverse scaling. This represents the coordinates of the m-th point on the edge function curve after inverse translation.

[0083] This embodiment uses a polynomial fitting approach for scene text detection, which requires less information. A cubic polynomial is sufficient for effective fitting, and only a dozen or so parameters are needed to represent a text region. These parameters characterize edge curve features and the first and last points of the edge. Compared to the Bezier control nodes that the ABCnet model needs to learn, this approach is more consistent with visual patterns, resulting in less learning and training pressure, faster training speed, and higher training efficiency. In the labeling stage before model training, an affine transformation is performed on the original text region bounding boxes. The transformed text region bounding boxes are centered at the origin, and the line connecting the first and last points of the upper long side is horizontal. This solves the problem of learning text region bounding boxes with excessively steep slopes, making the data easier to fit. The detection model trained in this way can accurately detect and recognize text regions of any shape, exhibiting strong generalization ability, high versatility, and wide applicability.

[0084] The steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this patent.

[0085] In one embodiment, to verify the performance superiority of the scene text detection method based on polynomial fitting proposed in this application, simulation experiments were conducted on different datasets.

[0086] (1) Simulation conditions

[0087] In the central processing unit Simulation was performed using MATLAB software on an i5-3470 3.2GHz CPU, 4GB of RAM, and a Windows 7 operating system.

[0088] The test image dataset used in the experiment comes from the CTW-1500 dataset, a Chinese dataset for text detection and recognition. This dataset contains 1500 images, including various text instances such as horizontal text, curved text, and multi-directional text.

[0089] (2) Simulation content

[0090] Figure 4 This is a diagram illustrating the visualization analysis of the model's fitting ability. Figure 4The blue box represents the ground truth of the text region, while the red box represents the fitted region (before rotation). If the edges of the text region are fitted using traditional polynomial fitting methods, the steep slopes of the two longer sides can lead to poor fitting. However, the method proposed in this application, which involves rotating the fitted image counterclockwise, yields the final result, overcoming the problem of excessively steep slopes on the two longer sides. The fitting effect can be characterized by the IOU (Intersection over Union) value. Figure 5 As can be seen, the IOU values ​​of the fitted cases are mostly between 0.9 and 1, indicating a good fitting effect. Figure 6 , Figure 7 The paper presents some fitting results on the CTW-1500 dataset. The scene text detection method based on multinomial fitting proposed in this application can accurately detect scene text of arbitrary shape and orientation, and is more conducive to model training compared with traditional regression methods.

[0091] Another embodiment of this application relates to an electronic device, such as... Figure 8 As shown, it includes: at least one processor 201; and a memory 202 communicatively connected to the at least one processor 201; wherein the memory 202 stores instructions executable by the at least one processor 201, the instructions being executed by the at least one processor 201 to enable the at least one processor 201 to perform the scene text detection method based on polynomial fitting in the above embodiments.

[0092] The memory and processor are connected via a bus, which can include any number of interconnecting buses and bridges, connecting various circuits of one or more processors and memories. The bus can also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and will not be described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor is transmitted over the wireless medium via an antenna, which further receives data and transmits it to the processor.

[0093] The processor manages the bus and general processing, and also provides various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory is used to store data used by the processor during operation.

[0094] Another embodiment of this application relates to a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the method embodiments described above.

[0095] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0096] Those skilled in the art will understand that the above embodiments are specific embodiments for implementing this application, and in practical applications, various changes can be made to them in form and detail without departing from the spirit and scope of this application.

Claims

1. A scene text detection method based on polynomial fitting, characterized in that, include: Perform an affine transformation on the text region bounding box marked on the sample image so that the center of the transformed text region bounding box is located at the origin and the line connecting the first and last points of the upper long side is horizontal. The transformed text region box is fitted with a polynomial to obtain polynomial coefficients, and the polynomial coefficients are labeled as training labels for the sample image. The sample image is input into the initial network to obtain the predicted polynomial coefficients output by the initial network. Based on the predicted polynomial coefficients output by the initial network and the training labels, the initial network is iteratively trained until the initial network converges to obtain the trained detection model. The image to be detected is input into the detection model, and the text region in the image to be detected is output. The step of performing an affine transformation on the text region bounding boxes marked on the sample image, such that the center of the transformed text region bounding box is located at the origin and the line connecting the first and last points of the upper longer side is horizontal, includes: Obtain the coordinates of the first and last points of the upper long side of the text region box, and determine the rotation angle and scaling size in the affine transformation based on the coordinates of the first and last points of the upper long side. Rotate the text area frame clockwise by the rotation angle, and divide the rotated text area by the scaling size to perform scaling; Based on the coordinates of each annotation point on the upper long side, calculate the average x-coordinate and the average y-coordinate of each annotation point on the upper long side; wherein, the annotation points include the first and last points. Based on the mean of the horizontal coordinate and the mean of the vertical coordinate, the scaled text region box is translated to obtain the transformed text region box. The step of inputting the image to be detected into the detection model and outputting the text region in the image to be detected includes: The image to be detected is input into the detection model, and the fitting module of the detection model is used to determine the fitting polynomial coefficients, rotation coefficients, scaling coefficients and translation coefficients corresponding to the text region in the image to be detected. Using the edge reproduction module of the detection model, a visualized edge function curve is generated based on the fitting polynomial coefficients. Then, an inverse affine transformation is performed on the edge function curve based on the rotation coefficient, the scaling coefficient, and the translation coefficient to output the text region in the image to be detected.

2. The scene text detection method based on polynomial fitting according to claim 1, characterized in that, The rotation angle and scaling factor in the affine transformation are determined using the following formula, based on the coordinates of the first and last points of the upper longer side: ； ； in, This represents the coordinates of the first point of the upper longer side. This represents the coordinates of the tail point of the upper longer side. Indicates the rotation angle, This indicates the scaling size.

3. The scene text detection method based on polynomial fitting according to claim 1, characterized in that, The mean x-coordinate and mean y-coordinate of each annotation point on the upper long side are calculated using the following formulas, based on their coordinates: ； ； in, The first side of the upper long side is represented by the first side. The x-coordinate of each gaze point The first side of the upper long side is represented by the first side. The ordinate of each gaze point This represents the mean of the x-axis. This represents the mean of the ordinate. This indicates the total number of the annotation points; The transformed text region box is obtained by translating the scaled text region box based on the mean of the horizontal and vertical coordinates, using the following formula: ； ； in, The text region box represents the first... The coordinates of the points The text region box represents the first... The coordinates of the points.

4. The scene text detection method based on polynomial fitting according to claim 1, characterized in that, The translation coefficients include horizontal coordinate translation coefficients and vertical coordinate translation coefficients. The inverse affine transformation of the edge function curve based on the rotation coefficient, the scaling coefficient, and the translation coefficients, outputting the text region in the image to be detected, includes: Rotate the edge function curve counterclockwise by the rotation angle corresponding to the rotation coefficient; Multiply the inversely rotated edge function curve by the scaling factor to perform inverse scaling; Based on the horizontal and vertical translation coefficients, the edge function curve after inverse scaling is inversely translated, and the region corresponding to the inversely translated edge function curve is used as the text region detected from the image to be detected and output.

5. The scene text detection method based on polynomial fitting according to claim 4, characterized in that, The inverse translation of the inversely scaled edge function curve based on the horizontal and vertical coordinate translation coefficients is achieved by the following formula: ； ； in, The horizontal axis translation coefficient is... The vertical coordinate translation coefficient is... The edge function curve after inverse scaling represents the first... The coordinates of the points The edge function curve after the inverse translation represents the first... The coordinates of the points.

6. The scene text detection method based on polynomial fitting according to any one of claims 1 to 5, characterized in that, The step of performing polynomial fitting on the transformed text region box to obtain polynomial coefficients includes: Based on the coordinates of two adjacent points on the transformed text region box, linear interpolation is performed between the two adjacent points to obtain the linearly interpolated text region box. Polynomial fitting is performed on the linearly interpolated text region box to obtain the polynomial coefficients.

7. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the scene text detection method based on polynomial fitting as described in any one of claims 1 to 6.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the scene text detection method based on polynomial fitting as described in any one of claims 1 to 6.

Images