A method and device for determining a text image region detection model

Abstract

The present invention discloses a method and device for determining a text image region detection model. The method comprises: constructing a detection network model to be trained according to a preset detection network construction algorithm; performing a preset image processing operation on an image to be processed according to the detection network model to be trained to obtain an image processing result; performing a feature comparison operation on the text detection frame coordinates included in the image processing result and the predetermined text pre-marked coordinates of the image to be processed according to a preset feature comparison module to obtain a feature comparison result; and determining that the detection network model to be trained is a target detection network model that has completed training when the feature comparison result indicates that the training parameters of the detection network to be trained are greater than or equal to a preset convergence threshold. It can be seen that the implementation of the present invention provides a method for constructing and training a text image region detection model, which is conducive to improving the detection accuracy and recognition accuracy of text in image regions.

Description

Technical Field

[0001] The present invention relates to the field of text detection technology, and in particular to a method and device for determining a text image region detection model. Background Art

[0002] In the era of big data, in order to better promote a product, multimodal information is usually used for product promotion, the most common of which are images, videos, text and voice.

[0003] In order to increase product sales, adding concise and concise promotional text to promotional templates is the most common promotional method. However, if the promotional text is presented neatly and neatly in the image, it is easy to cause visual fatigue to consumers, and the promotional effect of the promotional text will be greatly reduced. Therefore, in order to better attract consumers' attention, advertising designers usually transform ordinary text into various artistic fonts and use fan-shaped, circular, or even irregular shapes to display them, and the fonts themselves are also of different sizes. Precisely because of the various ways of displaying text, the existing text detection technology has extremely low text detection accuracy when processing the diverse images mentioned above. Therefore, it is particularly important to provide a method to improve text detection accuracy. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method and device for determining a text image region detection model, which can improve the detection accuracy of text in an image region and improve the recognition accuracy of text in an image region.

[0005] In order to solve the above technical problems, the first aspect of the present invention discloses a method for determining a text image region detection model, the method comprising:

[0006] According to a preset detection network construction algorithm, a to-be-trained detection network model is constructed for performing a preset image processing operation on an input image, wherein the to-be-trained detection network model includes a feature extraction network, a feature fusion network, and a multi-task branch;

[0007] Performing a preset image processing operation on the image to be processed according to the detection network model to be trained to obtain a corresponding image processing result, wherein the image processing result includes the coordinates of the text detection box corresponding to the image to be processed;

[0008] According to the determined feature comparison module, a feature comparison operation is performed on the coordinates of the text detection box and the predetermined coordinates of the text pre-marked of the image to be processed to obtain a corresponding feature comparison result, wherein the feature comparison result includes training parameters corresponding to the detection network to be trained, and the training parameters are used to determine the degree of training convergence corresponding to the detection network model to be trained;

[0009] When the feature comparison result indicates that the training parameter is greater than or equal to a preset convergence threshold, it is determined that the to-be-trained detection network model is a target detection network model that has completed training.

[0010] As an optional embodiment, in the first aspect of the present invention, the method further comprises:

[0011] When the feature comparison result indicates that the training parameter is less than a preset convergence threshold, repeatedly update the iteration number corresponding to the detection network model to be trained and update the detection network model to be trained according to the iteration number, execute the preset image processing operation on the image to be processed according to the detection network model to be trained, and obtain the corresponding image processing result, and execute the feature comparison operation on the text detection box coordinates and the pre-determined text pre-marked coordinates of the image to be processed according to the determined feature comparison module to obtain the corresponding feature comparison result, until when the feature comparison result indicates that the training parameter is greater than or equal to the preset convergence threshold, determine that the detection network model to be trained is the target detection network model that has completed training.

[0012] As an optional embodiment, in the first aspect of the present invention, performing a preset image processing operation on the image to be processed according to the detection network model to be trained to obtain a corresponding image processing result includes:

[0013] According to the to-be-trained detection network model, a preset feature processing operation is performed on the to-be-processed image to obtain a corresponding feature processing result, wherein the feature processing operation includes a feature extraction operation and a feature fusion operation, and the feature processing result includes a preset number of feature-processed images;

[0014] Inputting each of the feature-processed images into a corresponding preset convolution layer to obtain a convolution result corresponding to each of the feature-processed images, wherein the convolution result corresponding to each of the feature-processed images includes the coordinates of the text detection box corresponding to each of the feature-processed images;

[0015] All the convolution results are determined as image processing results corresponding to the image to be processed.

[0016] As an optional embodiment, in the first aspect of the present invention, before performing a feature comparison operation on the text detection frame coordinates and the predetermined text pre-marked coordinates of the image to be processed based on the determined feature comparison module and obtaining the corresponding feature comparison results, the method further includes:

[0017] Determining a first polygonal area corresponding to the text pre-marked coordinates of the image to be processed according to the predetermined text pre-marked coordinates, and determining a target rectangular area corresponding to the first polygonal area according to the first polygonal area and the text pre-marked coordinates;

[0018] Mapping the text detection frame coordinates to the target rectangular area, and splicing the text detection frame coordinates in the target rectangular area according to a preset connection order to obtain a corresponding second polygonal area;

[0019] and performing a feature comparison operation on the text detection frame coordinates and the predetermined text pre-marked coordinates of the image to be processed according to the determined feature comparison module to obtain a corresponding feature comparison result, including:

[0020] According to the determined feature comparison module, a preset convolution processing operation is performed on the first polygonal area and the second polygonal area in the target rectangular area to obtain a first feature vector corresponding to the first polygonal area and a second feature vector corresponding to the second polygonal area;

[0021] Calculating a target loss value based on the first eigenvector and the second eigenvector in combination with a preset loss function, wherein the target loss value is used to determine the degree of training convergence corresponding to the to-be-trained detection network model;

[0022] The target loss value is determined as a feature comparison result corresponding to the text detection box coordinates and the text pre-marked coordinates.

[0023] As an optional embodiment, in the first aspect of the present invention, before determining the target loss value as a feature comparison result corresponding to the text detection box coordinates and the text pre-marked coordinates, the method further includes:

[0024] Calculate the current accuracy value of the to-be-trained detection network model based on the determined validation data set and the target loss value, wherein the accuracy value is used to determine the degree of training convergence corresponding to the to-be-trained detection network model;

[0025] And determining the target loss value as a feature comparison result corresponding to the text detection box coordinates and the text pre-marked coordinates includes:

[0026] The target loss value and the accuracy value are determined as feature comparison results corresponding to the text detection box coordinates and the text pre-marked coordinates.

[0027] As an optional embodiment, in the first aspect of the present invention, before determining the first polygonal area corresponding to the pre-marked text coordinates of the image to be processed based on the pre-determined pre-marked text coordinates, the method further includes:

[0028] Obtaining an unlabeled test dataset, and performing a preset text annotation operation on the test dataset according to a predetermined text annotation method to obtain a text detection dataset, wherein the text detection dataset is composed of a plurality of labeled images, each of the labeled images including at least one labeled text, and the text detection dataset includes a labeled version corresponding to the image to be processed;

[0029] The step of determining, based on the predetermined text pre-marked coordinates of the image to be processed, a first polygonal area corresponding to the text pre-marked coordinates includes:

[0030] Determining the pre-marked coordinates of the text corresponding to the image to be processed according to the annotated version corresponding to the image to be processed in the text detection dataset;

[0031] According to the determined text pre-marked coordinates, a first polygonal area corresponding to the text pre-marked coordinates is determined.

[0032] As an optional embodiment, in the first aspect of the present invention, before performing a preset image processing operation on the image to be processed according to the to-be-trained detection network model, the method further includes:

[0033] Detecting the image size corresponding to the image to be processed, and determining whether the image size is a preset size, and when the determination result is yes, executing the preset image processing operation on the image to be processed according to the to-be-trained detection network model;

[0034] When the judgment result is no, performing a preset image scaling operation on the image to be processed to update the image to be processed, and performing the preset image processing operation on the image to be processed according to the to-be-trained detection network model, so that the image size corresponding to the updated image to be processed is the preset size;

[0035] After determining all the convolution results as image processing results corresponding to the image to be processed, the method further includes:

[0036] According to the original image size corresponding to the image to be processed, a normalization operation is performed on the image processing result to obtain a corresponding normalized result, and the image processing result is updated according to the normalized result.

[0037] A second aspect of the present invention discloses a device for determining a text image region detection model, the device comprising:

[0038] A construction module is used to construct a to-be-trained detection network model for performing a preset image processing operation on an input image according to a preset detection network construction algorithm, wherein the to-be-trained detection network model includes a feature extraction network, a feature fusion network, and a multi-task branch;

[0039] An image processing module is used to perform a preset image processing operation on the image to be processed according to the detection network model to be trained, and obtain a corresponding image processing result, wherein the image processing result includes the coordinates of the text detection box corresponding to the image to be processed;

[0040] a comparison processing module, configured to perform a feature comparison operation on the coordinates of the text detection frame and the predetermined coordinates of the text pre-mark of the image to be processed according to the determined feature comparison module, to obtain a corresponding feature comparison result, wherein the feature comparison result includes training parameters corresponding to the detection network to be trained, and the training parameters are used to determine the degree of training convergence corresponding to the detection network model to be trained;

[0041] A determination module is used to determine that the detection network model to be trained is a target detection network model that has completed training when the feature comparison result indicates that the training parameter is greater than or equal to a preset convergence threshold.

[0042] As an optional embodiment, in the second aspect of the present invention, the device further includes:

[0043] An iterative training module is used to repeatedly execute the updating of the iteration number corresponding to the to-be-trained detection network model and update the to-be-trained detection network model according to the iteration number when the feature comparison result indicates that the training parameter is less than a preset convergence threshold, trigger the image processing module to repeatedly execute the preset image processing operation on the to-be-trained image to obtain the corresponding image processing result according to the to-be-trained detection network model, and trigger the contrast processing module to execute the feature comparison module determined to perform the feature comparison operation on the text detection box coordinates and the pre-determined text pre-marked coordinates of the to-be-processed image to obtain the corresponding feature comparison result, until when the feature comparison result indicates that the training parameter is greater than or equal to the preset convergence threshold, it is determined that the to-be-trained detection network model is a target detection network model that has completed training.

[0044] As an optional embodiment, in the second aspect of the present invention, the image processing module performs a preset image processing operation on the image to be processed according to the detection network model to be trained, and a method for obtaining a corresponding image processing result specifically includes:

[0045] According to the to-be-trained detection network model, a preset feature processing operation is performed on the to-be-processed image to obtain a corresponding feature processing result, wherein the feature processing operation includes a feature extraction operation and a feature fusion operation, and the feature processing result includes a preset number of feature-processed images;

[0046] Inputting each of the feature-processed images into a corresponding preset convolution layer to obtain a convolution result corresponding to each of the feature-processed images, wherein the convolution result corresponding to each of the feature-processed images includes the coordinates of the text detection box corresponding to each of the feature-processed images;

[0047] All the convolution results are determined as image processing results corresponding to the image to be processed.

[0048] As an optional embodiment, in the second aspect of the present invention, the determination module is further configured to, in the contrast processing module, perform a feature comparison operation on the text detection frame coordinates and the predetermined text pre-marked coordinates of the image to be processed according to the determined feature comparison module, before obtaining the corresponding feature comparison result, determine a first polygonal area corresponding to the text pre-marked coordinates according to the predetermined text pre-marked coordinates of the image to be processed, and determine a target rectangular area corresponding to the first polygonal area according to the first polygonal area and the text pre-marked coordinates;

[0049] The device further comprises:

[0050] a splicing module, configured to map the text detection frame coordinates to the target rectangular area, and splice the text detection frame coordinates in the target rectangular area according to a preset connection order to obtain a corresponding second polygonal area;

[0051] The contrast processing module performs a feature comparison operation on the text detection frame coordinates and the predetermined text pre-marked coordinates of the image to be processed according to the determined feature comparison module, and obtains the corresponding feature comparison result in a manner specifically including:

[0052] According to the determined feature comparison module, a preset convolution processing operation is performed on the first polygonal area and the second polygonal area in the target rectangular area to obtain a first feature vector corresponding to the first polygonal area and a second feature vector corresponding to the second polygonal area;

[0053] Calculating a target loss value based on the first eigenvector and the second eigenvector in combination with a preset loss function, wherein the target loss value is used to determine the degree of training convergence corresponding to the to-be-trained detection network model;

[0054] The target loss value is determined as a feature comparison result corresponding to the text detection box coordinates and the text pre-marked coordinates.

[0055] As an optional embodiment, in the second aspect of the present invention, the device further includes:

[0056] a calculation module, configured to calculate a current accuracy value of the to-be-trained detection network model based on the determined validation data set and the target loss value before the determination module determines the target loss value as a result of feature comparison between the text detection box coordinates and the text pre-marked coordinates, wherein the accuracy value is used to determine a degree of training convergence corresponding to the to-be-trained detection network model;

[0057] The method in which the determination module determines the target loss value as a feature comparison result corresponding to the text detection frame coordinates and the text pre-marked coordinates specifically includes:

[0058] The target loss value and the accuracy value are determined as feature comparison results corresponding to the text detection box coordinates and the text pre-marked coordinates.

[0059] As an optional embodiment, in the second aspect of the present invention, the device further includes:

[0060] an acquisition module, configured to acquire an unlabeled test data set before the determination module determines the first polygonal area corresponding to the text pre-marked coordinates of the image to be processed based on the pre-determined text pre-marked coordinates;

[0061] a text annotation module, configured to perform a preset text annotation operation on the test dataset according to a predetermined text annotation method to obtain a text detection dataset, wherein the text detection dataset is composed of a plurality of annotated images, each of which includes at least one annotated text, and the text detection dataset includes an annotated version corresponding to the image to be processed;

[0062] The method in which the determining module determines the first polygonal area corresponding to the text pre-marked coordinates of the image to be processed according to the predetermined text pre-marked coordinates specifically includes:

[0063] Determining the pre-marked coordinates of the text corresponding to the image to be processed according to the annotated version corresponding to the image to be processed in the text detection dataset;

[0064] According to the determined text pre-marked coordinates, a first polygonal area corresponding to the text pre-marked coordinates is determined.

[0065] As an optional embodiment, in the second aspect of the present invention, the device further includes:

[0066] A detection module, configured to detect an image size corresponding to the image to be processed before the image processing module performs a preset image processing operation on the image to be processed according to the detection network model to be trained;

[0067] a judgment module, configured to judge whether the image size is a preset size, and when the judgment result is yes, trigger the image processing module to execute the preset image processing operation on the image to be processed according to the detection network model to be trained;

[0068] an image transformation module, configured to, when the judgment result is negative, perform a preset image scaling operation on the image to be processed to update the image to be processed, and trigger the image processing module to perform the preset image processing operation on the image to be processed according to the detection network model to be trained, so that the image size corresponding to the updated image to be processed is the preset size;

[0069] And the device also includes:

[0070] a data processing module configured to, after the determination module determines all the convolution results as image processing results corresponding to the image to be processed, perform a normalization operation on the image processing results according to an original image size corresponding to the image to be processed, to obtain a corresponding normalized result;

[0071] The data processing module is further configured to update the image processing result according to the normalization result.

[0072] A third aspect of the present invention discloses another apparatus for determining a text image region detection model, the apparatus comprising:

[0073] a memory storing executable program code;

[0074] a processor coupled to the memory;

[0075] The processor calls the executable program code stored in the memory to execute the method for determining the text image region detection model disclosed in the first aspect of the present invention.

[0076] A fourth aspect of the present invention discloses a computer storage medium storing computer instructions. When the computer instructions are called, they are used to execute the method for determining the text image region detection model disclosed in the first aspect of the present invention.

[0077] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0078] In an embodiment of the present invention, a method for determining a text image area detection model is provided, the method comprising: constructing a to-be-trained detection network model for performing preset image processing operations on an input image according to a preset detection network construction algorithm, the to-be-trained detection network model comprising a feature extraction network, a feature fusion network, and a multi-task branch; performing a preset image processing operation on the to-be-trained image according to the to-be-trained detection network model to obtain a corresponding image processing result, the image processing result comprising text detection box coordinates corresponding to the to-be-processed image; performing a feature comparison operation on the text detection box coordinates and pre-determined text pre-marked coordinates of the to-be-processed image according to a determined feature comparison module to obtain a corresponding feature comparison result, the feature comparison result comprising training parameters corresponding to the to-be-trained detection network, the training parameters being used to determine a degree of training convergence corresponding to the to-be-trained detection network model; and determining that the to-be-trained detection network model is a target detection network model that has completed training when the feature comparison result indicates that the training parameters are greater than or equal to a preset convergence threshold. It can be seen that the implementation of the present invention can automatically construct a detection network model to be trained according to a preset detection network construction algorithm, and intelligently perform corresponding training operations on the detection network model to be trained until the training parameters of the detection network model to be trained are greater than or equal to the preset convergence threshold, and the training is completed to obtain a target detection network model; the trained target detection network model is used to detect and recognize text in the image area, thereby improving the text detection accuracy and recognition accuracy of the image area. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0080] Figure 1 This is a flow chart of a method for determining a text image region detection model disclosed in an embodiment of the present invention;

[0081] Figure 2 This is a flow chart of another method for determining a text image region detection model disclosed in an embodiment of the present invention;

[0082] Figure 3 It is a structural diagram of a device for determining a text image region detection model disclosed in an embodiment of the present invention;

[0083] Figure 4 It is a structural diagram of another apparatus for determining a text image region detection model disclosed in an embodiment of the present invention;

[0084] Figure 5 It is a structural diagram of another apparatus for determining a text image region detection model disclosed in an embodiment of the present invention;

[0085] Figure 6 This is a schematic diagram of an application of detecting polygonal text boxes in a text image area disclosed in an embodiment of the present invention;

[0086] Figure 7 This is a schematic diagram of an application for determining the minimum rectangular area of ​​a polygonal text box disclosed in an embodiment of the present invention;

[0087] Figure 8 This is a schematic diagram of an application for processing a minimum rectangular area disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0088] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0089] The terms "first," "second," and so on, in the description and claims of the present invention and the accompanying drawings are used to distinguish between different items, not to describe a specific order. Furthermore, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product, or end comprising a series of steps or elements is not limited to the listed steps or elements but may optionally include steps or elements not listed therein, or may optionally include other steps or elements inherent to such process, method, product, or end.

[0090] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present invention. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute a separate or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0091] The present invention discloses a method and device for determining a text image region detection model. The method automatically constructs a to-be-trained detection network model based on a preset detection network construction algorithm and intelligently performs corresponding training operations on the to-be-trained detection network model until the training parameters of the to-be-trained detection network model are greater than or equal to a preset convergence threshold, completing the training and obtaining a target detection network model. The trained target detection network model is used to detect and recognize text in image regions, thereby improving the accuracy of text detection and recognition in image regions. These are described in detail below.

[0092] Example 1

[0093] See also Figure 1 , Figure 1 This is a flow chart of a method for determining a text image region detection model disclosed in an embodiment of the present invention. Figure 1 The method for determining a text image region detection model described above can be applied to a device for determining a text image region detection model, and the embodiment of the present invention does not limit this. Figure 1 As shown, the method for determining the text image region detection model may include the following operations:

[0094] 101. According to a preset detection network construction algorithm, a detection network model to be trained is constructed for performing a preset image processing operation on an input image.

[0095] In an embodiment of the present invention, the detection network model to be trained includes a feature extraction network, a feature fusion network and a multi-task branch, wherein an ultra-lightweight YOLOv5 network can be used as the backbone feature extraction network, and the feature fusion network can adopt a PAN structure or an FPN structure as the feature fusion network. The feature fusion network can be used to fuse feature maps of different resolutions. The multi-task branch is used to perform multi-task synchronous processing on several images obtained after processing by the feature extraction network and the feature fusion network. When the multi-task synchronous processing operation is specifically applied, a convolution processing operation can be performed through a preset convolution layer, that is, the fused feature map is input into a subsequent preset convolution layer to output the text detection box coordinates; wherein, the preset convolution layer can be a 1*1 convolution layer, which is not limited in the embodiment of the present invention.

[0096] 102. According to the detection network model to be trained, perform a preset image processing operation on the image to be processed to obtain a corresponding image processing result.

[0097] In the embodiment of the present invention, the image processing result includes the coordinates of the text detection box corresponding to the image to be processed.

[0098] In the embodiment of the present invention, step 102 performs a preset image processing operation on the image to be processed according to the detection network model to be trained, and the method of obtaining the corresponding image processing result may specifically include the following operations:

[0099] According to the detection network model to be trained, a preset feature processing operation is performed on the image to be processed to obtain a corresponding feature processing result, wherein the feature processing operation includes a feature extraction operation and a feature fusion operation, and the feature processing result includes a preset number of feature-processed images;

[0100] Input each feature-processed image into the corresponding preset convolution layer to obtain the convolution result corresponding to each feature-processed image. The convolution result corresponding to each feature-processed image includes the coordinates of the text detection box corresponding to each feature-processed image.

[0101] All convolution results are determined as image processing results corresponding to the image to be processed.

[0102] It can be seen that in the embodiment of the present invention, the training of the detection network model to be trained is achieved through preset feature extraction, feature fusion and convolution processing operations. At the same time, the different feature-processed images included in the above-mentioned feature processing results correspond to different resolutions, which can adapt to different detection requirements, and are conducive to improving the detection accuracy and detection success rate for texts of different sizes.

[0103] 103. According to the determined feature comparison module, perform a feature comparison operation on the coordinates of the text detection frame and the predetermined coordinates of the text pre-mark of the image to be processed to obtain a corresponding feature comparison result.

[0104] In an embodiment of the present invention, the determined feature comparison module may include a feature comparison result including training parameters corresponding to the to-be-trained detection network, the training parameters being used to determine a degree of training convergence corresponding to the to-be-trained detection network model;

[0105] 104. When the feature comparison result indicates that the training parameter is greater than or equal to a preset convergence threshold, the detection network model to be trained is determined to be a target detection network model that has completed training.

[0106] It can be seen that implementation Figure 1 The described method for determining the text image area detection model can automatically construct a detection network model to be trained according to a preset detection network construction algorithm, and intelligently perform corresponding training operations on the detection network model to be trained until the training parameters of the detection network model to be trained are greater than or equal to a preset convergence threshold, and the training is completed to obtain a target detection network model; the trained target detection network model is used to detect and recognize text in the image area, thereby improving the text detection accuracy and recognition accuracy of the image area.

[0107] In an optional embodiment, the method may further include the following operations:

[0108] When the feature comparison result indicates that the training parameters are less than the preset convergence threshold, repeatedly execute the iteration number corresponding to the update of the detection network model to be trained and update the detection network model to be trained according to the iteration number, execute the above-mentioned image processing operation based on the detection network model to be trained, and obtain the corresponding image processing result, and execute the above-mentioned feature comparison module determined according to the determination, perform the feature comparison operation on the text detection box coordinates and the pre-determined text pre-marked coordinates of the image to be processed, and obtain the corresponding feature comparison result, until the feature comparison result indicates that the training parameters are greater than or equal to the preset convergence threshold, determine that the detection network model to be trained is the target detection network model that has completed training.

[0109] It can be seen that in this optional embodiment, when the training parameters of the detection network model to be trained are less than the preset convergence threshold, the number of iterations of the detection network model to be trained can be automatically updated. At the same time, the detection network model to be trained will be updated in each iteration, and the previous training steps will be repeated until the training parameters are greater than or equal to the preset convergence threshold. The training is completed to obtain the target detection network model, which realizes the closed-loop iterative training of the detection network model to be trained before the training is completed, thereby improving the training success rate of the target detection network model.

[0110] In this optional embodiment, further optionally, when the feature comparison result indicates that the training parameter is less than a preset convergence threshold, after updating the number of iterations corresponding to the to-be-trained detection network model and updating the to-be-trained detection network model according to the number of iterations, the method may further include the following operations:

[0111] Determine whether the number of iterations corresponding to the detection network model to be trained is greater than a preset number. When the judgment result is no, perform the above-mentioned repeated execution of the preset image processing operation on the image to be processed according to the detection network model to be trained to obtain the corresponding image processing result, and execute the operation of performing a feature comparison operation on the text detection box coordinates and the predetermined text pre-marked coordinates of the image to be processed according to the determined feature comparison module to obtain the corresponding feature comparison result, until when the feature comparison result indicates that the training parameter is greater than or equal to a preset convergence threshold, determine that the detection network model to be trained is a target detection network model that has completed training;

[0112] When the judgment result is yes, the training parameters of each training of the to-be-trained detection network model are obtained, and all the training parameters are analyzed to obtain iterative change data corresponding to all the training parameters;

[0113] Determine whether the iteration conditions corresponding to all training parameters match the predetermined expected iteration change data. If the determination result is negative, determine that the to-be-trained detection network is a detection network that has failed training, and generate elimination information for the detection network that has failed training based on the iteration change data. The elimination information is provided to the processing personnel responsible for the to-be-trained detection network model so that the processing personnel can adjust the training strategy of the to-be-trained network model based on the elimination information.

[0114] When it is determined that the iteration conditions corresponding to all training parameters match the predetermined expected iteration change data, the above-mentioned steps of repeatedly executing the preset image processing operation on the image to be processed according to the detection network model to be trained to obtain the corresponding image processing result, and executing the step of executing the feature comparison module according to the determination to perform the feature comparison operation on the text detection box coordinates and the predetermined text pre-marked coordinates of the image to be processed to obtain the corresponding feature comparison result, are performed until the feature comparison result indicates that the training parameters are greater than or equal to the preset convergence threshold, and determining that the detection network model to be trained is the target detection network model that has completed training.

[0115] It can be seen that in this optional embodiment, an iteration number determination mechanism is set up, which is conducive to reducing the situation where the number of iterations of a certain detection network to be trained is too many, but the convergence of the detection network to be trained does not match the expected convergence (that is, it cannot be successfully trained), and resources are still occupied for training, thereby improving the efficiency of discovering training failures of the detection network to be trained.

[0116] In another optional embodiment, before performing the preset image processing operation on the image to be processed according to the detection network model to be trained, the method may further include the following operations:

[0117] Detect the image size corresponding to the image to be processed and determine whether the image size is a preset size. If the judgment result is yes, execute the above-mentioned detection network model to be trained to perform the preset image processing operation on the image to be processed;

[0118] When the judgment result is no, a preset image scaling operation is performed on the image to be processed to update the image to be processed, and the preset image processing operation is performed on the image to be processed according to the to-be-trained detection network model, and the image size corresponding to the updated image to be processed is the preset size;

[0119] After all the convolution results are determined as image processing results corresponding to the image to be processed, the method further includes:

[0120] According to the original image size corresponding to the image to be processed, a normalization operation is performed on the image processing result to obtain a corresponding normalized result, and the image processing result is updated according to the normalized result.

[0121] In this optional embodiment, for ease of understanding, the following examples are given for illustration:

[0122] Assuming that the size of the input image is 640*640, the three output feature map sizes will be downsampled by 8 times, 16 times, and 32 times respectively to obtain 80*80, 40*40, and 20*20.

[0123] The largest feature map, 80*80, is usually used to detect small targets. The number of output channels of the corresponding convolutional layer is (20+20+k+2+1)=43+k, where k represents the prediction confidence of k categories, 20+20 represents the width and height offset values ​​of 20 contour coordinate points relative to the center point, 2 represents the x and y coordinate values ​​of the center point, and 1 represents the probability that the target is an object.

[0124] The second largest feature map, that is, 40*40, has a corresponding convolutional layer output channel number of (30+30+k+2+1)=63+k, where 30+30 represents the width and height offset values ​​of the 30 contour coordinate points relative to the center point, and the rest is the same as above.

[0125] The smallest feature map is 20*20. Small feature maps have stronger semantic information, but also lack fine contour information. Therefore, they are often used to detect larger targets. The number of output channels of the corresponding convolutional layer is (50+50+k+2+1)=103+k, where 50+50 represents the width and height offset values ​​of the 50 contour coordinate points relative to the center point. The rest is the same as above.

[0126] It should be noted that the above description can be summarized as: (bs, 3, 80, 80, 44), where bs represents the batch size during training and can be adjusted according to the hardware device. Since this task corresponds to a text detection task, the number of categories is set to 1; 3 represents three preset anchor boxes of different sizes, 80 represents the feature map size, and 44 represents the number of output channels. 44 includes 20 x and 20 y coordinates, 2 center point x and y coordinates, 1 category probability, and 1 target probability. That is, an RGB three-channel image includes 80*80 feature blocks, each of which corresponds to 2 coordinate values ​​(x, y). When the prediction confidence k is 1, the 2 x, y coordinates and other contour coordinates are stacked together to obtain 20+20+k+2+1=44; or it can be simply understood that a feature map is divided into 80*80 grids, each grid corresponds to three anchor boxes of different sizes, and each anchor box corresponds to 44 coordinate values, totaling 3*44=132 values.

[0127] It should be noted that the large targets and small targets mentioned above do not have a particularly specific definition. For example, if the original input image size has a resolution of 640*640, then when the target area is less than 32*32, it can be determined as a small target, or when the length / width of the target is less than one twentieth of the length / width of the original image, it can be determined as a small target. The embodiments of the present invention do not limit this.

[0128] In this optional embodiment, in order to accelerate the training convergence of the model, all the above image output results will be normalized, wherein the coordinate center point is normalized by dividing by the original size of the image, and the value range is (0,1); the offset value of the predicted contour coordinate point (text detection box coordinate) relative to the center point is normalized by dividing by the original size of the image, wherein when the x coordinate is to the left of the center point or the y coordinate is below the center point, its value range is (-1,0), and when the x coordinate is to the right of the center point and the y coordinate is above the center point, its value range is (0,1). K represents the prediction confidence of k categories, with a value range of (0,1), and 1 represents the probability that the target is an object, with a value range of (0,1).

[0129] It can be seen that in this optional embodiment, for images to be processed of different image sizes, image scaling operations can be performed intelligently to scale the images to be processed to images of a uniform preset size, thereby improving the accuracy of subsequent convolution operations; in addition, normalization operations can also be performed on the image processing results, which is beneficial to improving the training convergence speed of the detection network model to be trained.

[0130] Example 2

[0131] See also Figure 2 , Figure 2This is a flow chart of another method for determining a text image region detection model disclosed in an embodiment of the present invention. Figure 2 The method for determining a text image region detection model described above can be applied to a device for determining a text image region detection model, and the embodiment of the present invention does not limit this. Figure 2 As shown, the method for determining the text image region detection model may include the following operations:

[0132] 201. According to a preset detection network construction algorithm, a detection network model to be trained is constructed for performing a preset image processing operation on an input image.

[0133] 202. According to the detection network model to be trained, perform a preset image processing operation on the image to be processed to obtain a corresponding image processing result.

[0134] 203 . Determine a first polygonal area corresponding to the text pre-marked coordinates according to the predetermined text pre-marked coordinates of the image to be processed.

[0135] 204. Determine a target rectangular area corresponding to the first polygonal area according to the first polygonal area and the pre-marked text coordinates.

[0136] 205 . Map the coordinates of the text detection frame to the target rectangular area, and splice the coordinates of the text detection frame in the target rectangular area according to a preset connection order to obtain a corresponding second polygonal area.

[0137] 206. According to the determined feature comparison module, perform preset convolution processing operations on the first polygonal area and the second polygonal area in the target rectangular area respectively to obtain a first feature vector corresponding to the first polygonal area and a second feature vector corresponding to the second polygonal area.

[0138] 207. Calculate a target loss value based on the first eigenvector and the second eigenvector in combination with a preset loss function. The target loss value is used to determine the degree of training convergence corresponding to the to-be-trained detection network model.

[0139] 208. Determine the target loss value as a feature comparison result corresponding to the coordinates of the text detection box and the coordinates of the text pre-marker.

[0140] 209. When the feature comparison result indicates that the training parameter is greater than or equal to a preset convergence threshold, the detection network model to be trained is determined to be a target detection network model that has completed training.

[0141] In the embodiment of the present invention, for descriptions of steps 201-202 and 209, as well as other steps, please refer to the other specific descriptions of steps 101-102 and 104 in the first embodiment, which will not be repeated in the embodiment of the present invention.

[0142] In an embodiment of the present invention, the target rectangular area is determined by the determined text pre-marked coordinates, so that the above-mentioned text detection box coordinates are spliced ​​into a second polygonal area in the target rectangular area, and the target loss value is calculated by comparing the feature vectors corresponding to the text pre-marked coordinates and the text detection box coordinates. The target loss value is used to determine the training convergence degree of the detection network model to be trained.

[0143] In the embodiment of the present invention, please refer to Figure 6-Figure 8 , Figure 6-Figure 8 This is a flow chart of cutting out the text polygon area in the image area in an embodiment of the present invention. In actual application, the pre-marked polygon box will cut out the minimum rectangular area (target rectangular box) where the corresponding polygon area is located on the original image, and then the value within the polygon area is equal to the pixel value of the original image, and the value outside the polygon is set to 0, which is considered to be the background. Similarly, the positive sample box predicted by the polygon branch of the model (that is, the second polygon area obtained by splicing the text detection coordinates) will also be mapped to the minimum rectangular area cut out by the corresponding real box, and then the polygon coordinates predicted by the model will be connected in sequence into a polygon, so that the value within the polygon area remains unchanged, and the value outside the polygon is set to 0, which is considered to be the background. The reason for mapping to the same minimum rectangular area is to align the image features and facilitate the calculation of losses.

[0144] The above two image areas (the minimum rectangular area corresponding to the first polygonal area and the minimum rectangular area corresponding to the second polygonal area) will pass through the same feature comparison module respectively. This module consists of three layers of 3×3 convolutional layers, gradually reducing the image size, expanding the number of channels, and finally converting the feature map output by each convolutional layer into a feature vector, which is then spliced ​​in the channel dimension. The spliced ​​feature vector is used to calculate the loss. The loss function uses the optimized CIOU+SmoothL1 loss method to calculate the image feature loss between the image area corresponding to the first polygonal box and the second polygonal area. By minimizing this loss, the polygon coordinates are predicted by regression, thereby bringing the two polygonal areas closer.

[0145] Through this training method, the three tasks of detection, classification, and segmentation can be coupled together, and the prediction of coordinates and segmentation results can be completed at one time through regression. This not only speeds up the training process and reduces the model size, but also enhances the model's learning ability by utilizing multi-task features.

[0146] It can be seen that implementation Figure 2The described method for determining the text image area detection model accurately calculates the feature loss between the area where the detected text is located and the area where the marked text is located by mapping the coordinates of the text detection box to the target rectangular area corresponding to the text pre-marked coordinates, thereby improving the accuracy of the determined loss value and, to a certain extent, improving the accuracy and reliability of the training convergence degree of the determined detection network model to be trained.

[0147] In an optional embodiment, before determining the target loss value as a feature comparison result corresponding to the coordinates of the text detection box and the coordinates of the pre-marked text, the method may further include the following operations:

[0148] Based on the determined validation data set and target loss value, the current accuracy value of the to-be-trained detection network model is calculated. The accuracy value is used to determine the degree of training convergence corresponding to the to-be-trained detection network model.

[0149] The above-mentioned method of determining the target loss value as the feature comparison result corresponding to the coordinates of the text detection box and the coordinates of the pre-marked text specifically includes:

[0150] The target loss value and the precision value are determined as the feature comparison results corresponding to the coordinates of the text detection box and the coordinates of the text pre-marked.

[0151] It can be seen that in this optional embodiment, the network accuracy of the to-be-trained detection network model can be further calculated by verifying the data set as a parameter for determining the degree of training convergence of the to-be-trained detection network model, thereby further improving the reliability and accuracy of the determined degree of training convergence of the network model.

[0152] In another optional embodiment, before determining the first polygonal area corresponding to the text pre-marked coordinates of the image to be processed in step 203, the method may further include the following operations:

[0153] Obtain an unlabeled test dataset and perform a preset text annotation operation on the test dataset according to a predetermined text annotation method to obtain a text detection dataset, wherein the text detection dataset is composed of a plurality of labeled images, each labeled image including at least one labeled text, and the text detection dataset includes a labeled version corresponding to the image to be processed;

[0154] Determining a first polygonal area corresponding to the text pre-marked coordinates of the image to be processed according to the predetermined text pre-marked coordinates includes:

[0155] Determine the pre-labeled text coordinates corresponding to the image to be processed based on the labeled version corresponding to the image to be processed in the text detection dataset;

[0156] According to the determined text pre-marked coordinates, a first polygonal area corresponding to the text pre-marked coordinates is determined.

[0157] In this optional embodiment, the above-mentioned text annotation method is similar to the annotation method in instance segmentation, and a sentence-level polygon annotation method is adopted for the test data set: starting from the upper left corner of the sentence, annotate in clockwise order. There is no limit on the number of coordinate points annotated in a paragraph of text, and the category can be uniformly marked as text.

[0158] It can be seen that in this optional embodiment, the test data set is pre-annotated by a preset text annotation method to obtain multiple annotated images, thereby improving the accuracy of the determined text pre-marked coordinates.

[0159] Example 3

[0160] See also Figure 3 , Figure 3 It is a structural diagram of a device for determining a text image region detection model disclosed in an embodiment of the present invention. The device for determining the text image region detection model can be a terminal, device, system or server for determining the text image region detection model. The server for determining the text image region detection model can be a local server, a remote server or a cloud server (also known as a cloud server). When the server for determining the text image region detection model is a non-cloud server, the non-cloud server can communicate with the cloud server, which is not limited in the embodiment of the present invention. Figure 3 As shown, the apparatus for determining the text image region detection model may include a construction module 301, an image processing module 302, a comparison processing module 303, and a determination module 304, wherein:

[0161] The construction module 301 is used to construct a detection network model to be trained for performing preset image processing operations on the input image according to a preset detection network construction algorithm. The detection network model to be trained includes a feature extraction network, a feature fusion network and a multi-task branch.

[0162] The image processing module 302 is used to perform a preset image processing operation on the image to be processed according to the detection network model to be trained to obtain a corresponding image processing result, which includes the coordinates of the text detection box corresponding to the image to be processed.

[0163] The comparison processing module 303 is used to perform a feature comparison operation on the coordinates of the text detection box and the pre-determined text pre-marked coordinates of the image to be processed according to the determined feature comparison module, and obtain corresponding feature comparison results. The feature comparison results include training parameters corresponding to the detection network to be trained, and the training parameters are used to determine the degree of training convergence corresponding to the detection network model to be trained.

[0164] The determination module 304 is used to determine that the to-be-trained detection network model is a target detection network model that has completed training when the feature comparison result indicates that the training parameter is greater than or equal to a preset convergence threshold.

[0165] It can be seen that implementation Figure 3 The described device for determining the text image area detection model can automatically construct a detection network model to be trained according to a preset detection network construction algorithm, and intelligently perform corresponding training operations on the detection network model to be trained until the training parameters of the detection network model to be trained are greater than or equal to a preset convergence threshold, and the training is completed to obtain a target detection network model; the trained target detection network model is used to detect and recognize text in the image area, thereby improving the text detection accuracy and recognition accuracy of the image area.

[0166] In another alternative embodiment, see Figure 4 , Figure 4 3 is a schematic structural diagram of another apparatus for determining a text image region detection model disclosed in an embodiment of the present invention. The apparatus further includes an iterative training module 305, wherein:

[0167] The iterative training module 305 is used to repeatedly execute the iteration number corresponding to the update of the detection network model to be trained and update the detection network model to be trained according to the iteration number when the feature comparison result indicates that the training parameters are less than the preset convergence threshold, trigger the image processing module 302 to repeatedly execute the above-mentioned operation according to the detection network model to be trained, perform the preset image processing operation on the image to be processed, and obtain the corresponding image processing result, and trigger the comparison processing module 303 to execute the above-mentioned operation according to the determined feature comparison module, perform the feature comparison operation on the text detection box coordinates and the pre-determined text pre-marked coordinates of the image to be processed, and obtain the corresponding feature comparison result, until the feature comparison result indicates that the training parameters are greater than or equal to the preset convergence threshold, and determine that the detection network model to be trained is the target detection network model that has completed training.

[0168] It can be seen that implementation Figure 4 The described device for determining the text image area detection model can automatically update the number of iterations of the detection network model to be trained when the training parameters of the detection network model to be trained are less than a preset convergence threshold. At the same time, the detection network model to be trained will be updated at each iteration, and the previous training steps will be repeated until the training parameters are greater than or equal to the preset convergence threshold. The training is completed to obtain the target detection network model, thereby realizing closed-loop iterative training of the detection network model to be trained before the training is completed, and improving the training success rate of the target detection network model.

[0169] In another optional embodiment, the image processing module 302 performs a preset image processing operation on the image to be processed according to the detection network model to be trained, and obtains the corresponding image processing result in the following manner:

[0170] According to the detection network model to be trained, a preset feature processing operation is performed on the image to be processed to obtain a corresponding feature processing result, wherein the feature processing operation includes a feature extraction operation and a feature fusion operation, and the feature processing result includes a preset number of feature-processed images;

[0171] Input each feature-processed image into the corresponding preset convolution layer to obtain the convolution result corresponding to each feature-processed image. The convolution result corresponding to each feature-processed image includes the coordinates of the text detection box corresponding to each feature-processed image.

[0172] All convolution results are determined as image processing results corresponding to the image to be processed.

[0173] It can be seen that in this optional embodiment, the training of the detection network model to be trained is achieved through preset feature extraction, feature fusion and convolution processing operations. At the same time, the different feature-processed images included in the above-mentioned feature processing results correspond to different resolutions, which can adapt to different detection requirements, and is conducive to improving the detection accuracy and detection success rate for texts of different sizes.

[0174] In another optional embodiment, the determination module 304 is also used to perform a feature comparison operation on the text detection box coordinates and the text pre-marked coordinates of the image to be processed determined in advance according to the feature comparison module determined by the comparison processing module 303, and before obtaining the corresponding feature comparison result, determine the first polygonal area corresponding to the text pre-marked coordinates according to the text pre-marked coordinates of the image to be processed determined in advance, and determine the target rectangular area corresponding to the first polygonal area according to the first polygonal area and the text pre-marked coordinates.

[0175] like Figure 4 As shown, the apparatus may further include a splicing module 306, wherein:

[0176] The splicing module 306 is configured to map the text detection frame coordinates obtained by the image processing module 302 to the target rectangular area, and splice the text detection frame coordinates in the target rectangular area according to a preset connection order to obtain a corresponding second polygonal area.

[0177] The comparison processing module 303 performs a feature comparison operation on the text detection frame coordinates and the predetermined text pre-marked coordinates of the image to be processed according to the determined feature comparison module, and obtains the corresponding feature comparison result in the following manner:

[0178] According to the determined feature comparison module, a preset convolution processing operation is performed on the first polygonal area and the second polygonal area in the target rectangular area respectively to obtain a first feature vector corresponding to the first polygonal area and a second feature vector corresponding to the second polygonal area;

[0179] A target loss value is calculated based on the first eigenvector and the second eigenvector in combination with a preset loss function. The target loss value is used to determine the degree of training convergence corresponding to the to-be-trained detection network model;

[0180] The target loss value is determined as the feature comparison result corresponding to the coordinates of the text detection box and the coordinates of the text pre-marked.

[0181] It can be seen that implementation Figure 4 The described device for determining the text image area detection model accurately calculates the feature loss between the area where the detected text is located and the area where the marked text is located by mapping the text detection box coordinates to the target rectangular area corresponding to the text pre-marked coordinates, thereby improving the accuracy of the determined loss value and, to a certain extent, improving the accuracy and reliability of the training convergence degree of the determined detection network model to be trained.

[0182] In another optional embodiment, Figure 4 As shown, the apparatus may further include a calculation module 307, wherein:

[0183] The calculation module 307 is used to calculate the current accuracy value of the detection network model to be trained based on the determined verification data set and the target loss value before the determination module 304 determines the target loss value as the feature comparison result corresponding to the text detection box coordinates and the text pre-marked coordinates. The accuracy value is used to determine the training convergence degree corresponding to the detection network model to be trained.

[0184] The method in which the determination module 304 determines the target loss value as the feature comparison result corresponding to the text detection box coordinates and the text pre-marked coordinates specifically includes:

[0185] The target loss value and the precision value are determined as the feature comparison results corresponding to the coordinates of the text detection box and the coordinates of the text pre-marked.

[0186] It can be seen that implementation Figure 4 The described device for determining the text image area detection model can further calculate the network accuracy of the detection network model to be trained through a verification data set, which serves as a parameter for determining the training convergence degree of the detection network model to be trained, thereby further improving the reliability and accuracy of the determined training convergence degree of the network model.

[0187] In another optional embodiment, as Figure 4As shown, the device further includes an acquisition module 308 and a text annotation module 309, wherein:

[0188] The acquisition module 308 is configured to acquire an unlabeled test data set before the determination module 304 determines the first polygonal area corresponding to the text pre-labeled coordinates of the image to be processed according to the predetermined text pre-labeled coordinates.

[0189] The text annotation module 309 is used to perform a preset text annotation operation on the test data set according to a predetermined text annotation method to obtain a text detection data set. The text detection data set consists of a number of annotated images, each annotated image includes at least one annotated text, and the text detection data set includes an annotated version corresponding to the image to be processed.

[0190] The determining module 304 determines the first polygonal area corresponding to the text pre-marked coordinates according to the predetermined text pre-marked coordinates of the image to be processed, specifically including:

[0191] Determine the pre-labeled text coordinates corresponding to the image to be processed based on the labeled version corresponding to the image to be processed in the text detection dataset;

[0192] According to the determined text pre-marked coordinates, a first polygonal area corresponding to the text pre-marked coordinates is determined.

[0193] It can be seen that implementation Figure 4 The described device for determining a text image region detection model pre-annotates a test data set using a preset text annotation method to obtain multiple annotated images, thereby improving the accuracy of the determined text pre-marked coordinates.

[0194] In another optional embodiment, Figure 4 As shown, the device may further include a detection module 310, a judgment module 311 and a data processing module 312, wherein:

[0195] The detection module 310 is used to detect the image size corresponding to the image to be processed before the image processing module 302 performs a preset image processing operation on the image to be processed according to the detection network model to be trained.

[0196] The judgment module 311 is used to judge whether the image size is a preset size. When the judgment result is yes, the image processing module 302 is triggered to execute the above-mentioned preset image processing operation on the image to be processed based on the detection network model to be trained.

[0197] The image processing module 302 is further configured to, when the judgment result is negative, perform a preset image scaling operation on the image to be processed to update the image to be processed, and perform the above-mentioned preset image processing operation on the image to be processed based on the detection network model to be trained, and the image size corresponding to the updated image to be processed is the preset size.

[0198] And the device also includes:

[0199] The data processing module 312 is used to perform a normalization operation on the image processing results according to the original image size corresponding to the image to be processed after the determination module 304 determines all the convolution results as the image processing results corresponding to the image to be processed, so as to obtain the corresponding normalized results.

[0200] The data processing module 312 is further configured to update the image processing result according to the normalization result.

[0201] It can be seen that implementation Figure 4 The described device for determining a text image region detection model can intelligently perform image scaling operations for images to be processed of different sizes, so that the images to be processed are scaled to images of a uniform preset size, thereby improving the accuracy of subsequent convolution operations; in addition, it can also perform normalization processing operations on the image processing results, which is beneficial to improving the training convergence speed of the detection network model to be trained.

[0202] Example 4

[0203] See also Figure 5 , Figure 5 This is a structural diagram of another apparatus for determining a text image region detection model disclosed in an embodiment of the present invention. Figure 5 As shown, the determination device of the text image area detection model may include:

[0204] A memory 401 storing executable program code;

[0205] a processor 402 coupled to the memory 401;

[0206] The processor 402 calls the executable program code stored in the memory 401 to execute the steps of the method for determining the text image region detection model described in the first embodiment or the second embodiment of the present invention.

[0207] Example 5

[0208] An embodiment of the present invention discloses a computer storage medium storing computer instructions. When the computer instructions are called, they are used to execute the steps in the method for determining a text image region detection model described in Embodiment 1 or Embodiment 2 of the present invention.

[0209] Example 6

[0210] An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer storage medium storing a computer program, and the computer program is operable to enable a computer to execute the steps in the method for determining a text image region detection model described in Example 1 or Example 2.

[0211] The device embodiments described above are merely illustrative, wherein the modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, i.e., they may be located in one place or distributed across multiple network modules. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Those skilled in the art can understand and implement the present invention without inventive effort.

[0212] Through the detailed description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus the necessary general hardware platform, or of course, by means of hardware. Based on this understanding, the above technical solution, in essence, or the portion that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a computer storage medium, including a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0213] Finally, it should be noted that the method and device for determining a text image region detection model disclosed in the embodiments of the present invention are only preferred embodiments of the present invention, and are only used to illustrate the technical solutions of the present invention, rather than to limit them. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for determining a text image region detection model, characterized in that: The method comprises: According to a preset detection network construction algorithm, a to-be-trained detection network model is constructed for performing a preset image processing operation on an input image, wherein the to-be-trained detection network model includes a feature extraction network, a feature fusion network, and a multi-task branch; Performing a preset image processing operation on the image to be processed according to the detection network model to be trained to obtain a corresponding image processing result, wherein the image processing result includes the coordinates of the text detection box corresponding to the image to be processed; According to the determined feature comparison module, a feature comparison operation is performed on the coordinates of the text detection box and the predetermined coordinates of the text pre-marked of the image to be processed to obtain a corresponding feature comparison result, wherein the feature comparison result includes training parameters corresponding to the detection network to be trained, and the training parameters are used to determine the degree of training convergence corresponding to the detection network model to be trained; When the feature comparison result indicates that the training parameter is greater than or equal to a preset convergence threshold, determining that the to-be-trained detection network model is a target detection network model that has completed training; Before performing a feature comparison operation on the text detection frame coordinates and the predetermined text pre-marked coordinates of the image to be processed according to the determined feature comparison module to obtain a corresponding feature comparison result, the method further includes: Determining a first polygonal area corresponding to the text pre-marked coordinates of the image to be processed according to the predetermined text pre-marked coordinates, and determining a target rectangular area corresponding to the first polygonal area according to the first polygonal area and the text pre-marked coordinates; Mapping the text detection frame coordinates to the target rectangular area, and splicing the text detection frame coordinates in the target rectangular area according to a preset connection order to obtain a corresponding second polygonal area; and performing a feature comparison operation on the text detection frame coordinates and the predetermined text pre-marked coordinates of the image to be processed according to the determined feature comparison module to obtain a corresponding feature comparison result, including: According to the determined feature comparison module, a preset convolution processing operation is performed on the first polygonal area and the second polygonal area in the target rectangular area to obtain a first feature vector corresponding to the first polygonal area and a second feature vector corresponding to the second polygonal area; Calculating a target loss value based on the first eigenvector and the second eigenvector in combination with a preset loss function, wherein the target loss value is used to determine the degree of training convergence corresponding to the to-be-trained detection network model; The target loss value is determined as a feature comparison result corresponding to the text detection box coordinates and the text pre-marked coordinates.

2. The method for determining a text image region detection model according to claim 1, wherein: The method further comprises: When the feature comparison result indicates that the training parameter is less than a preset convergence threshold, repeatedly update the iteration number corresponding to the detection network model to be trained and update the detection network model to be trained according to the iteration number, execute the preset image processing operation on the image to be processed according to the detection network model to be trained, and obtain the corresponding image processing result, and execute the feature comparison operation on the text detection box coordinates and the pre-determined text pre-marked coordinates of the image to be processed according to the determined feature comparison module to obtain the corresponding feature comparison result, until when the feature comparison result indicates that the training parameter is greater than or equal to the preset convergence threshold, determine that the detection network model to be trained is the target detection network model that has completed training.

3. The method for determining a text image region detection model according to claim 2, wherein: The method of performing a preset image processing operation on the image to be processed according to the detection network model to be trained to obtain a corresponding image processing result includes: According to the to-be-trained detection network model, a preset feature processing operation is performed on the to-be-processed image to obtain a corresponding feature processing result, wherein the feature processing operation includes a feature extraction operation and a feature fusion operation, and the feature processing result includes a preset number of feature-processed images; Inputting each of the feature-processed images into a corresponding preset convolution layer to obtain a convolution result corresponding to each of the feature-processed images, wherein the convolution result corresponding to each of the feature-processed images includes the coordinates of the text detection box corresponding to each of the feature-processed images; All the convolution results are determined as image processing results corresponding to the image to be processed.

4. The method for determining a text image region detection model according to claim 1, wherein: Before determining the target loss value as a feature comparison result corresponding to the text detection frame coordinates and the text pre-marked coordinates, the method further includes: Calculate the current accuracy value of the to-be-trained detection network model based on the determined validation data set and the target loss value, wherein the accuracy value is used to determine the degree of training convergence corresponding to the to-be-trained detection network model; And determining the target loss value as a feature comparison result corresponding to the text detection box coordinates and the text pre-marked coordinates includes: The target loss value and the accuracy value are determined as feature comparison results corresponding to the text detection box coordinates and the text pre-marked coordinates.

5. The method for determining a text image region detection model according to claim 1, wherein: Before determining the first polygonal area corresponding to the text pre-marked coordinates of the image to be processed according to the predetermined text pre-marked coordinates, the method further includes: Obtaining an unlabeled test dataset, and performing a preset text annotation operation on the test dataset according to a predetermined text annotation method to obtain a text detection dataset, wherein the text detection dataset is composed of a plurality of labeled images, each of the labeled images including at least one labeled text, and the text detection dataset includes a labeled version corresponding to the image to be processed; The step of determining, based on the predetermined text pre-marked coordinates of the image to be processed, a first polygonal area corresponding to the text pre-marked coordinates includes: Determining the pre-marked coordinates of the text corresponding to the image to be processed according to the annotated version corresponding to the image to be processed in the text detection dataset; According to the determined text pre-marked coordinates, a first polygonal area corresponding to the text pre-marked coordinates is determined.

6. The method for determining a text image region detection model according to claim 3, wherein: Before performing a preset image processing operation on the image to be processed according to the to-be-trained detection network model, the method further comprises: Detecting the image size corresponding to the image to be processed, and determining whether the image size is a preset size, and when the determination result is yes, executing the preset image processing operation on the image to be processed according to the to-be-trained detection network model; When the judgment result is no, performing a preset image scaling operation on the image to be processed to update the image to be processed, and performing the preset image processing operation on the image to be processed according to the to-be-trained detection network model, so that the image size corresponding to the updated image to be processed is the preset size; After determining all the convolution results as image processing results corresponding to the image to be processed, the method further includes: According to the original image size corresponding to the image to be processed, a normalization operation is performed on the image processing result to obtain a corresponding normalized result, and the image processing result is updated according to the normalized result.

7. A device for determining a text image region detection model, characterized in that: The device comprises: A construction module is used to construct a to-be-trained detection network model for performing a preset image processing operation on an input image according to a preset detection network construction algorithm, wherein the to-be-trained detection network model includes a feature extraction network, a feature fusion network, and a multi-task branch; An image processing module is used to perform a preset image processing operation on the image to be processed according to the detection network model to be trained, and obtain a corresponding image processing result, wherein the image processing result includes the coordinates of the text detection box corresponding to the image to be processed; a comparison processing module, configured to perform a feature comparison operation on the coordinates of the text detection frame and the predetermined coordinates of the text pre-mark of the image to be processed according to the determined feature comparison module, to obtain a corresponding feature comparison result, wherein the feature comparison result includes training parameters corresponding to the detection network to be trained, and the training parameters are used to determine the degree of training convergence corresponding to the detection network model to be trained; A determination module, configured to determine that the to-be-trained detection network model is a target detection network model that has completed training when the feature comparison result indicates that the training parameter is greater than or equal to a preset convergence threshold; The determination module is further configured to perform a feature comparison operation on the text detection frame coordinates and the pre-determined text pre-marked coordinates of the image to be processed according to the feature comparison module determined by the comparison processing module, and before obtaining a corresponding feature comparison result, determine a first polygonal area corresponding to the text pre-marked coordinates according to the pre-determined text pre-marked coordinates of the image to be processed, and determine a target rectangular area corresponding to the first polygonal area according to the first polygonal area and the text pre-marked coordinates; The device further comprises: a splicing module, configured to map the text detection frame coordinates to the target rectangular area, and splice the text detection frame coordinates in the target rectangular area according to a preset connection order to obtain a corresponding second polygonal area; The contrast processing module performs a feature comparison operation on the text detection frame coordinates and the predetermined text pre-marked coordinates of the image to be processed according to the determined feature comparison module, and obtains the corresponding feature comparison result in a manner specifically including: According to the determined feature comparison module, a preset convolution processing operation is performed on the first polygonal area and the second polygonal area in the target rectangular area to obtain a first feature vector corresponding to the first polygonal area and a second feature vector corresponding to the second polygonal area; Calculating a target loss value based on the first eigenvector and the second eigenvector in combination with a preset loss function, wherein the target loss value is used to determine the degree of training convergence corresponding to the to-be-trained detection network model; The target loss value is determined as a feature comparison result corresponding to the text detection box coordinates and the text pre-marked coordinates.

8. A device for determining a text image region detection model, characterized in that: The device comprises: a memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the method for determining the text image area detection model according to any one of claims 1 to 6.

9. A computer storage medium, characterized in that The computer storage medium stores computer instructions, and when the computer instructions are called, they are used to execute the method for determining the text image area detection model according to any one of claims 1 to 6.

Images