Detection model training method and apparatus, and text bounding box detection method and apparatus

Abstract

The present disclosure relates to a detection model training method and apparatus, and a text bounding box detection method and apparatus. The training method comprises: inputting into a detection model to be trained a sample image carrying a text label, wherein the text label comprises a reference text bounding box and a reference probability value of a preset deletion symbol being drawn on text within the reference text bounding box; performing detection on the sample image by means of the detection model, so as to determine a text detection result of the sample image, wherein the text detection result comprises a predicted text bounding box and a predicted probability value of the preset deletion symbol being drawn on text in the predicted text bounding box; and on the basis of a target loss function, converging the text detection result and the text label, so as to obtain a trained detection model, wherein the target loss function comprises a first loss function used for evaluating the accuracy of the predicted text bounding box, and when the overlap ratio between the predicted text bounding box and the reference text bounding box is in different overlap ratio intervals, the first loss function is configured with penalty terms of different weights.

Description

Detection model training method, text box detection method and device Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a detection model training method, a text box detection method and a device. Background Art

[0002] Text correction has been applied in various fields, including smart education and office work. Text box detection is a crucial component of text correction, and the accuracy of text boxes significantly impacts the results. This can usually be achieved by training models to detect text boxes. Summary of the Invention

[0003] The present disclosure provides a detection model training method, a text box detection method and a device.

[0004] According to one aspect of the present disclosure, a method for training a detection model is provided, comprising: inputting a sample image carrying a text label into a detection model to be trained; wherein the text label includes a reference text box and a reference probability value that a preset deletion symbol is drawn on the text in the reference text box; detecting the sample image by the detection model to determine a text detection result of the sample image; wherein the text detection result includes a predicted text box and a predicted probability value that a preset deletion symbol is drawn on the text in the predicted text box; converging the text detection result and the text label according to a target loss function to obtain a trained detection model; wherein the target loss function includes a first loss function for evaluating the accuracy of the predicted text box, and when the overlap between the predicted text box and the reference text box is in different overlap intervals, the first loss function is provided with penalty items with different weights.

[0005] According to another aspect of the present disclosure, a method for detecting a text box is provided, comprising: obtaining a text image to be detected; detecting the text image using a pre-trained detection model to obtain a target probability value of a target text box and text within the target text box having a preset deletion symbol drawn on it; wherein the detection model is trained using the above-mentioned detection model training method; and determining whether the target detection box is a valid box or an invalid box based on the target probability value.

[0006] According to another aspect of the present disclosure, a training device for a detection model is provided, comprising: an image input module for inputting a sample image carrying a text label into a detection model to be trained; wherein the text label comprises a reference text box and a reference probability value that a preset deletion symbol is drawn on the text in the reference text box; a first image detection module for detecting the sample image through the detection model to determine a text detection result of the sample image; wherein the text detection result comprises a predicted text box and a predicted probability value that a preset deletion symbol is drawn on the text in the predicted text box; a model convergence module for converging the text detection result and the text label according to a target loss function to obtain a trained detection model; wherein the target loss function comprises a first loss function for evaluating the accuracy of the predicted text box, and when the overlap between the predicted text box and the reference text box is in different overlap intervals, the first loss function is provided with penalty items with different weights.

[0007] According to another aspect of the present disclosure, a text box detection device is provided, comprising: an image acquisition module for acquiring a text image to be detected; a second image detection module for detecting the text image using a pre-trained detection model to obtain a target probability value of a target text box and the text in the target text box having a preset deletion symbol drawn on it; wherein the detection model is trained using the above-mentioned detection model training method; and a detection box determination module for determining whether the target detection box is a valid box or an invalid box based on the target probability value.

[0008] According to another aspect of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing instructions executable by the processor; and the processor for reading the executable instructions from the memory and executing the instructions to implement the above-mentioned detection model training method or text box detection method.

[0009] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein, when the computer instructions are executed on a terminal device, the terminal device implements the above-mentioned detection model training method or text box detection method. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0011] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required for describing the embodiments. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0012] FIG1 is a flow chart of a method for training a detection model according to an embodiment of the present disclosure;

[0013] FIG2 is a schematic diagram of a preset deletion symbol on a text according to an embodiment of the present disclosure;

[0014] FIG3 is a schematic diagram of the structure of a detection model provided by an embodiment of the present disclosure;

[0015] FIG4 is a schematic diagram of text box overlap provided by an embodiment of the present disclosure;

[0016] FIG5 is a flow chart of a text box detection method provided by an embodiment of the present disclosure;

[0017] FIG6 is a schematic diagram of the structure of a training device for a detection model provided in an embodiment of the present disclosure;

[0018] FIG7 is a schematic structural diagram of a text box detection device provided by an embodiment of the present disclosure;

[0019] FIG8 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0020] In order to more clearly understand the above-mentioned purposes, features and advantages of the present disclosure, embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein. Instead, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0021] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0022] The term "including" and its variations used in this document are open inclusions, that is, "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the description below. It should be noted that the concepts of "first", "second", etc. mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0023] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0024] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0025] Text correction mainly involves OCR (Optical Character Recognition) technology, and text box detection is an important part of OCR technology. Currently, when training models for text box detection, a predetermined, unchanging loss function is used. However, there may be large differences between predicted text boxes, and the fixed loss function cannot match predicted text boxes with large differences, resulting in low text box detection accuracy of the trained model. In addition, when correcting text, there are often texts deleted by symbols such as horizontal lines, strikethroughs, or difference signs. Currently, these deleted texts are generally considered negative samples. Negative samples mean that the model must not only be able to detect where handwriting is, but also whether the handwriting is correct and should be retained. Therefore, negative samples make it difficult for the model to detect text boxes and are very unfriendly to the model. It is difficult for the model to give accurate text box detection results, and the text box recall is low.

[0026] Therefore, the text box detection of the model obtained by using the traditional training method is inaccurate and difficult to meet the needs of text correction scenarios. Based on this, the embodiment of the present disclosure provides a training method for a detection model.

[0027] Figure 1 is a flowchart of a detection model training method provided by an embodiment of the present disclosure. This method can be applied to text correction in fields such as education and office work, such as dictation correction, essay correction, and copywriting correction. The method is executed by a terminal device equipped with a detection model. Referring to Figure 1, the detection model training method includes the following steps S102 to S106.

[0028] In step S102, a sample image with text labels is input to a detection model to be trained; wherein the text labels include: a reference text box and a reference probability value of a preset deletion symbol drawn on the text in the reference text box.

[0029] In this embodiment, the sample image can be a static image or a video frame from a dynamic video, obtained by a user through an image selection operation, image capture operation, or image upload operation on the terminal. The sample image is an image containing text content, for example, an image containing essay content in a teaching setting, an image containing copywriting content in an office setting, etc.

[0030] The labels pre-marked in the sample image include: a base text box and a base probability value of a preset deletion symbol drawn on the text in the base text box. The above-mentioned base text box is generally a rectangular box that accurately surrounds the text. As shown in Figure 2, the above-mentioned base probability value is used to indicate whether the text is drawn with a preset deletion symbol such as a stroke, horizontal line, oblique line or difference sign, indicating that the text has been deleted; illustratively, the base probability value can be set to 0 or 1, where 0 indicates that the text in the base text box does not have a preset deletion symbol drawn on it, such as the text "you" in Figure 2; 1 indicates that the text in the base text box has a preset deletion symbol drawn on it, such as the text "good" in Figure 2.

[0031] This embodiment assigns a baseline probability value to the text in the sample image, and can distinguish whether the text is drawn with a preset deletion symbol by the difference in the baseline probability value. Therefore, all texts including the text drawn with the preset deletion symbol are regarded as positive samples without the need to set negative samples; accordingly, in the subsequent detection and other image processing processes, the detection model does not need to process negative samples, and the positive samples are very friendly to the detection model, which can greatly simplify the processing requirements of the detection model.

[0032] In step S104, the sample image is detected by the detection model to be trained to determine the text detection result of the sample image, wherein the text detection result includes: a predicted text box and a predicted probability value of a preset deletion symbol being drawn on the text in the predicted text box.

[0033] In one embodiment, the detection model to be trained extracts image features of the sample image, such as handwriting features, position features, texture features, and color features. Based on these image features, the following text detection results are determined: a predicted text box surrounding the text and a predicted probability value of a predetermined deletion symbol being drawn on the text.

[0034] In step S106, the text detection results and text labels are converged according to the target loss function to obtain a trained detection model. The target loss function includes a first loss function for evaluating the accuracy of the predicted text box, and the first loss function is provided with penalty terms with different weights when the overlap between the predicted text box and the reference text box is in different overlap intervals.

[0035] The first loss function in this embodiment can be determined based on a preset IoU (Intersection over Union) loss function and a weighted penalty term. In one example, the IoU loss function is a function representing the degree of overlap between the predicted text box and the reference text box; the penalty term is used to represent the ratio of the overlapping area between the reference text box and the predicted text box to the area of ​​the reference text box; the weight of the penalty term can be set to different values ​​according to different overlap intervals, or in other words, different overlap intervals correspond to different weights. Of course, the weight can be 0.

[0036] Based on this, the first loss function can be flexibly adjusted according to the degree of overlap between the predicted text box and the reference text box, so that the adjusted first loss function pays more attention to the actual position of the predicted text box, increases the fit between the first loss function and the predicted text box, and thus uses different first loss functions to more accurately evaluate the accuracy of the predicted text box.

[0037] In this embodiment, the target loss function includes the first loss function described above and also includes a second loss function for evaluating the accuracy of the predicted probability value. The text detection results and text labels are converged according to the first and second loss functions, and a trained detection model is obtained when convergence conditions are met; convergence conditions may include, for example, the loss function value reaching a preset loss threshold, the number of iterative training iterations reaching a preset threshold, etc.

[0038] The training method of the detection model provided in this embodiment first inputs a sample image with a text label into the detection model to be trained; wherein the text label includes: a baseline text box and a baseline probability value that a preset deletion symbol is drawn on the text in the baseline text box; then, the sample image is detected by the detection model to be trained to determine the text detection result of the sample image, and the text detection result includes: a predicted text box and a predicted probability value that a preset deletion symbol is drawn on the text in the predicted text box; finally, the text detection result and the text label are converged according to a target loss function to obtain a trained detection model; the target loss function includes: a first loss function for evaluating the accuracy of the predicted text box, and when the overlap between the predicted text box and the baseline text box is in different overlap intervals, the first loss function is provided with penalty items with different weights.

[0039] In this solution, when the degree of overlap between the predicted text box and the reference text box is different, the first loss function used to evaluate the accuracy of the predicted text box is also different accordingly. That is, the first loss function will be adjusted as the degree of overlap between the text boxes changes. In this way, when using the target loss function including the first loss function to train the detection model, adjusting different first loss functions for different degrees of overlap of text boxes can pay more attention to the predicted text box, thereby improving the accuracy and recall of text box detection. At the same time, for text with preset deletion symbols such as horizontal lines or difference signs, they are generally regarded as negative samples, which are not friendly to model detection and will affect the model training effect. However, in comparison, the solution according to the embodiment of the present disclosure regards all texts as positive samples and expresses whether there is a preset deletion symbol on the text in the form of a probability value. In short, text with a preset deletion symbol is also a positive sample. Therefore, for the detection model, only the text of the positive sample needs to be detected, which greatly reduces the detection requirements and thus can improve the detection accuracy and recall of the text box.

[0040] According to the above embodiment, to implement the detection model training method, the sample images are first labeled. That is, the acquired sample images are marked with text labels. The text labels include: a reference text box and a reference probability value for whether the text within the reference text box has a preset deletion symbol drawn on it. If the text does not have the preset deletion symbol drawn on it, the reference probability value for the text is assigned to 0, indicating a positive sample; if the text does have the preset deletion symbol drawn on it, the reference probability value for the text is assigned to 1, also indicating a positive sample.

[0041] Then, the sample image carrying the text label is input into the detection model to be trained, and the sample image is detected by the detection model to be trained to determine the text detection result of the sample image.

[0042] In this embodiment, referring to FIG3 , the detection model to be trained includes: a backbone network, a first branch network, and a second branch network. Accordingly, the sample image is detected by the detection model to be trained to determine the text detection result of the sample image, which can be achieved through the following process.

[0043] The original image features of the sample image are extracted through the backbone network. Specifically, the backbone network can be a backbone model, which is the main component of the detection model, usually a convolutional neural network (CNN) or a residual neural network (ResNet). The backbone model usually has many layers and many parameters, which can be used to extract high-level feature representations of the sample image for subsequent processing and analysis.

[0044] The first branch network extracts handwriting features from the original image features and determines the predicted probability value of a preset deletion symbol drawn on the text based on the handwriting features. The second branch network extracts position features from the original image features and determines the predicted text box surrounding the text based on the position features.

[0045] The first and second branch networks are detection head models, typically employing multi-layer convolutional structures. While the first and second branch networks utilize the same backbone model to extract original image features, they employ different convolutional layer structures to extract specific features from the original image. Specifically, the first branch network extracts handwriting features from the original image, while the second branch network extracts positional features from the original image.

[0046] This embodiment assigns a probability value to positive samples to indicate whether a preset deletion symbol is drawn. The first branch network in the detection model extracts handwriting features from the original image features and predicts the predicted probability value of the preset deletion symbol being drawn on the text based on the handwriting features. The predicted probability value is a value in the range [0, 1], which is used to represent the probability that the text has the preset deletion symbol drawn on it. If it is close to 1, there is a high probability that the text has the preset deletion symbol and has been crossed out. If it is close to 0, there is a high probability that the text is written normally and has not been deleted or crossed out.

[0047] The second branch network extracts the position features from the original image features, and predicts the position parameters corresponding to the text based on the position features, such as length, width, and position coordinates of key points; and determines the predicted text box surrounding the text based on the above position parameters.

[0048] According to the above method, the predicted text box corresponding to each text in the sample image and the predicted probability value corresponding to the predicted text box are finally output.

[0049] In this embodiment, since texts with or without preset deletion symbols are regarded as positive samples, the detection model only needs to extract the original image features and subsequent handwriting features and position features of the positive samples to predict the predicted probability value of the preset deletion symbol drawn on the text box and the text. This greatly simplifies the requirements for feature extraction of the model and reduces the difficulty of the model processing images.

[0050] After detecting and obtaining the predicted text box of the sample image and the predicted probability value of a preset deletion symbol drawn on the text in the predicted text box, the text detection results and text labels are converged according to the target loss function. A possible embodiment can be provided for the first loss function included in the target loss function.

[0051] In some approaches, the loss function used by the detection model is often the IoU loss function shown in the following formula (1), that is, the ratio of the overlapping area between the predicted text box and the reference text box to the total area occupied by the two detection boxes (total area - overlapping area):

[0052] Among them, S1 represents the area of ​​the reference text box, S2 represents the area of ​​the predicted text box, and Inter(S1, S2) represents the overlapping area between the reference text box and the predicted text box. In practical applications, it is hoped that the predicted text box can cover more of the entire text, that is, overlap more with the reference text box. Therefore, based on the above IoU loss function, this embodiment changes the denominator to the area of ​​the reference text box, indicating that the overlapping area is allowed to cover the reference text box as much as possible. The formula is as follows:

[0053] After actual model training, it was found that directly using the new_IoU shown in the above formula (2) as the loss function is not easy to train the detection model. Therefore, this embodiment considers whether the above two formulas can be combined. On this basis, this embodiment can determine the first loss function according to the following method:

[0054] Determine a target overlap interval corresponding to the overlap between the predicted text box and the reference text box; determine a target weight corresponding to the target overlap interval based on the correspondence between the overlap interval and the weight of the penalty item; and determine a first loss function based on the IoU loss function and the penalty item with the target weight.

[0055] In one example, the degree of overlap between the predicted text box and the reference text box can be used as an IoU loss function; the ratio of the overlapping area between the reference text box and the predicted text box to the area of ​​the reference text box can be used as a penalty term; and the sum of the IoU loss function and the penalty term with a target weight can be determined as the first loss function. The above can be understood as: the IoU loss function shown in formula (1) is used as the basic term, the new_IoU loss function shown in formula (2) is used as the penalty term, and an adjustable weight is set for the penalty term. The weight is determined according to the degree of overlap between the predicted text box and the reference text box, and different overlap areas correspond to different weights, thereby determining the first loss function.

[0056] This embodiment adjusts the weight of the penalty term as the degree of overlap between the predicted text box and the reference text box differs (ie, they are located in different overlap intervals), thereby achieving the purpose of adjusting the first loss function.

[0057] To facilitate understanding of the first loss function, two specific first loss functions are provided below in conjunction with FIG4 , taking two overlap intervals of an overlap less than a preset overlap threshold (e.g., 0.5) and an overlap not less than a preset overlap threshold as examples.

[0058] As shown in (a) of Figure 4, when the overlap between the predicted text box and the reference text box is less than the preset overlap threshold, the first loss function determined based on the IoU loss function is used. The overlap between the two text boxes is less than the overlap threshold, which means that the overlapping area cannot cover the reference text box well. At this time, more attention can be paid to the overlap represented by formula (1), while the overlapping area represented by formula (2) covers the reference text box as much as possible and can be paid less attention or even ignored; thus, the weight of the penalty term can be set to a small value or even set to 0. When the weight is set to 0, that is, the first loss function is determined based on the IoU loss function and the detection model is optimized using the first loss function, the first loss function includes:

[0059] Among them, total IoU Represents the first loss function, S1 represents the area of ​​the reference text box, S2 represents the area of ​​the predicted text box, Inter(S1, S2) represents the overlapping area between the reference text box and the predicted text box, Indicates the degree of overlap between the predicted text box and the reference text box.

[0060] As shown in (b) of Figure 4, when the overlap between the predicted text box and the reference text box is not less than the preset overlap threshold, the first loss function determined based on the IoU loss function and the penalty term with the target weight is adopted. The overlap between the two text boxes is not less than the overlap threshold, which means that the two boxes overlap to a certain extent and the overlapping area can cover the reference text box to a large extent. At this time, more attention should be paid to formula (2). Therefore, the first loss function is determined based on the IoU loss function and the penalty term with the target weight and the detection model is optimized using the first loss function. The first loss function includes:

[0061] Among them, total IoU Represents the first loss function, S1 represents the area of ​​the reference text box, S2 represents the area of ​​the predicted text box, Inter(S1, S2) represents the overlapping area between the reference text box and the predicted text box, Indicates the overlap between the predicted text box and the reference text box, represents the penalty term, and ω represents the target weight of the penalty term.

[0062] In addition, it can be understood that when the overlap is not less than the preset overlap threshold, in order to improve the accuracy of the first loss function, a more detailed overlap interval and corresponding weight can be set, and the greater the overlap, the greater the weight of the penalty term, so that the detection model can fit new_IoU more.

[0063] In the above embodiment, according to the different overlaps between the predicted text box and the reference text box, the different weights of the penalty items are adjusted, and then the first loss function is adjusted by setting penalty items with different weights, so that the first loss function allocates different attention to IoU and new_IoU when the overlaps of the two text boxes are different, thereby better optimizing the detection model and improving the detection accuracy and recall rate of the text boxes.

[0064] In this embodiment, converging the text detection results and text labels according to the target loss function to obtain a trained detection model may include: first, determining a target loss value between the text detection results and the text labels according to the target loss function.

[0065] According to the above embodiment, the target loss function includes: a first loss function for evaluating the accuracy of the predicted text box, and a second loss function for evaluating the accuracy of the predicted probability value. In this case, determining the target loss value between the text detection result and the text label according to the target loss function includes: determining a first loss value between the predicted text box and the reference text box according to the first loss function; determining a second loss value between the predicted probability value and the reference probability value according to the second loss function; and determining the weighted sum of the first loss value and the second loss value as the target loss value between the text detection result and the text label.

[0066] In an embodiment according to the present disclosure, the second loss function is a loss function commonly used in detection models, for example, a Focal loss function.

[0067] In a specific embodiment, for the predicted text box output by the second branch network, the initial loss value between the position parameters of the predicted text box and the position parameters of the reference text box is determined according to the first loss function; since the optimization model is performed by minimizing the loss function, the total IoU The larger the better, so in this embodiment, 1 minus the initial loss value can be used as the first loss value.

[0068] For the predicted probability value output by the first branch network, a second loss value is determined between the predicted probability value and the baseline probability value based on the second loss function. Since most image samples are relatively easy to predict, such as when the entire text is blacked out or there are no preset deletion symbols, the second loss function used in this embodiment focuses more on the probability value, which results in more accurate predictions for those difficult-to-predict areas.

[0069] The weighted sum of the first loss value and the second loss value is determined as the target loss value between the text detection result and the text label.

[0070] The parameters of the detection model to be trained are then adjusted according to the target loss value until convergence conditions are reached, thereby obtaining a trained detection model. The convergence conditions, such as the target loss value reaching a preset loss threshold, the number of iterative optimizations of the detection model reaching a preset threshold, and the number of sample images used in model training reaching a preset threshold, are not limited here.

[0071] The detection model trained according to the above embodiment can be used for detecting text boxes in scenarios such as text correction and text recognition.

[0072] FIG5 is a flowchart of a text box detection method provided by an embodiment of the present disclosure, which can be executed by a terminal device configured with a detection model. Referring to FIG5 , the text box detection method includes the following steps S202 to S204.

[0073] In step S202, a text image to be detected is obtained. For example, the text image can be an image obtained by taking a photo of the composition.

[0074] In step S204, the text image is detected using a pre-trained detection model to obtain target probability values ​​of whether a preset deletion symbol is drawn on the target text box and the text in the target text box.

[0075] The detection model is trained by the detection model training method provided in the above embodiment, so the detection model has a high recall rate and accuracy.

[0076] In specific implementation, the image features of the text image are extracted through the backbone network; the handwriting features in the above image features are extracted through the first branch network, and the target probability value of the preset deletion symbol drawn on the text is determined based on the handwriting features; the position features in the above image features are extracted through the second branch network, and the target text box surrounding the text is determined based on the position features.

[0077] This embodiment adds the output of the target probability value to the text to indicate whether the text in the target text box is drawn with a preset deletion symbol, which can assist the detection model in improving the OCR effect.

[0078] In step S206 , the target detection frame is determined to be a valid frame or an invalid frame according to the target probability value.

[0079] In practical applications, a probability threshold can be set (such as using 0.7 as the probability threshold). If the target probability value corresponding to the target text box is higher than the probability threshold, it means that the text in the target text box has a high probability of being marked with a preset deletion symbol, that is, the text is crossed out or smeared out and is not a valid text; thus, the target text box is determined to be an invalid box. Conversely, if the target probability value corresponding to the target text box is not higher than the probability threshold, the target text box is determined to be a valid box. Subsequently, the valid box is subjected to normal recognition and other processing, which can greatly improve the recall value of the detection model, allowing more text that is written in a disorderly manner but is not crossed out to be detected.

[0080] FIG6 is a block diagram of a detection model training apparatus provided in an embodiment of the present disclosure. The apparatus is used to implement the detection model training method provided in the above embodiment. The apparatus can be implemented using software and / or hardware. Referring to FIG6 , the detection model training apparatus 300 includes an image input module 310, a first image detection module 320, and a model convergence module 330.

[0081] The image input module 310 is used to input sample images carrying text labels to the detection model to be trained; wherein the text labels include: a reference text box and a reference probability value of a preset deletion symbol drawn on the text in the reference text box.

[0082] The first image detection module 320 is used to detect the sample image through the detection model to be trained to determine the text detection result of the sample image; wherein the text detection result includes: a predicted probability value of a preset deletion symbol being drawn on the predicted text box and the text in the predicted text box.

[0083] The model convergence module 330 is used to converge the text detection results and the text labels according to the target loss function to obtain a trained detection model.

[0084] In an embodiment of the present disclosure, the target loss function includes: a first loss function for evaluating the accuracy of the predicted text box, and when the overlap between the predicted text box and the reference text box is in different overlap intervals, a penalty term with different weights is provided in the first loss function.

[0085] In one embodiment, the detection model to be trained includes: a backbone network, a first branch network, and a second branch network; the first image detection module 320 is also used to: extract the original image features of the sample image through the backbone network; extract the handwriting features in the original image features through the first branch network, and determine the predicted probability value of a preset deletion symbol drawn on the text based on the handwriting features; extract the position features in the original image features through the second branch network, and determine the predicted text box surrounding the text based on the position features.

[0086] In one embodiment, the device also includes a function determination module, which is used to: determine a target overlap interval corresponding to the overlap between the predicted text box and the reference text box; determine a target weight corresponding to the target overlap interval based on the correspondence between the overlap interval and the weight of the penalty item; and determine a first loss function based on the overlap IoU loss function and the penalty item with the target weight.

[0087] In one embodiment, the function determination module is further used to: use the degree of overlap between the predicted text box and the reference text box as the IoU loss function; use the ratio of the overlapping area between the reference text box and the predicted text box to the area of ​​the reference text box as a penalty term; and determine the sum of the IoU loss function and the penalty term set with the target weight as the first loss function.

[0088] In one embodiment, the function determination module is also used to: when the overlap between the predicted text box and the reference text box is less than the preset overlap threshold, adopt a first loss function determined based on the overlap IoU loss function; when the overlap between the predicted text box and the reference text box is not less than the preset overlap threshold, adopt a first loss function determined based on the IoU loss function and a penalty item with a target weight.

[0089] In one embodiment, the first loss function determined based on the overlap IoU loss function includes: Among them, total IoU represents the first loss function, S1 represents the area of ​​the reference text box, S2 represents the area of ​​the predicted text box, Inter(S1, S2) represents the overlapping area between the reference text box and the predicted text box, Indicates the degree of overlap between the predicted text box and the reference text box.

[0090] In one embodiment, the first loss function determined based on the IoU loss function and the penalty term with the target weight includes: Among them, total IoUrepresents the first loss function, S1 represents the area of ​​the reference text box, S2 represents the area of ​​the predicted text box, Inter(S1, S2) represents the overlapping area between the reference text box and the predicted text box, represents the overlap between the predicted text box and the reference text box, Denotes the penalty term, and ω denotes the target weight of the penalty term.

[0091] In one embodiment, the model convergence module 330 is also used to: determine the target loss value between the text detection result and the text label according to the target loss function; adjust the parameters of the detection model to be trained according to the target loss value, and terminate the training until the convergence condition is reached to obtain the trained detection model.

[0092] In one embodiment, the target loss function also includes: a second loss function for evaluating the accuracy of the predicted probability value; the model convergence module 330 is also used to: determine a first loss value between the predicted text box and the reference text box according to the first loss function; determine a second loss value between the predicted probability value and the reference probability value according to the second loss function; and determine the weighted sum of the first loss value and the second loss value as the target loss value between the text detection result and the text label.

[0093] FIG7 is a block diagram of a text box detection apparatus provided in an embodiment of the present disclosure. This apparatus is used to implement the text box detection method provided in the above embodiments. This apparatus can be implemented using software and / or hardware. Referring to FIG7 , text box detection apparatus 400 includes an image acquisition module 410, a second image detection module 420, and a detection box determination module 430.

[0094] The image acquisition module 410 is used to acquire a text image to be detected.

[0095] The second image detection module 420 is used to detect the text image through a pre-trained detection model to obtain a target probability value of a preset deletion symbol drawn on the target text box and the text in the target text box; wherein, the detection model is trained by the above-mentioned detection model training method.

[0096] The detection frame determination module 430 is configured to determine whether the target detection frame is a valid frame or an invalid frame according to the target probability value.

[0097] The device provided in this embodiment has the same implementation principle and technical effects as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.

[0098] The exemplary embodiments of the present disclosure further provide an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores a computer program executable by the at least one processor, the computer program being configured to cause the electronic device to perform a method according to an exemplary embodiment of the present disclosure when executed by the at least one processor.

[0099] Exemplary embodiments of the present disclosure further provide a computer program product, including a computer program, wherein when the computer program is executed by a processor of a computer, it is used to cause the computer to perform the method according to the embodiment of the present disclosure.

[0100] With reference to Figure 8, a block diagram of an electronic device 500 that can serve as a server or client of the present disclosure will now be described, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0101] As shown in Figure 8, electronic device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. In RAM 503, various programs and data required for the operation of device 500 can also be stored. Computing unit 501, ROM 502, and RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to bus 504.

[0102] Multiple components within electronic device 500 are connected to I / O interface 505, including an input unit 506, an output unit 507, a storage unit 508, and a communication unit 509. Input unit 506 can be any type of device capable of inputting information into electronic device 500. Input unit 506 can receive input numeric or character information and generate key input signals related to user settings and / or function control of the electronic device. Output unit 507 can be any type of device capable of presenting information and may include, but is not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. Storage unit 508 may include, but is not limited to, a magnetic disk or an optical disk. Communication unit 509 allows electronic device 500 to exchange information / data with other devices via computer networks such as the Internet and / or various telecommunication networks and may include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver and / or chipset, such as a Bluetooth™ device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0103] The computing unit 501 can be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 501 performs the various methods and processes described above. For example, in some embodiments, the training method of the detection model or the detection method of the text box can be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 508. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 500 via the ROM 502 and / or the communication unit 509. In some embodiments, the computing unit 501 can be configured to execute the training method of the detection model or the detection method of the text box by any other appropriate means (e.g., by means of firmware).

[0104] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0105] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0106] As used in this disclosure, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., a magnetic disk, an optical disk, a memory, a programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0107] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0108] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0109] Computer systems may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The client and server relationship arises through computer programs running on the respective computers and having a client-server relationship to each other.

[0110] The foregoing description is intended only to provide specific embodiments of the present disclosure, intended to enable those skilled in the art to understand and implement the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the embodiments described herein, but rather to be construed in the broadest manner consistent with the principles and novel features disclosed herein.

Claims

1. A method for training a detection model, comprising: Inputting a sample image with a text label to the detection model to be trained; wherein the text label includes a reference text box and a reference probability value of a preset deletion symbol drawn on the text in the reference text box; Detecting the sample image using the detection model to determine a text detection result of the sample image; wherein the text detection result includes a predicted probability value of the preset deletion symbol being drawn on a predicted text box and text within the predicted text box; Converging the text detection results and the text labels according to a target loss function to obtain a trained detection model; The target loss function includes a first loss function for evaluating the accuracy of the predicted text box, and when the overlap between the predicted text box and the reference text box is in different overlap intervals, the first loss function is provided with penalty items with different weights.

2. The method according to claim 1, wherein the detection model comprises: A trunk network, a first branch network, and a second branch network; The detecting the sample image by using the detection model to determine a text detection result of the sample image includes: Extracting original image features of the sample image through the backbone network; Extracting handwriting features from the original image features through the first branch network, and determining the predicted probability value of the preset deletion symbol being drawn on the text based on the handwriting features; The second branch network extracts position features from the original image features, and determines the predicted text box surrounding the text based on the position features.

3. The method according to claim 1, further comprising: determining a target overlap interval corresponding to the overlap between the predicted text frame and the reference text frame; Determining a target weight corresponding to the target overlap interval according to a correspondence between the target overlap interval and the weight of the penalty item; The first loss function is determined based on an overlap IoU loss function and a penalty term having the target weight.

4. The method according to claim 3, wherein determining the first loss function based on the overlap IoU loss function and the penalty term provided with the target weight comprises: Using the overlap between the predicted text box and the reference text box as the IoU loss function; The ratio of the overlapping area between the reference text box and the predicted text box to the area of the reference text box is used as a penalty term; The sum of the IoU loss function and the penalty term set with the target weight is determined as the first loss function.

5. The method according to claim 1 or 3, further comprising: When the overlap between the predicted text box and the reference text box is less than a preset overlap threshold, adopting a first loss function determined based on an overlap IoU loss function; When the degree of overlap between the predicted text box and the reference text box is not less than the preset overlap threshold, a first loss function determined based on the IoU loss function and a penalty term with a target weight is adopted.

6. The method according to claim 5, wherein the first loss function determined based on the overlap IoU loss function comprises: Among them, total IoU represents the first loss function, S1 represents the area of the reference text box, S2 represents the area of the predicted text box, Inter(S1, S2) represents the overlapping area between the reference text box and the predicted text box, Indicates the degree of overlap between the predicted text box and the reference text box.

7. The method according to claim 5, wherein the first loss function determined based on the IoU loss function and the penalty term with the target weight comprises: Among them, total IoU represents the first loss function, S1 represents the area of the reference text box, S2 represents the area of the predicted text box, Inter(S1, S2) represents the overlapping area between the reference text box and the predicted text box, represents the overlap between the predicted text box and the reference text box, Denotes the penalty term, and ω denotes the target weight of the penalty term.

8. The method according to claim 1, wherein the text detection results and the text labels are converged according to a target loss function to obtain a trained detection model, comprising: Determine a target loss value between the text detection result and the text label according to the target loss function; The parameters of the detection model are adjusted according to the target loss value until the training is terminated when the convergence condition is reached, thereby obtaining the trained detection model.

9. The method according to claim 8, wherein the objective loss function further comprises: a second loss function for evaluating the accuracy of the predicted probability value; The determining a target loss value between the text detection result and the text label according to a target loss function includes: Determine a first loss value between the predicted text box and the reference text box according to the first loss function; Determining a second loss value between the predicted probability value and the reference probability value according to the second loss function; A weighted sum of the first loss value and the second loss value is determined as a target loss value between the text detection result and the text label.

10. A method for detecting a text box, comprising: Get the text image to be detected; Detecting the text image using a pre-trained detection model to obtain a target probability value of a preset deletion symbol drawn on a target text box and the text in the target text box; wherein the detection model is trained using the method of any one of claims 1 to 9; Determine whether the target detection frame is a valid frame or an invalid frame according to the target probability value.

11. A training device for a detection model, comprising: An image input module, configured to input a sample image carrying a text label to the detection model to be trained; wherein the text label includes a reference text box and a reference probability value of a preset deletion symbol being drawn on the text in the reference text box; a first image detection module, configured to detect the sample image using the detection model and determine a text detection result of the sample image; wherein the text detection result includes a predicted probability value of the predetermined deletion symbol being drawn on a predicted text box and text within the predicted text box; A model convergence module, configured to converge the text detection results and the text labels according to a target loss function to obtain a trained detection model; The target loss function includes a first loss function for evaluating the accuracy of the predicted text box, and when the overlap between the predicted text box and the reference text box is in different overlap intervals, the first loss function is provided with penalty items with different weights.

12. A text box detection device, comprising: An image acquisition module is used to acquire the text image to be detected; a second image detection module, configured to detect the text image using a pre-trained detection model to obtain a target probability value of a target text box and the text within the target text box having a preset deletion symbol drawn thereon; wherein the detection model is trained using the method of any one of claims 1 to 9; A detection frame determination module is used to determine whether the target detection frame is a valid frame or an invalid frame according to the target probability value.

13. An electronic device, characterized in that: The electronic device comprises: processor; a memory for storing instructions executable by the processor; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the method of any one of claims 1 to 9 or the method of claim 10.

14. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: When the computer instructions are executed on a terminal device, the terminal device implements the method according to any one of claims 1 to 9 or the method according to claim 10.

Images