Text detection model training method and device, computer device, and storage medium

Abstract

The application relates to a text detection model training method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: obtaining an image sample set, the image sample set comprising at least one image sample; obtaining a first feature map and a first prediction result of the image sample by referring to a reference model; obtaining a second feature map and a second prediction result of the image sample by using a to-be-trained model; determining a similarity loss of the image sample based on the first feature map and the second feature map; determining a pixel-level loss of the image sample according to the first prediction result and the second prediction result; determining a true value loss of the image sample according to the second prediction result and a label result of the image sample; and training the to-be-trained model based on the similarity loss, the pixel-level loss and the true value loss to obtain a text detection model. The method can improve the accuracy of text detection.

Description

Technical Field

[0001] This application relates to the field of machine learning technology, and in particular to a method, apparatus, computer device, storage medium, and computer program product for training a text detection model. Background Technology

[0002] With the development of computer vision technology, high accuracy is required for text detection in financial scenarios. Existing text detection methods often train a model using image samples. However, this method suffers from low accuracy when detecting long texts in images. Summary of the Invention

[0003] Therefore, it is necessary to provide a training method, apparatus, computer equipment, computer-readable storage medium, and computer program product for text detection models that can improve the detection accuracy of traditional text detection models, in order to address the problem of low detection accuracy.

[0004] Firstly, this application provides a method for training a text detection model. The method includes:

[0005] Obtain an image sample set, which includes at least one image sample;

[0006] The first feature map and the first prediction result of the image sample are obtained by the reference model. The first prediction result represents the prediction result of the reference model on the category to which each pixel in the image sample belongs.

[0007] The second feature map and the second prediction result of the image sample are obtained by the model to be trained. The second prediction result represents the prediction result of the model to be trained on the category of each pixel in the image sample.

[0008] Based on the first feature map and the second feature map, the similarity loss of the image samples is determined; based on the first prediction result and the second prediction result, the pixel-level loss of the image samples is determined; based on the second prediction result and the annotation result of the image samples, the ground value loss of the image samples is determined.

[0009] The text detection model is trained using similarity loss, pixel-level loss, and ground truth loss.

[0010] In one embodiment, the similarity loss of image samples is determined based on the first feature map and the second feature map, including:

[0011] Obtain the correlation between pixels in the first feature map and the correlation between pixels in the second feature map;

[0012] The similarity loss of image samples is determined based on the correlation between pixels in the first feature map and the correlation between pixels in the second feature map.

[0013] In one embodiment, the first feature map and the second feature map are of equal size;

[0014] Based on the correlation between pixels in the first feature map and the correlation between pixels in the second feature map, the similarity loss of image samples is determined, including:

[0015] Obtain the width and height of the first feature map;

[0016] The first similarity map is determined based on the correlation between pixels in the first feature map;

[0017] The second similarity map is determined based on the correlation between pixels in the second feature map;

[0018] For each pixel in the first similarity map, obtain the square of the difference between the current pixel and the corresponding pixel in the second similarity map;

[0019] The summation result is obtained by summing the squares of the differences between each pixel in the first similarity map;

[0020] Based on the summation result, the width and height of the first feature map, the similarity loss of the image samples is determined.

[0021] In one embodiment, the first prediction result further includes a first probability map and a first approximate binary map. The first probability map represents the probability that each pixel in the image sample predicted by the reference module belongs to text, and the first approximate binary map represents the binary classification result of each pixel in the image sample predicted by the reference model as either text or non-text. The second prediction result further includes a second probability map and a second approximate binary map. The second probability map represents the probability that each pixel in the image sample predicted by the model to be trained belongs to text, and the first approximate binary map represents the binary classification result of each pixel in the image sample predicted by the model to be trained as either text or non-text. Based on the first prediction result and the second prediction result, the pixel-level loss of the image sample is determined, including:

[0022] The first probability map is expanded using an expansion function to obtain the first expansion result;

[0023] The first approximate binary image is expanded using an expansion function to obtain the second expansion result;

[0024] Based on the first dilation result, the second dilation result, the second probability map, and the second approximate binary map, the pixel-level loss of the image sample is determined.

[0025] In one embodiment, the first prediction result includes a first threshold map, which represents the probability that each pixel in the image sample predicted by the reference model belongs to a text boundary; based on the second prediction result and the annotation result of the image sample, the ground value loss of the image sample is determined, including:

[0026] Based on the first probability map and the annotation results of the image samples, the first ground truth loss is determined;

[0027] Based on the first threshold map and the annotation results of the image samples, the second ground truth loss is determined;

[0028] Based on the annotation results of the first approximate binary map and image samples, the third ground truth loss is determined;

[0029] The ground value loss of the image sample is determined based on the first ground value loss, the second ground value loss, and the third ground value loss.

[0030] In one embodiment, the model to be trained is trained based on similarity loss, pixel-level loss, and ground truth loss to obtain a text detection model, including:

[0031] The target loss is determined based on similarity loss, pixel-level loss, and ground truth loss;

[0032] Based on the target loss, adjust the parameters in the model to be trained;

[0033] Training continues based on the adjusted model until the training stops when the stopping condition is met. The model obtained after training stops is then used as the text detection model.

[0034] In one embodiment, the training method for the text detection model further includes:

[0035] Acquire the image to be tested;

[0036] The image to be tested is input into the text detection model to obtain the predicted category of each pixel in the image to be tested;

[0037] Based on the prediction results of the category of each pixel in the image to be tested, the text region is identified and marked in the image to be tested.

[0038] Secondly, this application also provides a training device for a text detection model. The device includes:

[0039] The sample acquisition module is used to acquire an image sample set, which includes at least one image sample.

[0040] The first result acquisition module is used to acquire the first feature map and the first prediction result of the image sample through the reference model. The first prediction result represents the prediction result of the reference model for the category to which each pixel in the image sample belongs.

[0041] The second result acquisition module is used to acquire the second feature map and the second prediction result of the image sample through the model to be trained. The second prediction result represents the prediction result of the model to be trained on the category to which each pixel on the image sample belongs.

[0042] The loss determination module is used to determine the similarity loss of image samples based on the first feature map and the second feature map; determine the pixel-level loss of image samples based on the first prediction result and the second prediction result; and determine the ground value loss of image samples based on the second prediction result and the annotation result of image samples.

[0043] The model training module is used to train the model to be trained based on similarity loss, pixel-level loss, and ground truth loss to obtain a text detection model.

[0044] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0045] Obtain an image sample set, which includes at least one image sample;

[0046] The first feature map and the first prediction result of the image sample are obtained by using a reference model;

[0047] The second feature map and the second prediction result of the image sample are obtained through the model to be trained;

[0048] Based on the first feature map and the second feature map, the similarity loss of the image samples is determined;

[0049] Based on the first and second prediction results, determine the pixel-level loss of the image samples;

[0050] Based on the second prediction result and the annotation result of the image sample, determine the ground value loss of the image sample;

[0051] The text detection model is trained using similarity loss, pixel-level loss, and ground truth loss.

[0052] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0053] Obtain an image sample set, which includes at least one image sample;

[0054] The first feature map and the first prediction result of the image sample are obtained by the reference model. The first prediction result represents the prediction result of the reference model on the category to which each pixel in the image sample belongs.

[0055] The second feature map and the second prediction result of the image sample are obtained by the model to be trained. The second prediction result represents the prediction result of the model to be trained on the category of each pixel in the image sample.

[0056] Based on the first feature map and the second feature map, the similarity loss of the image samples is determined; based on the first prediction result and the second prediction result, the pixel-level loss of the image samples is determined; based on the second prediction result and the annotation result of the image samples, the ground value loss of the image samples is determined.

[0057] The text detection model is trained using similarity loss, pixel-level loss, and ground truth loss.

[0058] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0059] Obtain an image sample set, which includes at least one image sample;

[0060] The first feature map and the first prediction result of the image sample are obtained by the reference model. The first prediction result represents the prediction result of the reference model on the category to which each pixel in the image sample belongs.

[0061] The second feature map and the second prediction result of the image sample are obtained by the model to be trained. The second prediction result represents the prediction result of the model to be trained on the category of each pixel in the image sample.

[0062] Based on the first feature map and the second feature map, the similarity loss of the image samples is determined; based on the first prediction result and the second prediction result, the pixel-level loss of the image samples is determined; based on the second prediction result and the annotation result of the image samples, the ground value loss of the image samples is determined.

[0063] The text detection model is trained using similarity loss, pixel-level loss, and ground truth loss.

[0064] The aforementioned text detection model training method, apparatus, computer equipment, storage medium, and computer program product obtain first feature maps and first prediction results for image samples in an image sample set through a reference model, and second feature maps and second prediction results for image samples in the same set through a model to be trained. Based on the first and second feature maps, a similarity loss for the image samples is determined; based on the first and second prediction results, a pixel-level loss for the image samples is determined; and based on the second prediction result and the annotation results of the image samples, a ground truth loss for the image samples is determined. Based on the similarity loss, pixel-level loss, and ground truth loss, the model to be trained is trained to obtain the text detection model. This text detection model training method, combining the feature maps and prediction results corresponding to the reference model and the model to be trained, respectively, obtains the similarity loss, pixel-level loss, and ground truth loss for image samples. Training the model based on multiple losses improves the detection accuracy of the text detection model compared to direct training. Attached Figure Description

[0065] Figure 1 This is a diagram illustrating the application environment of a text detection model training method in one embodiment.

[0066] Figure 2 This is a flowchart illustrating the training method of a text detection model in one embodiment;

[0067] Figure 3 This is a flowchart illustrating the training method for the text detection model in yet another embodiment;

[0068] Figure 4 This is a schematic diagram of a sub-process of S204 in one embodiment;

[0069] Figure 5 This is a schematic diagram of a sub-process of S204 in another embodiment;

[0070] Figure 6 This is a schematic diagram of the overall process of training a text detection model in one embodiment;

[0071] Figure 7 This is a structural block diagram of a training device for a text detection model in one embodiment;

[0072] Figure 8 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0073] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0074] The training method for the text detection model provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104, or it can be located in the cloud or on another network server. The text detection model training method provided in this application embodiment can be executed by terminal 102 or server 104 alone, or by terminal 102 and server 104 in cooperation. Taking the execution by terminal 102 alone as an example: terminal 102 acquires an image sample set, which includes at least one image sample; it acquires a first feature map and a first prediction result of the image sample through a reference model, where the first prediction result represents the prediction result of the reference model for the category to which each pixel in the image sample belongs; it acquires a second feature map and a second prediction result of the image sample through a model to be trained, where the second prediction result represents the prediction result of the model to be trained for the category to which each pixel in the image sample belongs; it determines the similarity loss of the image sample based on the first feature map and the second feature map; it determines the pixel-level loss of the image sample based on the first prediction result and the second prediction result; it determines the ground value loss of the image sample based on the second prediction result and the annotation result of the image sample; and it trains the model to be trained based on the similarity loss, the pixel-level loss, and the ground value loss to obtain the text detection model. The terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle systems. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. The server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0075] In one embodiment, such as Figure 2 As shown, a training method for a text detection model is provided, which is then applied to a computer device (the computer device can be...). Figure 1 Taking a terminal or server as an example, the explanation includes the following steps:

[0076] S201, Obtain an image sample set, which includes at least one image sample.

[0077] The computer device acquires an image sample set. Each image sample is an image containing text information. The image sample set includes at least one image sample. Each sample in the image sample set is used to train the text detection model.

[0078] S202, obtain the first feature map and the first prediction result of the image sample through the reference model. The first prediction result represents the prediction result of the reference model on the category to which each pixel on the image sample belongs.

[0079] The reference model is a machine learning model. For example, the reference model can be a DBNet (Differentiable Binarization NET) model with a ResNet50 backbone network. Using the DBNet model enables adaptive binarization of each pixel in the image sample. The binarization threshold is learned by the network, completely incorporating the binarization step into the network during training. The final output image exhibits good robustness to the threshold.

[0080] A computer device inputs image samples into a reference model to obtain a first feature map and a first prediction result for the image samples. The reference model includes a backbone module, a feature fusion and enhancement module (neck), and a prediction module (head). Specifically, the image samples are input into the backbone module of the reference model for feature extraction, obtaining feature maps for each feature layer, and the feature map of the last feature layer is used as the first feature map. In some embodiments, the feature layers are represented as C2, C3, C4, and C5, respectively. The image sample input is downsampled sequentially through each feature layer: C2 layer is downsampled by 4 times, C3 layer by 8 times, C4 layer by 16 times, and C5 layer by 32 times. The feature map of C5 layer is used as the first feature map. The feature maps of each feature layer are concatted in the feature fusion and enhancement module to obtain an intermediate feature map. The intermediate feature map is then input into the prediction module to obtain the first prediction result for the image samples. The first prediction result represents the reference model's prediction result for the category to which each pixel in the image sample belongs.

[0081] S203, obtain the second feature map and the second prediction result of the image sample through the model to be trained. The second prediction result represents the prediction result of the model to be trained on the category to which each pixel on the image sample belongs.

[0082] The model to be trained is a machine learning model. In some embodiments, the model to be trained can be a DBNet model with ResNet50 as the backbone network. That is, the model to be trained has the same network structure as the reference model. Using the same network structure can enable the trained text detection model to achieve higher accuracy and recall than the reference model. In other embodiments, the model to be trained can also be a DBNet model with the lightweight network MobileNetV3 (an efficient model proposed for mobile and embedded devices) as the backbone network. Using a lightweight backbone network in the model to be trained enables the trained text detection model to have better computational cost and inference speed than the reference model, although there is a certain loss in accuracy compared to the reference model. However, this loss in accuracy is smaller than the loss of directly training a model with the same network structure.

[0083] The computer device acquires a second feature map and a second prediction result for the image sample using the model to be trained. Specifically, the computer device inputs the image sample into the backbone module of the model to be trained for feature extraction, obtaining feature maps for each feature layer, and using the feature map of the last feature layer as the second feature map. After the feature maps of each feature layer are merged and connected in the feature fusion and enhancement module, a second intermediate feature map is obtained. The second intermediate feature map is then input into the prediction module to obtain the second prediction result for the image sample. The second prediction result represents the reference model's prediction of the category to which each pixel in the image sample belongs.

[0084] S204, based on the first feature map and the second feature map, determine the similarity loss of the image samples; based on the first prediction result and the second prediction result, determine the pixel-level loss of the image samples; based on the second prediction result and the annotation result of the image samples, determine the ground value loss of the image samples.

[0085] The computer device determines the similarity loss of image samples based on a first feature map and a second feature map. Specifically, the similarity loss of image samples can be calculated based on the correlation between pixels in the first feature map and the correlation between pixels in the second feature map. The similarity loss is represented as the pairwise loss between any two pixels of the image sample. The computer device determines the pixel-level loss of the image sample based on the first and second prediction results. The pixel-level loss represents the loss of each pixel in the image sample. The computer device determines the ground truth loss of the image sample based on the second prediction result and the annotation result of the image sample. The ground truth loss represents the original loss of the image sample during the training of the model to be trained. The annotation result refers to the text in the image sample annotated using annotation software. The annotation result can be an image of annotated bounding boxes containing text. The size of the annotation result is equal to the size of the image sample.

[0086] S205, based on similarity loss, pixel-level loss and ground truth loss, trains the model to be trained to obtain a text detection model.

[0087] During the training process, the parameters of the training model are adjusted based on similarity loss, pixel-level loss, and ground truth loss. The model after training is the text detection model.

[0088] In the training method of the aforementioned text detection model, the first feature map and first prediction result of the image samples in the image sample set are obtained through the reference model, and the second feature map and second prediction result of the image samples in the image sample set are obtained through the model to be trained. Based on the first and second feature maps, the similarity loss of the image samples is determined; based on the first and second prediction results, the pixel-level loss of the image samples is determined; based on the second prediction result and the annotation results of the image samples, the ground truth loss of the image samples is determined. Based on the similarity loss, pixel-level loss, and ground truth loss, the model to be trained is trained to obtain the text detection model. This training method of the text detection model, combining the feature maps and prediction results corresponding to the reference model and the model to be trained respectively, obtains the similarity loss, pixel-level loss, and ground truth loss of the image samples. Training the model based on multiple losses improves the detection accuracy of the text detection model compared to direct training.

[0089] In one embodiment, determining the similarity loss of image samples based on a first feature map and a second feature map includes: obtaining the correlation between pixels in the first feature map and the correlation between pixels in the second feature map; and determining the similarity loss of image samples based on the correlation between pixels in the first feature map and the correlation between pixels in the second feature map.

[0090] The correlation between individual pixels refers to the correlation between two adjacent pixels in the feature map. This applies to the case where the feature map is a 3D image. For example, a... ij This indicates the correlation between two adjacent pixels. i and f j These represent the pixel values ​​of two adjacent rows of pixels in the feature map. ||f i ||2 and||f j ||2 represent the magnitudes of two adjacent rows of pixel vectors in the feature map. The formula for calculating pixel correlation is: a ij =f i T ·f j / (||f i ||2·||f j ||2).

[0091] The computer device inputs the pixel values ​​of each pixel in the first feature image into the pixel correlation calculation formula to obtain the correlation between pixels in the first feature image. The computer device then inputs the pixel values ​​of each pixel in the second feature image into the pixel correlation calculation formula to obtain the correlation between pixels in the second feature image. Based on the correlations between pixels in the first and second feature images, the computer device determines the similarity loss of the image samples.

[0092] In this embodiment, the similarity loss of image samples is determined by the correlation between pixels in the first feature map and the correlation between pixels in the second feature map. Combining the similarity loss in model training is beneficial to improving the detection accuracy of the text detection model.

[0093] In one embodiment, such as Figure 3 As shown, the first feature map and the second feature map are of equal size; based on the correlation between pixels in the first feature map and the correlation between pixels in the second feature map, the similarity loss of the image samples is determined, including:

[0094] S301, Obtain the width and height of the first feature map.

[0095] Specifically, the computer device acquires the width and height of the first feature map. The width refers to the width of the first feature map, and the height refers to the height of the first feature map.

[0096] S302, determine a first similarity map based on the correlation between pixels in the first feature map; determine a second similarity map based on the correlation between pixels in the second feature map.

[0097] In this process, the computer device uses the correlation between pixels in the first feature map as the image pixel value, resulting in a new image, which is the first similarity map. Similarly, the computer device uses the correlation between pixels in the second feature map as the image pixel value, resulting in a new image, which is the second similarity map.

[0098] S303, for each pixel in the first similarity map, obtain the square of the difference between the current pixel and the corresponding pixel in the second similarity map.

[0099] The first feature map and the second feature map are of equal size, as are the first similarity map and the second similarity map. For each pixel in the first similarity map, the computer device calculates the square of the difference between the current pixel's pixel value and the corresponding pixel's pixel value in the second similarity map.

[0100] S304, sum the squares of the differences between each pixel in the first similarity map to obtain the summation result.

[0101] The computer device sums the squares of the differences between each pixel in the first similarity map to obtain the summation result.

[0102] S305, based on the summation result, the width and height of the first feature map, determine the similarity loss of the image samples.

[0103] The computer device multiplies the width and height of the first feature map and squares the result to obtain a first squared result. The summation result is divided by the first squared result, and the resulting value is used as the similarity loss of the image samples. For example, the similarity loss calculation formula is as follows:

[0104]

[0105] Among them, l pa (S) represents the similarity loss of the image samples, W represents the width of the first feature map, and H represents the height of the first feature map. Represents the pixel values ​​of the first similarity map. R represents the pixel value of the second similarity map, and R represents the region of the first similarity map.

[0106] The computer device inputs the width and height of the first feature map, the pixel values ​​of each pixel in the first similarity map, and the pixel values ​​of each pixel in the second similarity map into the similarity loss calculation formula to obtain the similarity loss of the image samples.

[0107] In this embodiment, the similarity loss of image samples is determined by the width and height of the first feature map, the first similarity map, and the second similarity map. Combining the similarity loss during model training is beneficial to improving the detection accuracy of the text detection model.

[0108] In one embodiment, such as Figure 4 As shown, the first prediction result also includes a first probability map and a first approximate binary map. The first probability map represents the probability that each pixel in the image sample predicted by the reference module belongs to text, and the first approximate binary map represents the binary classification result of each pixel in the image sample predicted by the reference model as either text or non-text. The second prediction result also includes a second probability map and a second approximate binary map. The second probability map represents the probability that each pixel in the image sample predicted by the model to be trained belongs to text, and the first approximate binary map represents the binary classification result of each pixel in the image sample predicted by the model to be trained as either text or non-text. Based on the first and second prediction results, the pixel-level loss of the image sample is determined, including:

[0109] S402, the first probability map is expanded using the expansion function to obtain the first expansion result; the first approximate binary map is expanded using the expansion function to obtain the second expansion result.

[0110] The dilation function is used to expand the highlighted areas in an image, resulting in a larger highlighted area compared to the original image. The computer device uses the dilation function to dilate the first probability map, obtaining a first dilation result. The computer device also uses the dilation function to dilate the first approximate binary map, obtaining a second dilation result. The first probability map represents the probability that each pixel in the image sample predicted by the reference module belongs to text, while the first approximate binary map represents the binary classification result of each pixel in the image sample predicted by the reference model as either text or non-text. The probability of each pixel belonging to text represented in the first probability map is smaller than the labeled result of the image sample, and the probability of each pixel belonging to text represented in the first approximate binary map is smaller than the labeled result of the image sample. Therefore, using the dilation function to dilate both the first and second probability maps ensures that the first and second dilation results are closer to the labeled result of the image sample.

[0111] S404. Based on the first dilation result, the second dilation result, the second probability map, and the second approximate binary map, determine the pixel-level loss of the image sample.

[0112] The computer device calculates the first loss based on the first dilation result and the second probability map using the cross-entropy loss calculation formula. The computer device determines the second loss based on the second dilation result and the second approximate binary map using the dice loss calculation formula. The computer device then determines the pixel-level loss of the image sample based on the first and second losses. In some embodiments, the sum of the first and second losses can be used as the pixel-level loss of the image sample. In other embodiments, the sum of the first loss (a first preset multiple) and the second loss (a second preset multiple) can be used as the pixel-level loss of the image sample. In some embodiments, the pixel-level loss calculation formula is:

[0113] loss distill =γl p (S out ,f dila (T out ))+l b (S out ,f dila (T out ))

[0114] Where, loss distill γ represents the pixel-level loss of the image sample, and γ represents the hyperparameter; in some embodiments, γ can be 5. dilaThe expansion function is represented by f. In some embodiments, f dila The kernel can be a second-order matrix with a parameter of 1. p (S out ,f dila (T out )) represents the first loss, which is the cross-entropy loss. b (S out ,f dila (T out )) indicates the second loss, which is the loss from the dice.

[0115] The computer device inputs the first probability map, the second probability map, the first approximate binary map, the second approximate binary map, and the dilation function into the pixel-level loss calculation formula to obtain the pixel-level loss of the image sample.

[0116] In this embodiment, the first probability map and the first approximate binary map are dilated using a dilation function. Based on the dilated result, the second probability map, and the second approximate binary map, the pixel-level loss of the image sample is determined. Combining the pixel-level loss of the image sample in the model training is beneficial to improving the detection accuracy of the text detection model.

[0117] In one embodiment, such as Figure 5 As shown, the first prediction result includes a first threshold map, which represents the probability that each pixel in the image sample predicted by the reference model belongs to the text boundary; based on the second prediction result and the annotation result of the image sample, the ground value loss of the image sample is determined, including:

[0118] S502, based on the annotation results of the first probability map and the image samples, determine the first ground truth loss; based on the annotation results of the first threshold map and the image samples, determine the second ground truth loss; based on the annotation results of the first approximate binary map and the image samples, determine the third ground truth loss.

[0119] Specifically, the computer device determines the first ground truth loss based on the annotation results of the first probability map and image samples using the cross-entropy loss formula. The computer device then determines the second ground truth loss based on the annotation results of the first threshold map and image samples using the dice loss formula. Finally, the computer device determines the third ground truth loss based on the annotation results of the first approximate binary map and image samples using the threshold map loss formula.

[0120] S504, determine the ground value loss of the image sample based on the first ground value loss, the second ground value loss, and the third ground value loss.

[0121] In this method, the computer device adds the first truth loss (at a third preset multiple), the second truth loss (at a fourth preset multiple), and the third truth loss (at a fifth preset multiple), and uses the sum as the truth loss of the image sample. In some embodiments, the formula for calculating the truth loss is:

[0122] loss gt (S out ,gt)=l p (S out ,gt)+αl b (S out ,gt)+βl t (S out ,gt)

[0123] Where, loss gt (S out ,gt) represents the ground truth loss of the image samples, l p (S out ,gt) represents the first truth loss, which is the cross-entropy loss. t This represents the second ground truth loss, which is the threshold map loss. b (S out ,gt) represents the third truth value loss, which is the dice loss. α and β are preset coefficients.

[0124] In this embodiment, the first ground truth loss, the second ground truth loss, and the third ground truth loss are obtained from the first probability map, the first threshold map, the first approximate binary map, and the annotation results, respectively, thereby obtaining the ground truth loss of the image sample. Combining the ground truth loss of the image sample with the model training is beneficial to improving the detection accuracy of the text detection model.

[0125] In one embodiment, a text detection model is trained based on similarity loss, pixel-level loss, and ground truth loss, including: determining a target loss based on similarity loss, pixel-level loss, and ground truth loss; adjusting the parameters in the training model based on the target loss; continuing training based on the adjusted model until the training stopping condition is met, and using the model obtained after training stops as the text detection model.

[0126] The computer device sums the similarity loss, pixel-level loss, and ground truth loss of the image samples to obtain the overall loss of the image samples. The target loss is the average of the overall losses of all image samples in the image sample set. The computer device adjusts the parameters in the model to be trained based on the target loss to obtain the adjusted model. Training continues using the adjusted model until the training stopping condition is met, at which point the model obtained after training stops is used as the text detection model.

[0127] In this embodiment, a target loss is determined through similarity loss, pixel-level loss, and ground truth loss. This target loss is used to adjust the model to be trained. The adjusted model, after further training, becomes the text detection model. This method of training the model based on multiple losses improves the detection accuracy of the text detection model compared to direct training.

[0128] In one embodiment, the training method for the text detection model further includes: acquiring a test image; inputting the test image into the text detection model to obtain the prediction results of the category to which each pixel in the test image belongs; and determining and marking the text region in the test image based on the prediction results of the category to which each pixel in the test image belongs.

[0129] The process involves a computer acquiring a test image containing text. The computer inputs the test image into a text detection model to obtain a prediction of the category of each pixel in the test image. The prediction result indicates whether a pixel in the test image belongs to the text category or not. Based on the prediction results, the computer identifies and marks text regions in the test image. The marked text regions are output as bounding boxes.

[0130] In this embodiment, by inputting the image to be tested into the text detection model, the text region in the image to be tested is determined and marked based on the prediction results, and the text detection is performed based on the text detection model, thereby improving the accuracy of text detection.

[0131] To illustrate the training method and effectiveness of the text detection model in this solution in detail, a specific implementation example is provided below:

[0132] This addresses common text detection applications in financial form images. For example... Figure 6 The diagram illustrates the overall process of training and applying a text detection model. A computer device acquires an image sample set, which includes at least one image sample. A first feature map and a first prediction result are obtained from the image sample using a reference model. The first prediction result represents the reference model's prediction of the category of each pixel in the image sample. The first prediction result also includes a first probability map, a first approximate binary map, and a first threshold map. The first probability map represents the probability predicted by the reference module that each pixel in the image sample belongs to text; the first approximate binary map represents the binary classification result predicted by the reference model that each pixel in the image sample belongs to either text or non-text; and the first threshold map represents the probability predicted by the reference model that each pixel in the image sample belongs to a text boundary.

[0133] The training model obtains a second feature map and a second prediction result for the image sample. The second prediction result represents the training model's prediction of the category of each pixel in the image sample. The second prediction result also includes a second probability map and a second approximate binary map. The second probability map represents the probability that each pixel in the image sample is text, as predicted by the training model, and the first approximate binary map represents the binary classification result of each pixel in the image sample as either text or non-text, as predicted by the training model.

[0134] The process involves obtaining the correlation between pixels in the first feature map and the correlation between pixels in the second feature map; ensuring the first and second feature maps are of equal size; obtaining the width and height of the first feature map; determining a first similarity map based on the correlation between pixels in the first feature map; determining a second similarity map based on the correlation between pixels in the second feature map; for each pixel in the first similarity map, obtaining the square of the difference between the current pixel and the corresponding pixel in the second similarity map; summing the squares of the differences between corresponding pixels in the first similarity map to obtain a summation result; and determining the similarity loss of the image samples based on the summation result, the width, and the height of the first feature map. For example, the similarity loss calculation formula is as follows:

[0135]

[0136] Among them, l pa (S) represents the similarity loss of the image samples, W represents the width of the first feature map, and H represents the height of the first feature map. Represents the pixel values ​​of the first similarity map. R represents the pixel value of the second similarity map, and R represents the region of the first similarity map.

[0137] The first probability map is dilated using a dilation function to obtain a first dilation result; the first approximate binary map is then dilated using the same dilation function to obtain a second dilation result. Based on the first dilation result, the second dilation result, the second probability map, and the second approximate binary map, the pixel-level loss of the image samples is determined. The formula for calculating the pixel-level loss is:

[0138] loss distill =γl p (S out ,f dila (T out ))+l b (S out ,f dila (T out ))

[0139] Where, loss distill γ represents the pixel-level loss of the image sample, and γ represents the hyperparameter; in some embodiments, γ can be 5.dila The expansion function is represented by f. In some embodiments, f dila The kernel can be a second-order matrix with a parameter of 1. p (S out ,f dila (T out )) represents the first loss, which is the cross-entropy loss. b (S out ,f dila (T out )) indicates the second loss, which is the loss from the dice.

[0140] Based on the annotation results of the first probability map and image samples, a first ground truth loss is determined; based on the annotation results of the first threshold map and image samples, a second ground truth loss is determined; based on the annotation results of the first approximate binary map and image samples, a third ground truth loss is determined; and based on the first ground truth loss, the second ground truth loss, and the third ground truth loss, the ground truth loss of the image samples is determined. The formula for calculating the ground truth loss is:

[0141] loss gt (S out ,gt)=l p (S out ,gt)+αl b (S out ,gt)+βl t (S out ,gt)

[0142] Where, loss gt (S out ,gt) represents the ground truth loss of the image samples, l p (S out ,gt) represents the first truth loss, which is the cross-entropy loss. t This represents the second ground truth loss, which is the threshold map loss. b (S out ,gt) represents the third truth value loss, which is the dice loss. α and β are preset coefficients.

[0143] The target loss is determined based on similarity loss, pixel-level loss, and ground truth loss. Specifically, the computer device sums the similarity loss, pixel-level loss, and ground truth loss of the image samples to obtain the overall loss of the image samples. The target loss is the average of the overall losses of all image samples in the image sample set. Based on the target loss, the parameters in the model to be trained are adjusted; training continues based on the adjusted model until the training stopping condition is met, at which point the model obtained after training stops is used as the text detection model.

[0144] After training the text detection model, the process includes: acquiring the test image. The test image is input into the text detection model to obtain the predicted category of each pixel in the test image. Based on the predicted category of each pixel in the test image, the text region is identified and labeled in the test image.

[0145] The training device for the aforementioned text detection model obtains the first feature map and first prediction result of image samples in the image sample set through a reference model, and obtains the second feature map and second prediction result of image samples in the image sample set through the model to be trained. Based on the first and second feature maps, the similarity loss of the image samples is determined; based on the first and second prediction results, the pixel-level loss of the image samples is determined; based on the second prediction result and the annotation results of the image samples, the ground truth loss of the image samples is determined. Based on the similarity loss, pixel-level loss, and ground truth loss, the model to be trained is trained to obtain the text detection model. This training method for the text detection model, combining the feature maps and prediction results corresponding to the reference model and the model to be trained respectively, obtains the similarity loss, pixel-level loss, and ground truth loss of image samples. Training the model based on multiple losses improves the detection accuracy of the text detection model compared to direct training.

[0146] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0147] Based on the same inventive concept, this application also provides a training apparatus for a text detection model to implement the training method for the text detection model described above. The solution provided by this apparatus is similar to the implementation described in the above method. Therefore, the specific limitations of one or more text detection model training apparatus embodiments provided below can be found in the limitations of the text detection model training method described above, and will not be repeated here.

[0148] In one embodiment, such as Figure 7As shown, a training device 100 for a text detection model is provided, including: a sample acquisition module 110, a first result acquisition module 120, a second result acquisition module 130, a loss determination module 140, and a model training module 150, wherein:

[0149] The sample acquisition module 110 is used to acquire an image sample set, which includes at least one image sample.

[0150] The first result acquisition module 120 is used to acquire the first feature map and the first prediction result of the image sample through the reference model. The first prediction result represents the prediction result of the reference model for the category to which each pixel on the image sample belongs.

[0151] The second result acquisition module 130 is used to acquire the second feature map and the second prediction result of the image sample through the model to be trained. The second prediction result represents the prediction result of the model to be trained on the category to which each pixel on the image sample belongs.

[0152] The loss determination module 140 is used to determine the similarity loss of image samples based on the first feature map and the second feature map; determine the pixel-level loss of image samples based on the first prediction result and the second prediction result; and determine the ground value loss of image samples based on the second prediction result and the annotation result of image samples.

[0153] The model training module 150 is used to train the model to be trained based on similarity loss, pixel-level loss and ground truth loss to obtain a text detection model.

[0154] The training device for the aforementioned text detection model obtains the first feature map and first prediction result of image samples in the image sample set through a reference model, and obtains the second feature map and second prediction result of image samples in the image sample set through the model to be trained. Based on the first and second feature maps, the similarity loss of the image samples is determined; based on the first and second prediction results, the pixel-level loss of the image samples is determined; based on the second prediction result and the annotation results of the image samples, the ground truth loss of the image samples is determined. Based on the similarity loss, pixel-level loss, and ground truth loss, the model to be trained is trained to obtain the text detection model. This training method for the text detection model, combining the feature maps and prediction results corresponding to the reference model and the model to be trained respectively, obtains the similarity loss, pixel-level loss, and ground truth loss of image samples. Training the model based on multiple losses improves the detection accuracy of the text detection model compared to direct training.

[0155] In one embodiment, in determining the similarity loss of image samples based on the first feature map and the second feature map, the loss determination module 140 is further configured to: obtain the correlation between each pixel in the first feature map and the correlation between each pixel in the second feature map; and determine the similarity loss of image samples based on the correlation between each pixel in the first feature map and the correlation between each pixel in the second feature map.

[0156] In one embodiment, where the first feature map and the second feature map are of equal size; regarding determining the similarity loss of the image samples based on the correlation between pixels in the first feature map and the correlation between pixels in the second feature map, the loss determination module 140 is further configured to: obtain the width and height of the first feature map; determine a first similarity map based on the correlation between pixels in the first feature map; determine a second similarity map based on the correlation between pixels in the second feature map; for each pixel in the first similarity map, obtain the square of the difference between the current pixel and the corresponding pixel in the second similarity map; sum the squares of the differences between the corresponding pixels in the first similarity map to obtain a summation result; and determine the similarity loss of the image samples based on the summation result, the width and height of the first feature map.

[0157] In one embodiment, the first prediction result further includes a first probability map and a first approximate binary map. The first probability map represents the probability that each pixel in the image sample predicted by the reference module belongs to text, and the first approximate binary map represents the binary classification result of each pixel in the image sample predicted by the reference model as either text or non-text. The second prediction result further includes a second probability map and a second approximate binary map. The second probability map represents the probability that each pixel in the image sample predicted by the model to be trained belongs to text, and the first approximate binary map represents the binary classification result of each pixel in the image sample predicted by the model to be trained as either text or non-text. Regarding determining the pixel-level loss of the image sample based on the first and second prediction results, the loss determination module 140 is further configured to: dilate the first probability map using a dilation function to obtain a first dilation result; dilate the first approximate binary map using a dilation function to obtain a second dilation result; and determine the pixel-level loss of the image sample based on the first dilation result, the second dilation result, the second probability map, and the second approximate binary map.

[0158] In one embodiment, where the first prediction result includes a first threshold map, the first threshold map representing the probability that each pixel on the image sample predicted by the reference model belongs to a text boundary; and regarding determining the ground value loss of the image sample based on the second prediction result and the annotation result of the image sample, the loss determination module 140 is further configured to: determine a first ground value loss based on the first probability map and the annotation result of the image sample; determine a second ground value loss based on the first threshold map and the annotation result of the image sample; determine a third ground value loss based on the first approximate binary map and the annotation result of the image sample; and determine the ground value loss of the image sample based on the first ground value loss, the second ground value loss, and the third ground value loss.

[0159] In one embodiment, in training the model to obtain a text detection model based on similarity loss, pixel-level loss, and ground truth loss, the model training module 150 is further configured to: determine the target loss based on similarity loss, pixel-level loss, and ground truth loss; adjust the parameters in the model to be trained according to the target loss; continue training based on the adjusted model until the training stopping condition is met, and then stop training, using the model obtained after training stops as the text detection model.

[0160] In one embodiment, the training device 100 for the text detection model is further configured to: acquire a test image; input the test image into the text detection model to obtain the prediction results of the category to which each pixel in the test image belongs; and determine and mark the text region in the test image based on the prediction results of the category to which each pixel in the test image belongs.

[0161] The modules in the training device of the aforementioned text detection model can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0162] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 8As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores image sample sets, first feature maps, first prediction results, second feature maps, second prediction results, similarity loss, pixel-level loss, ground truth loss, a reference model, a model to be trained, and a text detection model. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external computer devices via a network connection. When the computer program is executed by the processor, it implements a training method for a text detection model.

[0163] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0164] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0165] An image sample set is obtained, including at least one image sample. A first feature map and a first prediction result of the image sample are obtained through a reference model. The first prediction result represents the prediction result of the reference model for the category of each pixel in the image sample. A second feature map and a second prediction result of the image sample are obtained through a model to be trained. The second prediction result represents the prediction result of the model to be trained for the category of each pixel in the image sample. Based on the first and second feature maps, the similarity loss of the image sample is determined. Based on the first and second prediction results, the pixel-level loss of the image sample is determined. Based on the second prediction result and the annotation result of the image sample, the ground value loss of the image sample is determined. Based on the similarity loss, pixel-level loss, and ground value loss, the model to be trained is trained to obtain a text detection model.

[0166] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0167] Obtain the correlation between each pixel in the first feature map and the correlation between each pixel in the second feature map; based on the correlation between each pixel in the first feature map and the correlation between each pixel in the second feature map, determine the similarity loss of the image samples.

[0168] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0169] The first feature map and the second feature map are of equal size; the width and height of the first feature map are obtained; a first similarity map is determined based on the correlation between pixels in the first feature map; a second similarity map is determined based on the correlation between pixels in the second feature map; for each pixel in the first similarity map, the square of the difference between the current pixel and the corresponding pixel in the second similarity map is obtained; the squares of the differences between the corresponding pixels in the first similarity map are summed to obtain the summation result; based on the summation result, the width and height of the first feature map, the similarity loss of the image samples is determined.

[0170] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0171] The first prediction result also includes a first probability map and a first approximate binary map. The first probability map represents the probability that each pixel in the image sample predicted by the reference module belongs to text, and the first approximate binary map represents the binary classification result of each pixel in the image sample predicted by the reference model as either text or non-text. The second prediction result also includes a second probability map and a second approximate binary map. The second probability map represents the probability that each pixel in the image sample predicted by the model to be trained belongs to text, and the first approximate binary map represents the binary classification result of each pixel in the image sample predicted by the model to be trained as either text or non-text. The first probability map is dilated using a dilation function to obtain a first dilation result. The first approximate binary map is dilated using a dilation function to obtain a second dilation result. Based on the first dilation result, the second dilation result, the second probability map, and the second approximate binary map, the pixel-level loss of the image sample is determined.

[0172] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0173] The first prediction result includes a first threshold map, which represents the probability that each pixel in the image sample predicted by the reference model belongs to the text boundary; a first ground truth loss is determined based on the first probability map and the annotation results of the image sample; a second ground truth loss is determined based on the first threshold map and the annotation results of the image sample; a third ground truth loss is determined based on the first approximate binary map and the annotation results of the image sample; and the ground truth loss of the image sample is determined based on the first ground truth loss, the second ground truth loss, and the third ground truth loss.

[0174] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0175] The target loss is determined based on similarity loss, pixel-level loss, and ground truth loss. The parameters in the model to be trained are adjusted according to the target loss. Training continues based on the adjusted model until the training stopping condition is met, and the model obtained after training stops is used as the text detection model.

[0176] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0177] Acquire the image to be tested; input the image to be tested into the text detection model to obtain the prediction results of the category of each pixel in the image to be tested; based on the prediction results of the category of each pixel in the image to be tested, determine and mark the text region in the image to be tested.

[0178] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0179] An image sample set is obtained, including at least one image sample. A first feature map and a first prediction result of the image sample are obtained through a reference model. The first prediction result represents the prediction result of the reference model for the category of each pixel in the image sample. A second feature map and a second prediction result of the image sample are obtained through a model to be trained. The second prediction result represents the prediction result of the model to be trained for the category of each pixel in the image sample. Based on the first and second feature maps, the similarity loss of the image sample is determined. Based on the first and second prediction results, the pixel-level loss of the image sample is determined. Based on the second prediction result and the annotation result of the image sample, the ground value loss of the image sample is determined. Based on the similarity loss, pixel-level loss, and ground value loss, the model to be trained is trained to obtain a text detection model.

[0180] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0181] Obtain the correlation between each pixel in the first feature map and the correlation between each pixel in the second feature map; based on the correlation between each pixel in the first feature map and the correlation between each pixel in the second feature map, determine the similarity loss of the image samples.

[0182] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0183] The first feature map and the second feature map are of equal size; the width and height of the first feature map are obtained; a first similarity map is determined based on the correlation between pixels in the first feature map; a second similarity map is determined based on the correlation between pixels in the second feature map; for each pixel in the first similarity map, the square of the difference between the current pixel and the corresponding pixel in the second similarity map is obtained; the squares of the differences between the corresponding pixels in the first similarity map are summed to obtain the summation result; based on the summation result, the width and height of the first feature map, the similarity loss of the image samples is determined.

[0184] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0185] The first prediction result also includes a first probability map and a first approximate binary map. The first probability map represents the probability that each pixel in the image sample predicted by the reference module belongs to text, and the first approximate binary map represents the binary classification result of each pixel in the image sample predicted by the reference model as either text or non-text. The second prediction result also includes a second probability map and a second approximate binary map. The second probability map represents the probability that each pixel in the image sample predicted by the model to be trained belongs to text, and the first approximate binary map represents the binary classification result of each pixel in the image sample predicted by the model to be trained as either text or non-text. The first probability map is dilated using a dilation function to obtain a first dilation result. The first approximate binary map is dilated using a dilation function to obtain a second dilation result. Based on the first dilation result, the second dilation result, the second probability map, and the second approximate binary map, the pixel-level loss of the image sample is determined.

[0186] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0187] The first prediction result includes a first threshold map, which represents the probability that each pixel in the image sample predicted by the reference model belongs to the text boundary; a first ground truth loss is determined based on the first probability map and the annotation results of the image sample; a second ground truth loss is determined based on the first threshold map and the annotation results of the image sample; a third ground truth loss is determined based on the first approximate binary map and the annotation results of the image sample; and the ground truth loss of the image sample is determined based on the first ground truth loss, the second ground truth loss, and the third ground truth loss.

[0188] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0189] The target loss is determined based on similarity loss, pixel-level loss, and ground truth loss. The parameters in the model to be trained are adjusted according to the target loss. Training continues based on the adjusted model until the training stopping condition is met, and the model obtained after training stops is used as the text detection model.

[0190] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0191] Acquire the image to be tested; input the image to be tested into the text detection model to obtain the prediction results of the category of each pixel in the image to be tested; based on the prediction results of the category of each pixel in the image to be tested, determine and mark the text region in the image to be tested.

[0192] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0193] An image sample set is obtained, including at least one image sample. A first feature map and a first prediction result of the image sample are obtained through a reference model. The first prediction result represents the prediction result of the reference model for the category of each pixel in the image sample. A second feature map and a second prediction result of the image sample are obtained through a model to be trained. The second prediction result represents the prediction result of the model to be trained for the category of each pixel in the image sample. Based on the first and second feature maps, the similarity loss of the image sample is determined. Based on the first and second prediction results, the pixel-level loss of the image sample is determined. Based on the second prediction result and the annotation result of the image sample, the ground value loss of the image sample is determined. Based on the similarity loss, pixel-level loss, and ground value loss, the model to be trained is trained to obtain a text detection model.

[0194] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0195] Obtain the correlation between each pixel in the first feature map and the correlation between each pixel in the second feature map; based on the correlation between each pixel in the first feature map and the correlation between each pixel in the second feature map, determine the similarity loss of the image samples.

[0196] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0197] The first feature map and the second feature map are of equal size. Based on the correlation between pixels in the first feature map and the correlation between pixels in the second feature map, the similarity loss of the image samples is determined, including: obtaining the width and height of the first feature map; determining a first similarity map based on the correlation between pixels in the first feature map; determining a second similarity map based on the correlation between pixels in the second feature map; for each pixel in the first similarity map, obtaining the square of the difference between the current pixel and the corresponding pixel in the second similarity map; summing the squares of the differences between corresponding pixels in the first similarity map to obtain a summation result; and determining the similarity loss of the image samples based on the summation result, the width and height of the first feature map.

[0198] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0199] The first prediction result also includes a first probability map and a first approximate binary map. The first probability map represents the probability that each pixel in the image sample predicted by the reference module belongs to text, and the first approximate binary map represents the binary classification result of each pixel in the image sample predicted by the reference model as either text or non-text. The second prediction result also includes a second probability map and a second approximate binary map. The second probability map represents the probability that each pixel in the image sample predicted by the model to be trained belongs to text, and the first approximate binary map represents the binary classification result of each pixel in the image sample predicted by the model to be trained as either text or non-text. The first probability map is dilated using a dilation function to obtain a first dilation result. The first approximate binary map is dilated using a dilation function to obtain a second dilation result. Based on the first dilation result, the second dilation result, the second probability map, and the second approximate binary map, the pixel-level loss of the image sample is determined.

[0200] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0201] The first prediction result includes a first threshold map, which represents the probability that each pixel in the image sample predicted by the reference model belongs to the text boundary; a first ground truth loss is determined based on the first probability map and the annotation results of the image sample; a second ground truth loss is determined based on the first threshold map and the annotation results of the image sample; a third ground truth loss is determined based on the first approximate binary map and the annotation results of the image sample; and the ground truth loss of the image sample is determined based on the first ground truth loss, the second ground truth loss, and the third ground truth loss.

[0202] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0203] The target loss is determined based on similarity loss, pixel-level loss, and ground truth loss. The parameters in the model to be trained are adjusted according to the target loss. Training continues based on the adjusted model until the training stopping condition is met, and the model obtained after training stops is used as the text detection model.

[0204] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0205] Acquire the image to be tested; input the image to be tested into the text detection model to obtain the prediction results of the category of each pixel in the image to be tested; based on the prediction results of the category of each pixel in the image to be tested, determine and mark the text region in the image to be tested.

[0206] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0207] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. The databases involved in the embodiments provided in this application can include at least one of relational databases and non-relational databases. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0208] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0209] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A training method for a text detection model, characterized in that, The method includes: Obtain an image sample set, wherein the image sample set includes at least one image sample; The first feature map and the first prediction result of the image sample are obtained by the reference model. The first prediction result represents the prediction result of the reference model for the category to which each pixel in the image sample belongs. The second feature map and the second prediction result of the image sample are obtained by the model to be trained. The second prediction result represents the prediction result of the model to be trained on the category to which each pixel in the image sample belongs. Based on the first feature map and the second feature map, the similarity loss of the image sample is determined; based on the first prediction result and the second prediction result, the pixel-level loss of the image sample is determined; based on the second prediction result and the annotation result of the image sample, the ground value loss of the image sample is determined. The text detection model is trained based on the similarity loss, the pixel-level loss, and the ground truth loss. The first prediction result further includes a first probability map and a first approximate binary map. The first probability map represents the probability that each pixel in the image sample belongs to text, as predicted by the reference model. The first approximate binary map represents the binary classification result of each pixel in the image sample belonging to text or non-text, as predicted by the reference model. The second prediction result further includes a second probability map and a second approximate binary map. The second probability map represents the probability that each pixel in the image sample belongs to text, as predicted by the model to be trained. The second approximate binary map represents the binary classification result of each pixel in the image sample belonging to text or non-text, as predicted by the model to be trained. The step of determining the pixel-level loss of the image sample based on the first prediction result and the second prediction result includes: The first probability map is expanded using an expansion function to obtain the first expansion result; The first approximate binary image is expanded using the expansion function to obtain a second expansion result; The pixel-level loss of the image sample is determined based on the first dilation result, the second dilation result, the second probability map, and the second approximate binary map.

2. The method according to claim 1, characterized in that, The step of determining the similarity loss of the image samples based on the first feature map and the second feature map includes: Obtain the correlation between pixels in the first feature map and the correlation between pixels in the second feature map; The similarity loss of the image samples is determined based on the correlation between pixels in the first feature map and the correlation between pixels in the second feature map.

3. The method according to claim 2, characterized in that, The first feature map and the second feature map are of equal size; The step of determining the similarity loss of the image samples based on the correlation between pixels in the first feature map and the correlation between pixels in the second feature map includes: Obtain the width and height of the first feature map; Based on the correlation between pixels in the first feature map, a first similarity map is determined; A second similarity map is determined based on the correlation between pixels in the second feature map; For each pixel in the first similarity map, obtain the square of the difference between the current pixel and the corresponding pixel in the second similarity map; The summation result is obtained by summing the squares of the differences corresponding to each pixel in the first similarity map; Based on the summation result, the width and height of the first feature map, the similarity loss of the image samples is determined.

4. The method according to claim 1, characterized in that, The first prediction result includes a first threshold map, which represents the probability that each pixel on the image sample belongs to the text boundary as predicted by the reference model. The step of determining the ground truth loss of the image samples based on the second prediction result and the annotation result of the image samples includes: Based on the first probability map and the annotation results of the image samples, a first ground truth loss is determined; Based on the first threshold map and the annotation results of the image samples, a second ground truth loss is determined; Based on the annotation results of the first approximate binary image and the image samples, a third ground truth loss is determined; The ground value loss of the image sample is determined based on the first ground value loss, the second ground value loss, and the third ground value loss.

5. The method according to claim 1, characterized in that, The text detection model is trained based on the similarity loss, the pixel-level loss, and the ground truth loss to obtain the text detection model, including: The target loss is determined based on the similarity loss, the pixel-level loss, and the ground truth loss. Based on the target loss, the parameters in the model to be trained are adjusted; Training continues based on the adjusted model until the training stops when the stopping condition is met. The model obtained after training stops is then used as the text detection model.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: Acquire the image to be tested; The image to be tested is input into the text detection model to obtain the prediction results of the category of each pixel in the image to be tested; Based on the prediction results of the category to which each pixel belongs in the image to be tested, the text region is determined and marked in the image to be tested.

7. A training device for a text detection model, characterized in that, The device includes: A sample acquisition module is used to acquire an image sample set, wherein the image sample set includes at least one image sample; The first result acquisition module is used to acquire the first feature map and the first prediction result of the image sample through the reference model. The first prediction result represents the prediction result of the reference model for the category to which each pixel in the image sample belongs. The second result acquisition module is used to acquire the second feature map and the second prediction result of the image sample through the model to be trained. The second prediction result represents the prediction result of the model to be trained on the category to which each pixel in the image sample belongs. The loss determination module is used to determine the similarity loss of the image sample based on the first feature map and the second feature map; determine the pixel-level loss of the image sample based on the first prediction result and the second prediction result; and determine the ground value loss of the image sample based on the second prediction result and the annotation result of the image sample. The model training module is used to train the model to be trained based on the similarity loss, the pixel-level loss, and the ground truth loss to obtain a text detection model. The first prediction result further includes a first probability map and a first approximate binary map. The first probability map represents the probability that each pixel in the image sample belongs to text, as predicted by the reference model. The first approximate binary map represents the binary classification result of each pixel in the image sample belonging to text or non-text, as predicted by the reference model. The second prediction result further includes a second probability map and a second approximate binary map. The second probability map represents the probability that each pixel in the image sample belongs to text, as predicted by the model to be trained. The second approximate binary map represents the binary classification result of each pixel in the image sample belonging to text or non-text, as predicted by the model to be trained. The loss determination module is further configured to dilate the first probability map using a dilation function to obtain a first dilation result; dilate the first approximate binary map using the dilation function to obtain a second dilation result; and determine the pixel-level loss of the image sample based on the first dilation result, the second dilation result, the second probability map, and the second approximate binary map.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images