Character detection method, device, electronic device and readable medium

Abstract

The embodiment of the present invention provides a method, device, electronic device and readable medium for character detection, the method comprising: obtaining an image containing characters to be detected, inputting the image into a feature extraction model to obtain image features of the image, inputting the image features of the image into a character positioning model, obtaining a positioning frame of the character and image features of the positioning frame, reconstructing the image features of the positioning frame to obtain image features of the reconstructed positioning frame, obtaining a reconstruction error value between the image features of the reconstructed positioning frame and the image features of the positioning frame, the reconstruction error value being used to determine whether the character is an abnormal character. By applying the embodiment of the present invention, the positioning frame of a single character in a vertical, horizontal, inclined and arc-shaped text can be determined, and then the single character in the positioning frame can be detected to determine whether it is an abnormal character, thereby realizing the detection of characters in vertical, horizontal, inclined and arc-shaped texts, and being applicable to various complex scenarios of text detection.

Description

Technical Field

[0001] The present invention relates to the technical field of character detection, and in particular to a character detection method, a character detection device, an electronic device and a computer-readable medium. Background Art

[0002] For printed characters, there are often cases where the characters are printed abnormally or incorrectly. Therefore, it is necessary to detect the printed characters and identify various characters with printing abnormalities or errors to improve the printing quality of printed products.

[0003] At present, various models are usually used to detect printed characters, such as deep learning models, neural network models, etc.; however, these models detect entire lines or entire paragraphs of text and are only suitable for detecting printed characters in specific scenarios, such as detecting characters in vertical or horizontal text. However, the effect of detecting characters in curved text or inclined text is not good, resulting in the inability to accurately detect printed abnormalities or erroneous characters. Summary of the invention

[0004] The embodiments of the present invention provide a character detection method, device, electronic device and computer-readable storage medium to solve the problem that printed characters that are not in a specific scene cannot be accurately detected.

[0005] The embodiment of the present invention discloses a character detection method, comprising:

[0006] Obtain an image containing characters to be detected;

[0007] Inputting the image into a feature extraction model to obtain image features of the image; wherein the feature extraction module is trained based on a standard image data set and a positive sample image data set containing characters without printing anomalies;

[0008] Inputting the image features of the image into a character positioning model to obtain a positioning frame of the character and the image features of the positioning frame; wherein the character positioning model is trained based on a positive sample image dataset annotated with the positioning frame and the positive sample image dataset;

[0009] Reconstructing the image features of the positioning frame to obtain reconstructed image features of the positioning frame;

[0010] A reconstruction error value between the image feature of the reconstructed positioning frame and the image feature of the positioning frame is obtained, and the reconstruction error value is used to determine whether the character is an abnormal character.

[0011] Optionally, the feature extraction model includes a MobileNet network and a feature pyramid network, and the step of inputting the image into the feature extraction model to obtain image features of the image includes:

[0012] Inputting the image into the MobileNet network to obtain multi-layer image features of the image;

[0013] The multi-layer image features are fused through the feature pyramid network to obtain the image features of the image.

[0014] Optionally, the character positioning model includes a region generation network, a classification network and a regression prediction network, and the inputting the image features of the image into the character positioning model to obtain the positioning box of the character and the image features of the positioning box includes:

[0015] Inputting the image features of the image into the region generation network to obtain a candidate box;

[0016] Determine the candidate box containing the character through the classification network;

[0017] The candidate box containing the character is subjected to frame regression processing through the regression prediction network to obtain a positioning box of the character, and image features of the positioning box are extracted.

[0018] Optionally, performing border regression processing on the candidate box containing the character through the regression prediction network to obtain a positioning box of the character includes:

[0019] Predicting the probability value of the pixel point in the candidate frame by the regression prediction network; wherein the probability value represents the probability that the pixel point is the character;

[0020] Based on the probability values ​​of the pixels in the candidate frame, a positioning frame of the character is determined.

[0021] Optionally, the reconstructing the image features of the positioning frame to obtain the reconstructed image features of the positioning frame includes:

[0022] Performing global pooling processing on the image features of the positioning frame to obtain a feature vector of the positioning frame;

[0023] Normalizing the feature vector of the positioning frame to obtain a normalized feature vector;

[0024] Reconstructing the normalized feature vector through an autoencoder to obtain a reconstructed feature vector;

[0025] The obtaining of the reconstruction error value between the image feature of the reconstructed positioning frame and the image feature of the positioning frame, wherein the reconstruction error value is used to determine whether the character is an abnormal character, includes:

[0026] A reconstruction error value between the reconstructed feature vector and the normalized feature vector is calculated, and the reconstruction error value is used to determine that the character is an abnormal character when it is greater than a first preset threshold.

[0027] Optionally, the standard image dataset is an ImageNet image dataset, and before inputting the image into the feature extraction model, the method further includes:

[0028] Obtain a feature extraction model pre-trained on the ImageNet image dataset;

[0029] Based on the positive sample image data set, the pre-trained feature extraction model is adjusted to obtain a trained feature extraction model.

[0030] Optionally, before inputting the image features of the image into the character positioning model, the method further includes:

[0031] Inputting a first positioning frame annotated image in the positioning frame annotated positive sample image data set into the feature extraction model to obtain image features of the first positioning frame annotated image;

[0032] Using the first positioning frame to mark the image features of the image, training the character positioning model to be trained;

[0033] When the global function loss value of the character positioning model to be trained is less than a second preset threshold, a preliminarily trained character positioning model is obtained, and a positive sample image in the positive sample image data set is input into the feature extraction model to obtain image features of the positive sample image;

[0034] Inputting the image features of the positive sample image into the preliminarily trained character positioning model to obtain a positioning frame of the character;

[0035] Performing a positioning frame annotation on the positive sample image based on the positioning frame of the character to obtain a second positioning frame annotation image;

[0036] Inputting the second positioning frame annotated image into the feature extraction model to obtain image features of the second positioning frame annotated image;

[0037] The image features of the image annotated by the second positioning frame are used to train the initially trained character positioning model so that the global function loss value of the initially trained character positioning model is less than a third preset threshold, thereby obtaining a trained character positioning model; wherein the third preset threshold is less than the second preset threshold.

[0038] Optionally, the adopting the second positioning frame to mark the image features of the image to train the preliminarily trained character positioning model includes:

[0039] Inputting the image features of the second positioning frame annotated image into the character positioning model to be trained to obtain the predicted number of the characters;

[0040] Obtaining the actual number of the characters and the initial global function loss value of the preliminarily trained character positioning model;

[0041] Determining a confidence probability based on the predicted number and the actual number of the characters; the confidence probability is used to measure the accuracy of the prediction of the feature extraction model;

[0042] The initial global function loss value is multiplied by the confidence probability to obtain the global function loss value of the preliminarily trained character positioning model.

[0043] Optionally, before reconstructing the normalized feature vector by the autoencoder, the method further includes:

[0044] Inputting the positive sample images in the positive sample image data set into the feature extraction model to obtain image features of the positive sample images;

[0045] Inputting the image features of the positive sample image into the character positioning model to obtain the image features of the positioning frame;

[0046] Perform global pooling on the image features of the positioning frame to obtain a feature vector of the positioning frame;

[0047] Normalizing the eigenvector to obtain a normalized eigenvector;

[0048] The normalized feature vector is used to train the autoencoder to be trained, thereby obtaining a trained autoencoder.

[0049] The embodiment of the present invention discloses a character detection device, the device comprising:

[0050] An image acquisition module, used for acquiring an image containing characters to be detected;

[0051] A first feature acquisition module is used to input the image into a feature extraction model to obtain image features of the image; wherein the feature extraction module is trained based on a standard image data set and a positive sample image data set containing characters without printing anomalies;

[0052] A second feature extraction module is used to input the image features of the image into a character positioning model to obtain a positioning frame of the character and the image features of the positioning frame; wherein the character positioning model is trained based on a positive sample image dataset annotated with the positioning frame and the positive sample image dataset;

[0053] A feature reconstruction module, used to reconstruct the image features of the positioning frame to obtain reconstructed image features of the positioning frame;

[0054] The abnormal character determination module is used to obtain a reconstruction error value between the image feature of the reconstructed positioning frame and the image feature of the positioning frame, and the reconstruction error value is used to determine whether the character is an abnormal character.

[0055] Optionally, the feature extraction model includes a MobileNet network and a feature pyramid network, and the first feature acquisition module includes:

[0056] A first feature acquisition submodule, used to input the image into the MobileNet network to obtain multi-layer image features of the image;

[0057] A feature fusion unit is used to perform feature fusion on the multi-layer image features through the feature pyramid network to obtain the image features of the image.

[0058] Optionally, the character positioning model includes a region generation network, a classification network and a regression prediction network, and the second feature acquisition module includes:

[0059] A candidate box acquisition submodule, used for inputting the image features of the image into the region generation network to obtain a candidate box;

[0060] A candidate box determination submodule, used to determine the candidate box containing the character through the classification network;

[0061] The positioning frame acquisition submodule is used to perform border regression processing on the candidate frame containing the character through the regression prediction network to obtain the positioning frame of the character and extract image features of the positioning frame.

[0062] Optionally, the positioning frame acquisition submodule includes:

[0063] A probability prediction unit, used to predict the probability value of the pixel point in the candidate frame through the regression prediction network; wherein the probability value represents the probability that the pixel point is the character;

[0064] The positioning frame determining unit is used to determine the positioning frame of the character based on the probability values ​​of the pixels in the candidate frame.

[0065] Optionally, the feature reconstruction module includes:

[0066] A feature pooling processing submodule, used to perform global pooling processing on the image features of the positioning frame to obtain a feature vector of the positioning frame;

[0067] A feature normalization processing submodule, used to normalize the feature vector of the positioning frame to obtain a normalized feature vector;

[0068] A feature reconstruction submodule, used to reconstruct the normalized feature vector through an autoencoder to obtain a reconstructed feature vector;

[0069] The abnormal character determination module comprises:

[0070] The abnormal character determination submodule is used to calculate the reconstruction error value between the reconstructed feature vector and the normalized feature vector, and the reconstruction error value is used to determine that the character is an abnormal character when it is greater than a first preset threshold.

[0071] Optionally, the standard image dataset is an ImageNet image dataset, and the device further comprises:

[0072] An extraction model acquisition module is used to obtain the feature extraction model pre-trained on the ImageNet image dataset;

[0073] The extraction model adjustment module is used to adjust the pre-trained feature extraction model based on the positive sample image data set to obtain a trained feature extraction model.

[0074] Optionally, the device further comprises:

[0075] A second feature acquisition module is used to input a first positioning frame annotated image in the positioning frame annotated positive sample image data set into the feature extraction model to obtain image features of the first positioning frame annotated image;

[0076] A positioning model training module, used for training a character positioning model to be trained by using the image features of the image annotated by the first positioning frame;

[0077] A third feature acquisition module is used to obtain a preliminarily trained character positioning model when the global function loss value of the character positioning model to be trained is less than a second preset threshold value, input the positive sample image in the positive sample image data set into the feature extraction model, and obtain the image feature of the positive sample image;

[0078] The positioning frame acquisition module is further used to input the image features of the positive sample image into the preliminarily trained character positioning model to obtain the positioning frame of the character;

[0079] An image annotation module, configured to annotate the positive sample image with a positioning frame based on the positioning frame of the character to obtain a second positioning frame annotated image;

[0080] The second feature acquisition module is further used to input the second positioning frame annotated image into the feature extraction model to obtain image features of the second positioning frame annotated image;

[0081] The positioning model training module is also used to use the image features of the positioning box annotated image to train the preliminarily trained character positioning model so that the global function loss value of the preliminarily trained character positioning model is less than a third preset threshold, thereby obtaining a trained character positioning model; wherein the third preset threshold is less than the second preset threshold.

[0082] Optionally, the positioning model training submodule includes:

[0083] A character prediction number acquisition unit, used for inputting the image features of the second positioning frame annotated image into the preliminarily trained character positioning model to obtain the predicted number of the characters;

[0084] A character actual number acquisition unit, used to acquire the actual number of the characters and the initial global function loss value of the preliminarily trained character positioning model;

[0085] A confidence probability determination unit, used to determine the confidence probability according to the predicted number and the actual number of the characters; the confidence probability is used to measure the accuracy of the prediction of the feature extraction model;

[0086] A global function loss value acquisition unit is used to multiply the initial global function loss value by the confidence probability to obtain the global function loss value of the preliminarily trained character positioning model.

[0087] Optionally, the device further comprises:

[0088] The second feature acquisition submodule is further used to input the positive sample image in the positive sample image data set into the feature extraction model to obtain the image features of the positive sample image;

[0089] A fourth feature acquisition submodule, used for inputting the image features of the positive sample image into the feature extraction model to obtain the image features of the positioning frame;

[0090] The feature pooling processing submodule is also used to perform global pooling processing on the image features of the positioning frame to obtain a feature vector of the positioning frame;

[0091] The feature normalization processing submodule is also used to normalize the feature vector to obtain a normalized feature vector;

[0092] The autoencoder training submodule is used to train the autoencoder to be trained by using the normalized feature vector to obtain a trained autoencoder.

[0093] The embodiment of the present invention further discloses an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0094] The memory is used to store computer programs;

[0095] The processor is used to implement the method described in the embodiment of the present invention when executing the program stored in the memory.

[0096] The embodiment of the present invention further discloses one or more computer-readable media on which instructions are stored. When executed by one or more processors, the processors are enabled to execute the method described in the embodiment of the present invention.

[0097] The embodiment of the present invention includes the following advantages: by using the embodiment of the present invention, the positioning frame of a single character is first determined, and the positioning frame of a single character in a vertical, horizontal, inclined and arc-shaped text can be determined, and then the single character in the positioning frame is detected to be an abnormal character, thereby realizing the detection of characters in vertical, horizontal, inclined and arc-shaped texts, and being applicable to various complex scenarios of text detection. The problem of being unable to accurately detect printing abnormalities or erroneous characters in scenarios such as arc-shaped text or inclined text is avoided. BRIEF DESCRIPTION OF THE DRAWINGS

[0098] Figure 1 is a flowchart of a character detection method provided in an embodiment of the present invention;

[0099] Figure 2 is a flowchart of a method for extracting image features provided in an embodiment of the present invention;

[0100] Figure 3 is a structural schematic diagram of an image feature fusion provided in an embodiment of the present invention;

[0101] Figure 4 is a flowchart of a method for determining a positioning frame provided in an embodiment of the present invention;

[0102] Figure 5 is a flowchart of a method for determining abnormal characters provided in an embodiment of the present invention;

[0103] Figure 6 Schematic diagram of a model framework of a character detection method provided in an embodiment of the present invention;

[0104] Figure 7 is a structural block diagram of a character detection device provided in an embodiment of the present invention;

[0105] Figure 8is a block diagram of an electronic device provided in an embodiment of the present invention;

[0106] Fig. 9 is a schematic diagram of a computer-readable medium provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0107] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0108] For the detection of characters in printed text, there are the following requirements:

[0109] 1) Capable of detecting the position of characters in various arrangements: In general, when detecting text, the layout of the text is not fixed. There are vertically arranged texts, tilted text lines, curved texts, and sometimes even tilted fonts.

[0110] 2) Text printing errors in various scenarios can be detected: Text printing comes in a variety of styles, and the printing defects that appear in different text sizes, font styles, typesetting methods, and backgrounds are different. The model can detect printing errors in various scenarios.

[0111] 3) Reduce the need for sample annotation for text position detection: Most position detection tasks require a large amount of labeled data for training, which is very costly. The appearance of printed materials changes frequently, and if the data needs to be re-annotated every time the appearance of the printed materials changes, it is difficult for general companies to afford such a large investment.

[0112] Therefore, there are the following problems in testing the quality of printed text: 1) The style of text varies greatly, and the layout, font, text size and background often change. 2) The workload of data annotation is large, and each character needs to be labeled with a position box, which is very labor-intensive. 3) There are too many styles of character printing defects, and it is difficult to collect sample images. 4) When Chinese is detected and the environment changes, it is difficult to obtain the character template.

[0113] Based on this, an embodiment of the present invention discloses a character detection method, device, electronic device and readable medium to solve the above-mentioned problems.

[0114] Reference Figure 1 , shows a flowchart of a character detection method provided in an embodiment of the present invention, which may specifically include the following steps:

[0115] Step 101: Acquire an image containing characters to be detected.

[0116] The image is an image including printed characters; characters refer to single Chinese characters, numbers, letters, symbols, etc. in the text.

[0117] Specifically, an image containing characters to be detected is acquired.

[0118] Step 102: Input the image into a feature extraction model to obtain image features of the image; wherein the feature extraction module is trained based on a standard image dataset and a positive sample image dataset containing characters without printing anomalies.

[0119] Among them, the feature extraction model is a model trained by a standard image dataset and a positive sample image dataset containing characters without printing anomalies; the standard image dataset contains images with various random backgrounds, and training the feature extraction model with images with various random backgrounds allows the feature training model to fully learn the features of images with different backgrounds, thereby accurately extracting image features of different images.

[0120] Specifically, after acquiring the image to be detected, the image to be detected is input into a trained feature extraction model, and the feature extraction model extracts image features of the image and outputs them.

[0121] Step 103: input the image features of the image into a character positioning model to obtain a positioning frame of the character and the image features of the positioning frame; wherein the character positioning model is trained based on a positive sample image dataset annotated with the positioning frame and the positive sample image dataset.

[0122] The positioning box is the minimum enclosing rectangle of the character, which is predicted by the character positioning model.

[0123] Specifically, after the image features of the image are extracted by the feature extraction model, the image features of the image are input into the character positioning model, and the character positioning model can determine the positioning frame of the character in the image, and obtain the x, y coordinates, width, length and angle of the center point of the positioning frame, so as to locate the accurate position of the character, and then detect the characters in the text in vertical, horizontal, inclined and arc. After determining the positioning frame of the character, the image features in the positioning frame are intercepted and output.

[0124] In addition, the character localization model is trained based on the positive sample image dataset annotated with the positioning box and the positive sample image dataset. For example, the character localization model can be first trained with the positive sample image dataset annotated with the positioning box. After the character localization model has a certain ability, the character localization model is used to annotate the positive sample images in the positive sample image dataset with character boxes, and then the character localization model is trained again with the positive sample images annotated with character boxes, which can reduce the cost of character box annotation.

[0125] Step 104: reconstruct the image features of the positioning frame to obtain reconstructed image features of the positioning frame.

[0126] Specifically, after the character positioning model outputs the image features of the positioning frame, the image features of the positioning frame are reconstructed to obtain the reconstructed image features of the positioning frame.

[0127] Step 105: Obtain a reconstruction error value between the image feature of the reconstructed positioning frame and the image feature of the positioning frame, wherein the reconstruction error value is used to determine whether the character is an abnormal character.

[0128] Among them, abnormal characters are characters with printing errors, printing defects, etc.

[0129] Specifically, after obtaining the image features of the reconstructed positioning frame, the difference between the image features of the reconstructed positioning frame and the image features of the positioning frame is obtained as a reconstruction error value, and then, whether the character is an abnormal character is determined according to the reconstruction error value.

[0130] In the embodiment of the present invention, the positioning frame of a single character is first determined, and the positioning frame of a single character in a vertical, horizontal, inclined, or arc-shaped text can be determined, and then the single character in the positioning frame is detected to be an abnormal character, thereby realizing the detection of characters in vertical, horizontal, inclined, or arc-shaped texts, and being applicable to various complex scenarios of text detection. The problem of being unable to accurately detect abnormal or erroneous characters in the printing of arc-shaped or inclined texts is avoided.

[0131] Reference Figure 2 , shows a flowchart of the steps of an image feature extraction method provided in an embodiment of the present invention, the embodiment of the present invention Figure 1 Based on the embodiment shown, a method for extracting image features through a feature extraction model and a method for training the feature extraction model are mainly described. The feature extraction model includes a MobileNet network and a feature pyramid network, such as Figure 2 As shown, the method of this embodiment may include:

[0132] Step 201: Input the image into the MobileNet network to obtain multi-layer image features of the image.

[0133] Among them, the feature extraction model uses a lightweight MobileNet as the backbone network (Backbone), and integrates the feature pyramid network (FPN, feature pyramid networks).

[0134] Specifically, the image is input into the MobileNet network, and the MobileNet network extracts image features of different depths from the image.

[0135] Step 202: performing feature fusion on the multi-layer image features through the feature pyramid network to obtain image features of the image.

[0136] Specifically, after the MobileNet network extracts image features of different depths from the image, the feature pyramid network fuses the extracted image features of different depths in a certain way. Figure 3 , shows a structural schematic diagram of an image feature fusion provided in an embodiment of the present invention. In the figure, the left one is called "bottom-up" and the right one is called "top-down". The bottom-up process is the ordinary forward propagation process of the neural network. The image features are calculated by the convolution kernel and usually become smaller and smaller. The top-down process is to upsample the more abstract and semantically stronger high-level image features, and then connect the image features to the previous layer of image features laterally. Therefore, the high-level image features are enhanced, and the image features used for prediction are a fusion of image features of different resolutions and different semantic strengths, which can detect large and small targets at the same time.

[0137] In the embodiment of the present invention, a lightweight Mobilenet is used as the backbone network, which reduces the memory usage of the feature extraction model and improves the inference time of the model, so that the model can be deployed on a device with only a CPU. This can reduce the cost of equipment while ensuring the accuracy and speed of inference.

[0138] With the help of the MobileNet network, image features of different depths are extracted from the image, and the shallow image features are fused with the deep image features, so that the subsequent detection steps can detect characters of different sizes with good detection effect.

[0139] In one embodiment of the present invention, the method also includes training a feature extraction model, specifically including: obtaining the feature extraction model pre-trained on the ImageNet image dataset; based on the positive sample image dataset, adjusting the pre-trained feature extraction model to obtain a trained feature extraction model.

[0140] Among them, the standard image dataset is the ImageNet image dataset.

[0141] Specifically, a MobileNet network pre-trained on the ImageNet image dataset is obtained, and then the pre-trained MobileNet network is fine-tuned using a small number of positive sample images in the positive sample image dataset, and then a trained feature extraction model is obtained.

[0142] In the embodiment of the present invention, the ImageNet image dataset contains images of various backgrounds, which allows the MobileNet network to fully learn the features of different background images, thereby accurately extracting image features of different pictures. The MobileNet network uses deep classifiable convolution and point-by-point convolution to reduce the amount of calculation.

[0143] Reference Figure 4 , shows a flowchart of the steps of a method for determining a positioning frame provided in an embodiment of the present invention, in which Figure 1 and / or Figure 2 Based on the embodiment shown, a method for determining a character positioning frame by a character positioning model and a method for training the character positioning model are mainly described. The character positioning model includes a region generation network, a classification network and a regression prediction network, such as Figure 4 As shown, the method of this embodiment may include:

[0144] Step 401: Input the image features of the image into the region generation network to obtain a candidate box.

[0145] Among them, the character localization model includes a region proposal network (RPN), a classification network (RPN class) and a regression prediction network (RPN bbox).

[0146] Specifically, after the image features of the image are input into the region generation network, the region generation network will intercept the image features through a sliding window to obtain several candidate boxes.

[0147] Step 402: Determine the candidate box containing the character through the classification network.

[0148] Specifically, after the region generation network intercepts the image features through a sliding window to obtain several candidate boxes, some candidate boxes contain characters (targets), while other candidate boxes only contain the background but not characters. Therefore, it is necessary to classify the candidate boxes through a classification network to determine the candidate boxes that contain characters and the candidate boxes that do not contain characters.

[0149] Step 403: Perform border regression processing on the candidate box containing the character through the regression prediction network to obtain the positioning box of the character, and extract image features of the positioning box.

[0150] In one embodiment of the present invention, step 403 includes: predicting the probability value of the pixel point in the candidate frame through the regression prediction network; wherein the probability value represents the probability that the pixel point is the character; and determining the positioning frame of the character based on the probability value of the pixel point in the candidate frame.

[0151] Specifically, after the candidate box containing the character is determined through the classification network, the candidate box containing the character is subjected to border regression processing, and the probability values ​​of the pixels in the candidate box are predicted through the regression prediction network. Based on the probability values ​​of the pixels in the candidate box, the positioning box where the character is located can be predicted from the candidate box containing the character, thereby accurately locating the position of the character; after predicting the positioning box where the character is located, the image features corresponding to the positioning box are obtained.

[0152] In addition, based on the probability values ​​of the pixels in the candidate box, the predicted number of characters in the image can also be predicted.

[0153] In the embodiment of the present invention, after the region generation network is used to intercept the image features through the sliding window to obtain the candidate box, the frame regression is performed on the candidate box of each character to obtain a more accurate positioning box, and the x, y coordinates, width, length and angle of the center point of the positioning box are determined, so that the positioning box of a single character in the vertical, horizontal, inclined and arc-shaped text can be determined, and then the single character in the positioning box is detected to be an abnormal character, thereby realizing the detection of characters in the vertical, horizontal, inclined and arc-shaped text, which is applicable to various complex scenarios of text detection. Avoid the problem of being unable to accurately detect abnormal or wrong characters in the scene of arc-shaped text or inclined text.

[0154] In one embodiment of the present invention, before inputting the image features of the image into the character positioning model, it also includes: inputting the first positioning frame annotated image in the positive sample image data set annotated with the positioning frame into the feature extraction model to obtain the image features of the first positioning frame annotated image; using the image features of the first positioning frame annotated image to train the character positioning model to be trained; when the global function loss value of the character positioning model to be trained is less than the second preset threshold, a preliminarily trained character positioning model is obtained, and the positive sample image in the positive sample image data set is input into the feature extraction model to obtain the image features of the positive sample image; the image features of the positive sample image are input into the preliminarily trained character positioning model to obtain the positioning frame of the character; based on the positioning frame of the character, the positive sample image is annotated with the positioning frame to obtain the second positioning frame annotated image; the second positioning frame annotated image is input into the feature extraction model to obtain the image features of the second positioning frame annotated image; using the image features of the second positioning frame annotated image to train the preliminarily trained character positioning model so that the global function loss value of the preliminarily trained character positioning model is less than the third preset threshold, and a trained character positioning model is obtained; wherein the third preset threshold is less than the second preset threshold.

[0155] The third preset threshold is smaller than the second preset threshold.

[0156] The first positioning frame annotated image is an image in which characters are annotated with positioning frames manually or in other ways, which is labor-intensive and costly; the second positioning frame annotated image is an image in which characters are annotated with positioning frames by a preliminarily trained character positioning model.

[0157] Each character in a positive sample image needs to be labeled with a position box, which is a lot of work. As a result, the cost of character box labeling is too high, and it is not suitable to use a large number of images labeled with character boxes to train the character positioning model. Based on this, the embodiment of the present invention provides a weak supervision method to solve the problem of large workload and high cost of character box labeling.

[0158] Specifically, the first positioning box annotated image in the positive sample image data set with positioning box annotation is first used to train the character positioning model to be trained. Specifically, the first positioning box annotated image in the positive sample image data set with positioning box annotation is input into the feature extraction model to obtain the image features of the first positioning box annotated image. The image features of the first positioning box annotated image are used to train the character positioning model to be trained.

[0159] When the global function loss value of the character positioning model to be trained is less than the second preset threshold, it indicates that the character positioning model to be trained has a certain prediction ability, and a preliminarily trained character positioning model is obtained. Therefore, the positive sample image can be annotated with a character frame through the preliminarily trained character positioning model. Specifically, the positive sample image in the positive sample image data set is input into the feature extraction model to obtain the image features of the positive sample image, and the image features of the positive sample image are input into the preliminarily trained character positioning model to obtain the character positioning frame. The positive sample image is annotated with a positioning frame based on the character positioning frame, thereby obtaining a second positioning frame annotated image.

[0160] After obtaining the second positioning frame annotated images, the preliminarily trained character positioning model is trained using these second positioning frame annotated images; specifically, the second positioning frame annotated images are input into the feature extraction model to obtain image features of the second positioning frame annotated images, and the preliminarily trained character positioning model is trained using the image features of the second positioning frame annotated images until the global function loss value of the preliminarily trained character positioning model is less than a third preset threshold, indicating that the character positioning model training is completed.

[0161] The character localization model is trained. The initial global function loss of the character localization model includes classification loss and regression loss. The classification network is used to determine whether there is a character (target) in the box. The classification loss is calculated using softmax. The candidate box is roughly regressed through the regression prediction network to obtain the positioning box. The regression loss is calculated using smooth L1. The initial global function loss is calculated as follows:

[0162]

[0163] Among them, L({pi},{ti}) is the initial global function loss value;

[0164] pi* is 0 or 1, 0 means there is no character in the candidate box, 1 means there is no character in the selected box; pi is 0 or 1, 0 means there is no character in the candidate box judged by the classification network, 1 means there is a character in the selected box judged by the classification network; Lcls(pi,pi*) is the classification loss, Ncls is the number of candidate boxes;

[0165] ti* is the position of the labeled positioning box; ti is the position of the positioning box determined by the regression prediction network; Lreg(ti,ti*) is the regression loss, and Ncls is the number of positioning boxes.

[0166] In one embodiment of the present invention, the image features of the image annotated by the second positioning frame are used to train the preliminarily trained character positioning model, including: inputting the image features of the image annotated by the second positioning frame into the preliminarily trained character positioning model to obtain the predicted number of the characters; obtaining the actual number of the characters and the initial global function loss value of the preliminarily trained character positioning model; determining a confidence probability based on the predicted number and the actual number of the characters; the confidence probability is used to measure the accuracy of the feature extraction model's prediction; and multiplying the initial global function loss value by the confidence probability to obtain the global function loss value of the preliminarily trained character positioning model.

[0167] Specifically, the accuracy of the positioning box predicted by the initially trained character positioning model is not guaranteed at this time. When taking into account the initial global function loss value, we need to multiply the corresponding initial global function loss value by a confidence probability. The character positioning box can be predicted through the regression network of the feature extraction model, and the predicted number of characters can also be obtained. Then, the actual number of characters in the second positioning box annotation image can be obtained. We can use the difference between the predicted and actual number of characters to measure the accuracy of the prediction. Confidence probability = 1-difference in the number of characters / actual number of characters. The difference in the number of characters is the absolute value of the predicted number of characters minus the actual number of characters. The initial global function loss value is multiplied by the confidence probability to obtain the global function loss value of the initially trained character positioning model.

[0168] It should be noted that when the character positioning model to be trained is trained by annotating the image with the first positioning frame, because it is a preliminary training, it does not need to be too precise. Therefore, the initial global function loss value can be directly used as the global function loss value of the character positioning model to be trained. Of course, the initial global function loss value can also be multiplied by the confidence probability to obtain the global function loss value of the character positioning model to be trained. This is not limited in the embodiments of the present invention.

[0169] In an embodiment of the present invention, a character positioning model to be trained is first trained using a first positioning frame annotated image in a positive sample image data set annotated with positioning frames. When the character positioning model to be trained has a certain prediction capability, a preliminarily trained character positioning model is obtained. The positive sample image is annotated with positioning frames using the preliminarily trained character positioning model to obtain a second positioning frame annotated image. The second positioning frame annotated image is returned to train the preliminarily trained character positioning model. This eliminates the need to use labor-intensive and costly manual annotation methods to annotate the characters of all positive sample images with positioning frames, thereby greatly saving the cost of positioning frame annotation.

[0170] Reference Figure 5 , shows a flowchart of a method for determining abnormal characters provided in an embodiment of the present invention, wherein the embodiment of the present invention Figure 1 Based on the embodiment shown, a method for reconstructing image features through an autoencoder and determining abnormal characters, as well as a method for training the autoencoder are mainly described. Figure 5 As shown, the method of this embodiment may include:

[0171] Step 501: Perform global pooling processing on the image features of the positioning frame to obtain a feature vector of the positioning frame.

[0172] Specifically, the image features of the positioning frame are input into the global pooling layer, the image features are mean pooled to form feature points, and these feature points are combined into a feature vector with a length of 2496.

[0173] Step 502: normalize the feature vector of the positioning frame to obtain a normalized feature vector.

[0174] Specifically, the purpose of normalizing the eigenvectors is to put the eigenvectors at the same order of magnitude, which will be needed for comprehensive comparative evaluation later.

[0175] Step 503: reconstruct the normalized feature vector through an autoencoder to obtain a reconstructed feature vector.

[0176] Among them, the autoencoder is an unsupervised learning model. Essentially, it uses a neural network to generate a low-dimensional representation of a high-dimensional input. The autoencoder is similar to principal component analysis PCA (Principal Component Analysis), but the autoencoder overcomes the linear limitations of PCA when using nonlinear activation functions. The autoencoder consists of two main parts, the encoder and the decoder. The encoder is used to find the compressed representation of the given data, and the decoder is used to reconstruct the original input. During training, the decoder forces the autoencoder to select the most informative features, which are ultimately saved in the compressed representation. The final compressed representation is in the middle encoder layer.

[0177] Specifically, the normalized feature vector is reconstructed by an autoencoder, the normalized feature vector is compressed by an encoder, and then the compressed normalized feature vector is reconstructed by a decoder to obtain a reconstructed feature vector.

[0178] Step 504: Calculate a reconstruction error value between the reconstructed feature vector and the normalized feature vector, wherein the reconstruction error value is used to determine the character as an abnormal character when the reconstruction error value is greater than a first preset threshold.

[0179] Specifically, the difference between the reconstructed feature vector and the normalized feature vector is calculated as a reconstruction error value, and the character can be evaluated based on the reconstruction error value. When the reconstruction error value is greater than a first preset threshold, it indicates that the character has defects and the character is determined to be an abnormal character.

[0180] In the embodiment of the present invention, the feature vector of each character is encoded and decoded separately by an autoencoder, and then the L1 distance between the original feature vector of each character and the encoded and decoded feature vector is calculated to determine whether the character is an abnormal character.

[0181] In one embodiment of the present invention, before reconstructing the normalized feature vector through the autoencoder, it also includes: inputting the positive sample image in the positive sample image data set into the feature extraction model to obtain the image features of the positive sample image; inputting the image features of the positive sample image into the feature extraction model to obtain the image features of the positioning frame; performing global pooling processing on the image features of the positioning frame to obtain the feature vector of the positioning frame; normalizing the feature vector to obtain a normalized feature vector; using the normalized feature vector to train the autoencoder to be trained to obtain a trained autoencoder.

[0182] Specifically, the positive sample images in the positive sample image data set are input into the feature extraction model to obtain the image features of the positive sample images, the image features of the positive sample images are input into the feature extraction model to obtain the image features of the positioning frame, the image features of the positioning frame are globally pooled to obtain the feature vector of the positioning frame, the feature vector is normalized to obtain the normalized feature vector, the normalized feature vector is used to train the autoencoder, and the function loss (reconstruction error value) of the autoencoder is L1, and the specific calculation method is as follows:

[0183]

[0184] Among them, y (i) is the target value of the input autoencoder, that is, the normalized feature vector; is the estimated value of the autoencoder output, that is, the reconstructed feature vector.

[0185] In the embodiment of the present invention, the feature vector of a detected single character is reconstructed using an autoencoder, and positive samples are used during training to make the reconstruction error value L1 0. The reconstruction error is calculated during detection. If it is close to 0, it is a normal character. If it exceeds the first preset threshold, it is an abnormal character. Therefore, the entire training process does not need to use negative samples containing various abnormal characters.

[0186] Reference Figure 6 , shows a model framework schematic diagram of a character detection method provided in an embodiment of the present invention, in which the multi-layer image features (Block1, ..., Block5) of the image to be detected are extracted through the MobileNet network, the multi-layer image features are input into the feature pyramid network for feature fusion (Fusion), the image features of the image are obtained, and are input into the RPN, the sliding window intercepts the candidate box, the candidate box is input into the class (classification network), the candidate box containing the character is determined and input into the bbox (regression prediction network), the frame regression processing is performed on the selection box, the character positioning box and the image features of the positioning box are obtained, the image features of the positioning box are input into the Auto encoder, and the Auto encoder is used to determine whether the character in the positioning box is an abnormal character.

[0187] In the embodiment of the present invention, a method is proposed to use a lightweight MobileNet network as the backbone, and use a self-designed feature pyramid network to fuse features. The final extracted features fuse shallow features and deep features, which are suitable for detecting small targets such as characters, and the reasoning speed is extremely fast.

[0188] A character localization model is proposed. The minimum rotated rectangle of each character is regressed to obtain the character localization box. The x, y coordinates, width, length and angle of the center point of the localization box are determined. The model can detect characters in vertical, horizontal, inclined and curved texts and is suitable for various complex scenarios of text detection.

[0189] A method using an autoencoder is proposed to encode and decode the feature vector of each character separately, and then calculate the L1 distance between the original feature and the encoded and decoded feature of each character. Characters whose distance exceeds a first preset threshold are abnormal characters.

[0190] A weakly supervised learning method is proposed. The character localization model is used to annotate the positive sample image with a localization box to obtain a localization box annotated image. The localization box annotated image is then returned to train the character localization model, which can greatly save the cost of localization box annotation.

[0191] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0192] Reference Figure 7 , shows a structural block diagram of a character detection device provided in an embodiment of the present invention, which may specifically include the following modules:

[0193] An image acquisition module 701 is used to acquire an image containing characters to be detected;

[0194] A first feature acquisition module 702 is used to input the image into a feature extraction model to obtain image features of the image; wherein the feature extraction module is trained based on a standard image dataset and a positive sample image dataset containing characters without printing anomalies;

[0195] The second feature extraction module 703 is used to input the image features of the image into a character positioning model to obtain a positioning frame of the character and the image features of the positioning frame; the character positioning model is trained based on a positive sample image dataset annotated with the positioning frame and the positive sample image dataset;

[0196] A feature reconstruction module 704 is used to reconstruct the image features of the positioning frame to obtain reconstructed image features of the positioning frame;

[0197] The abnormal character determination module 705 is used to obtain a reconstruction error value between the image feature of the reconstructed positioning frame and the image feature of the positioning frame, and the reconstruction error value is used to determine whether the character is an abnormal character.

[0198] Optionally, the feature extraction model includes a MobileNet network and a feature pyramid network, and the first feature acquisition module 702 includes:

[0199] A first feature acquisition submodule, used to input the image into the MobileNet network to obtain multi-layer image features of the image;

[0200] A feature fusion unit is used to perform feature fusion on the multi-layer image features through the feature pyramid network to obtain the image features of the image.

[0201] Optionally, the character positioning model includes a region generation network, a classification network and a regression prediction network, and the second feature acquisition module 703 includes:

[0202] A candidate box acquisition submodule, used for inputting the image features of the image into the region generation network to obtain a candidate box;

[0203] A candidate box determination submodule, used to determine the candidate box containing the character through the classification network;

[0204] The positioning frame acquisition submodule is used to perform border regression processing on the candidate frame containing the character through the regression prediction network to obtain the positioning frame of the character and extract image features of the positioning frame.

[0205] Optionally, the positioning frame acquisition submodule includes:

[0206] A probability prediction unit, used to predict the probability value of the pixel point in the candidate frame through the regression prediction network; wherein the probability value represents the probability that the pixel point is the character;

[0207] The positioning frame determining unit is used to determine the positioning frame of the character based on the probability values ​​of the pixels in the candidate frame.

[0208] Optionally, the feature reconstruction module 704 includes:

[0209] A feature pooling processing submodule, used to perform global pooling processing on the image features of the positioning frame to obtain a feature vector of the positioning frame;

[0210] A feature normalization processing submodule, used to normalize the feature vector of the positioning frame to obtain a normalized feature vector;

[0211] A feature reconstruction submodule, used to reconstruct the normalized feature vector through an autoencoder to obtain a reconstructed feature vector;

[0212] The abnormal character determination module 705 includes:

[0213] The abnormal character determination submodule is used to calculate the reconstruction error value between the reconstructed feature vector and the normalized feature vector, and the reconstruction error value is used to determine that the character is an abnormal character when it is greater than a first preset threshold.

[0214] Optionally, the standard image dataset is an ImageNet image dataset, and the device further comprises:

[0215] An extraction model acquisition submodule is used to obtain the feature extraction model pre-trained on the ImageNet image dataset;

[0216] The extraction model adjustment submodule is used to adjust the pre-trained feature extraction model based on the positive sample image data set to obtain a trained feature extraction model.

[0217] Optionally, the device further comprises:

[0218] A second feature acquisition module is used to input a first positioning frame annotated image in the positioning frame annotated positive sample image data set into the feature extraction model to obtain image features of the first positioning frame annotated image;

[0219] A positioning model training module, used for training a character positioning model to be trained by using the image features of the image annotated by the first positioning frame;

[0220] A third feature acquisition module is used to obtain a preliminarily trained character positioning model when the global function loss value of the character positioning model to be trained is less than a second preset threshold value, input the positive sample image in the positive sample image data set into the feature extraction model, and obtain the image feature of the positive sample image;

[0221] The positioning frame acquisition module is further used to input the image features of the positive sample image into the preliminarily trained character positioning model to obtain the positioning frame of the character;

[0222] An image annotation module, configured to annotate the positive sample image with a positioning frame based on the positioning frame of the character to obtain the second positioning frame annotated image;

[0223] The second feature acquisition module is further used to input the second positioning frame annotated image into a feature extraction model to obtain image features of the second positioning frame annotated image;

[0224] The positioning model training module is also used to use the image features of the image annotated by the second positioning frame to train the preliminarily trained character positioning model so that the global function loss value of the preliminarily trained character positioning model is less than a third preset threshold, thereby obtaining a trained character positioning model; wherein the third preset threshold is less than the second preset threshold.

[0225] Optionally, the positioning model training submodule includes:

[0226] A character prediction number acquisition unit, used for inputting the image features of the second positioning frame annotated image into the feature extraction model to obtain the predicted number of the characters;

[0227] A character actual number acquisition unit, used to acquire the actual number of the characters and the initial global function loss value of the preliminarily trained character positioning model;

[0228] A confidence probability determination unit, used to determine the confidence probability according to the predicted number and the actual number of the characters; the confidence probability is used to measure the accuracy of the prediction of the feature extraction model;

[0229] A global function loss value acquisition unit is used to multiply the initial global function loss value by the confidence probability to obtain the global function loss value of the preliminarily trained character positioning model.

[0230] Optionally, the device further comprises:

[0231] The second feature acquisition submodule is further used to input the positive sample image in the positive sample image data set into the feature extraction model to obtain the image features of the positive sample image;

[0232] A fourth feature acquisition submodule, used for inputting the image features of the positive sample image into the feature extraction model to obtain the image features of the positioning frame;

[0233] The feature pooling processing submodule is also used to perform global pooling processing on the image features of the positioning frame to obtain a feature vector of the positioning frame;

[0234] The feature normalization processing submodule is also used to normalize the feature vector to obtain a normalized feature vector;

[0235] The autoencoder training submodule is used to train the autoencoder to be trained by using the normalized feature vector to obtain a trained autoencoder.

[0236] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0237] In addition, an embodiment of the present invention further provides an electronic device, such as Figure 8 As shown, it includes a processor 801, a communication interface 802, a memory 803 and a communication bus 804, wherein the processor 801, the communication interface 802, and the memory 803 communicate with each other through the communication bus 804.

[0238] Memory 803, used for storing computer programs;

[0239] The processor 801 is used to implement the character detection method described in the above embodiment when executing the program stored in the memory 803.

[0240] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0241] The communication interface is used for communication between the above terminal and other devices.

[0242] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0243] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0244] like Fig. 9As shown, in another embodiment provided by the present invention, a computer-readable storage medium 901 is also provided, in which instructions are stored. When the computer-readable storage medium is run on a computer, the computer executes the character detection method described in the above embodiment.

[0245] In another embodiment of the present invention, a computer program product including instructions is provided. When the computer program product is run on a computer, the computer executes the character detection method described in the above embodiment.

[0246] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0247] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0248] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0249] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A character detection method, characterized in that: include: Obtain an image containing characters to be detected; Inputting the image into a feature extraction model to obtain image features of the image; wherein the feature extraction model is trained based on a standard image dataset and a positive sample image dataset containing characters without printing anomalies; Inputting the image features of the image into a character positioning model to obtain a positioning frame of the character and the image features of the positioning frame; wherein the character positioning model is trained based on a positive sample image dataset annotated with the positioning frame and the positive sample image dataset; Reconstructing the image features of the positioning frame to obtain reconstructed image features of the positioning frame; Obtaining a reconstruction error value between the image feature of the reconstructed positioning frame and the image feature of the positioning frame, wherein the reconstruction error value is used to determine whether the character is an abnormal character; Before inputting the image features of the image into the character positioning model, the method further includes: Inputting a first positioning frame annotated image in the positioning frame annotated positive sample image data set into the feature extraction model to obtain image features of the first positioning frame annotated image; Using the first positioning frame to mark the image features of the image, training the character positioning model to be trained; When the global function loss value of the character positioning model to be trained is less than a second preset threshold, a preliminarily trained character positioning model is obtained, and a positive sample image in the positive sample image data set is input into the feature extraction model to obtain image features of the positive sample image; Inputting the image features of the positive sample image into the preliminarily trained character positioning model to obtain a positioning frame of the character; Performing a positioning frame annotation on the positive sample image based on the positioning frame of the character to obtain a second positioning frame annotation image; Inputting the second positioning frame annotated image into the feature extraction model to obtain image features of the second positioning frame annotated image; The image features of the image annotated by the second positioning frame are used to train the initially trained character positioning model so that the global function loss value of the initially trained character positioning model is less than a third preset threshold, thereby obtaining a trained character positioning model; wherein the third preset threshold is less than the second preset threshold.

2. The method according to claim 1, characterized in that The feature extraction model includes a MobileNet network and a feature pyramid network, and the image is input into the feature extraction model to obtain the image features of the image, including: Inputting the image into the MobileNet network to obtain multi-layer image features of the image; The multi-layer image features are fused through the feature pyramid network to obtain the image features of the image.

3. The method according to claim 1, characterized in that The character positioning model includes a region generation network, a classification network and a regression prediction network. The image features of the image are input into the character positioning model to obtain the positioning frame of the character and the image features of the positioning frame, including: Inputting the image features of the image into the region generation network to obtain a candidate box; Determine the candidate box containing the character through the classification network; The candidate box containing the character is subjected to frame regression processing through the regression prediction network to obtain a positioning box of the character, and image features of the positioning box are extracted.

4. The method according to claim 3, characterized in that The step of performing border regression processing on the candidate box containing the character through the regression prediction network to obtain a positioning box of the character includes: Predicting the probability value of the pixel point in the candidate frame by the regression prediction network; wherein the probability value represents the probability that the pixel point is the character; Based on the probability values ​​of the pixels in the candidate frame, a positioning frame of the character is determined.

5. The method according to claim 1, characterized in that The reconstructing the image features of the positioning frame to obtain the reconstructed image features of the positioning frame includes: Performing global pooling processing on the image features of the positioning frame to obtain a feature vector of the positioning frame; Normalizing the feature vector of the positioning frame to obtain a normalized feature vector; Reconstructing the normalized feature vector through an autoencoder to obtain a reconstructed feature vector; The obtaining of the reconstruction error value between the image feature of the reconstructed positioning frame and the image feature of the positioning frame, wherein the reconstruction error value is used to determine whether the character is an abnormal character, includes: A reconstruction error value between the reconstructed feature vector and the normalized feature vector is calculated, and the reconstruction error value is used to determine that the character is an abnormal character when it is greater than a first preset threshold.

6. The method according to claim 1, characterized in that The standard image dataset is an ImageNet image dataset, and before the image is input into the feature extraction model, the method further includes: Obtain a feature extraction model pre-trained on the ImageNet image dataset; Based on the positive sample image data set, the pre-trained feature extraction model is adjusted to obtain a trained feature extraction model.

7. The method according to claim 1, characterized in that The step of using the second positioning frame to mark the image features of the image and training the preliminarily trained character positioning model includes: Inputting the image features of the second positioning frame annotated image into the preliminarily trained character positioning model to obtain the predicted number of the characters; Obtaining the actual number of the characters and the initial global function loss value of the preliminarily trained character positioning model; Determining a confidence probability based on the predicted number and the actual number of the characters; the confidence probability is used to measure the accuracy of the prediction of the feature extraction model; The initial global function loss value is multiplied by the confidence probability to obtain the global function loss value of the preliminarily trained character positioning model.

8. The method according to claim 5, characterized in that Before reconstructing the normalized feature vector by the autoencoder, the method further includes: Inputting the positive sample images in the positive sample image data set into the feature extraction model to obtain image features of the positive sample images; Inputting the image features of the positive sample image into the character positioning model to obtain the image features of the positioning frame; Perform global pooling on the image features of the positioning frame to obtain a feature vector of the positioning frame; Normalizing the eigenvector to obtain a normalized eigenvector; The normalized feature vector is used to train the autoencoder to be trained, thereby obtaining a trained autoencoder.

9. A character detection device, characterized in that: The device comprises: An image acquisition module, used for acquiring an image containing characters to be detected; A first feature acquisition module, used for inputting the image into a feature extraction model to obtain image features of the image; wherein the feature extraction model is trained based on a standard image data set and a positive sample image data set containing characters without printing anomalies; A second feature extraction module is used to input the image features of the image into a character positioning model to obtain a positioning frame of the character and the image features of the positioning frame; wherein the character positioning model is trained based on a positive sample image dataset annotated with the positioning frame and the positive sample image dataset; A feature reconstruction module, used to reconstruct the image features of the positioning frame to obtain reconstructed image features of the positioning frame; an abnormal character determination module, used for obtaining a reconstruction error value between the image feature of the reconstructed positioning frame and the image feature of the positioning frame, and determining whether the character is an abnormal character based on the reconstruction error value; Before inputting the image features of the image into the character positioning model, the method further includes: Inputting a first positioning frame annotated image in the positioning frame annotated positive sample image data set into the feature extraction model to obtain image features of the first positioning frame annotated image; Using the first positioning frame to mark the image features of the image, training the character positioning model to be trained; When the global function loss value of the character positioning model to be trained is less than a second preset threshold, a preliminarily trained character positioning model is obtained, and a positive sample image in the positive sample image data set is input into the feature extraction model to obtain image features of the positive sample image; Inputting the image features of the positive sample image into the preliminarily trained character positioning model to obtain a positioning frame of the character; Performing a positioning frame annotation on the positive sample image based on the positioning frame of the character to obtain a second positioning frame annotation image; Inputting the second positioning frame annotated image into the feature extraction model to obtain image features of the second positioning frame annotated image; The image features of the image annotated by the second positioning frame are used to train the initially trained character positioning model so that the global function loss value of the initially trained character positioning model is less than a third preset threshold, thereby obtaining a trained character positioning model; wherein the third preset threshold is less than the second preset threshold.

10. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; The memory is used to store computer programs; The processor is used to implement the method according to any one of claims 1 to 8 when executing the program stored in the memory.

11. One or more computer-readable media having instructions stored thereon, which, when executed by one or more processors, cause the processors to perform the method of any one of claims 1-8.

Images