Text detection method and device, storage medium and electronic device

Abstract

The present disclosure relates to the technical field of image processing, and provides a text detection method and device, a storage medium and an electronic device. The text detection method comprises: acquiring a to-be-detected image containing to-be-detected text; inputting the to-be-detected image into a pre-trained text region detection model to obtain a target text region output by the text region detection model, wherein the text region detection model is determined from at least two student models, the text region detection model comprises a backbone network layer, a feature fusion layer and an output layer, the backbone network layer is used for feature extraction on the to-be-detected image to obtain a feature vector, the feature fusion network is used for fusion on the feature vector to obtain a fusion result, and the output layer is used for outputting the target text region according to the fusion result; and performing text recognition on the target text region in the to-be-detected image to obtain a text detection result.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of image processing, in particular, to a text detection method and device, a storage medium and an electronic device. BACKGROUND

[0002] Text detection refers to a process of extracting characters in a text image. In existing methods, the recognition of characters in pictures is generally performed by manually checking picture information. This way has poor accuracy and efficiency in recognizing characters in massive and multi-type picture data on the Internet.

[0003] In related technologies, optical character recognition (OCR) is performed on characters in an image, which is an important scenario in the field of deep learning. OCR refers to a process of analyzing and recognizing a text image of text data to obtain character and layout information. However, when using OCR to recognize characters in a text image, the characters in the text image cannot be accurately positioned, which can easily lead to a low accuracy of recognized characters. SUMMARY

[0004] The purpose of the present disclosure is to provide a text detection method, device, storage medium and electronic device to solve the above problems in related technologies.

[0005] To achieve the above purpose, the first aspect of the disclosed embodiment provides a text detection method, which comprises:

[0006] obtaining a to-be-detected image containing to-be-detected text;

[0007] inputting the to-be-detected image into a pre-trained text region detection model to obtain a target text region output by the text region detection model, wherein the text region detection model is determined from at least two student models, the text region detection model comprises a backbone network layer, a feature fusion layer and an output layer, the backbone network layer is used to extract features of the to-be-detected image to obtain a feature vector, the feature fusion network is used to fuse the feature vector to obtain a fusion result, and the output layer is used to output the target text region according to the fusion result;

[0008] performing text recognition on the target text region in the to-be-detected image to obtain a text detection result.

[0009] Optionally, the training method of the text region detection model comprises:

[0010] training at least two teacher models using an online knowledge distillation strategy of mutual learning, and determining a target teacher model therefrom;

[0011] The at least two student models are trained by using an offline knowledge distillation strategy based on the target teacher model and an online knowledge distillation strategy of mutual learning, and a target student model is determined from the at least two student models as the text region detection model.

[0012] Optionally, the online knowledge distillation strategy of mutual learning is used to train the at least two teacher models, including:

[0013] The at least two teacher models are obtained, and a first training sample is input into each of the teacher models to obtain a first result output by each of the teacher models;

[0014] A first loss value is calculated by using a first loss function according to the first result output by each of the teacher models;

[0015] A second loss value is calculated by using a second loss function according to the first result output by each of the teacher models and a first sample label;

[0016] Parameters of the at least two teacher models are adjusted according to a teacher model loss value, and the teacher model loss value includes the first loss value and the second loss value.

[0017] Optionally, a convolution kernel size of a feature fusion layer of the teacher model is one of 3*3, 5*5, 7*7 and 9*9.

[0018] Optionally, the at least two student models are trained by using the offline knowledge distillation strategy based on the target teacher model and the online knowledge distillation strategy of mutual learning, including:

[0019] The at least two student models are obtained, and a second training sample is input into each of the student models to obtain a second result output by each of the student models;

[0020] A third loss value is calculated by using the first loss function according to the second result output by each of the student models,

[0021] A fourth loss value is calculated by using the second loss function according to the second result output by each of the student models and a second sample label;

[0022] The second training sample is input into the target teacher model to obtain a third result output by the target teacher model;

[0023] A fifth loss value is calculated by using a third loss function according to the second result output by each of the student models and the third result output by the target teacher model;

[0024] The parameters of the at least two student models are adjusted according to student model loss values, the student model loss values including the third loss value, the fourth loss value, and the fifth loss value.

[0025] According to a second aspect of embodiments of the present disclosure, a text detection apparatus is provided, the text detection apparatus comprising:

[0026] An acquisition module is configured to acquire a to-be-detected image containing text to be detected.

[0027] A text region detection module is configured to input the to-be-detected image into a pre-trained text region detection model to obtain a target text region output by the text region detection model, wherein the text region detection model is determined from at least two student models, and the text region detection model comprises a backbone network layer, a feature fusion layer, and an output layer.

[0028] A text recognition module is configured to perform text recognition on the target text region in the to-be-detected image to obtain a text detection result.

[0029] Optionally, the text detection apparatus further comprises:

[0030] A teacher model training module is configured to train at least two teacher models using an online knowledge distillation strategy of mutual learning, and determine a target teacher model therefrom.

[0031] A student model training module is configured to train at least two student models using an offline knowledge distillation strategy based on the target teacher model and the online knowledge distillation strategy of mutual learning, and determine a target student model therefrom as a text region detection model.

[0032] Optionally, the teacher model training module is specifically configured to:

[0033] The teacher model training module is specifically configured to:

[0034] The teacher model training module is specifically configured to:

[0035] The teacher model training module is specifically configured to:

[0036] The parameters of the at least two teacher models are adjusted according to teacher model loss values, the teacher model loss values including the first loss value and the second loss value.

[0037] According to a third aspect of the embodiments of the present disclosure, a non-transitory computer-readable storage medium is provided, which has stored thereon a computer program, and the computer program is executed by a processor to implement the steps of the text detection method provided in the first aspect of the present disclosure.

[0038] According to a fourth aspect of the embodiments of the present disclosure, an electronic device is provided, which comprises:

[0039] a memory having stored thereon a computer program;

[0040] a processor configured to execute the computer program in the memory to implement the steps of the text detection method provided in the first aspect of the present disclosure.

[0041] According to the above technical solution, first, a to-be-detected image containing to-be-detected text is obtained; then, the to-be-detected image is input into a pre-trained text region detection model to obtain a target text region output by the text region detection model, wherein the text region detection model is determined from at least two student models, and the text region detection model comprises a backbone network layer, a feature fusion layer, and an output layer. The backbone network layer is configured to perform feature extraction on the to-be-detected image to obtain a feature vector, the feature fusion network is configured to fuse the feature vector to obtain a fusion result, and the output layer is configured to output the target text region according to the fusion result. Finally, text recognition is performed on the target text region in the to-be-detected image to obtain a text detection result. The accurate target text region in the to-be-detected image is recognized by the text region detection model determined from the at least two student models, so as to perform text recognition thereon to obtain an accurate text detection result, thereby improving the accuracy of text detection.

[0042] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF DRAWINGS

[0043] The accompanying drawings are included to provide a further understanding of the present disclosure and constitute a part of the specification, and are used together with the following detailed description to explain the present disclosure, but do not constitute a limitation of the present disclosure. In the drawings:

[0044] Figure 1 is a flowchart of a text detection method according to an exemplary embodiment;

[0045] Figure 2 is a schematic diagram of a text region detection model according to an exemplary embodiment;

[0046] Figure 3This is a flowchart illustrating a training method for a text region detection model according to an exemplary embodiment;

[0047] Figure 4 This is an implementation illustrated according to an exemplary embodiment. Figure 3 Flowchart of step S201;

[0048] Figure 5 This is an implementation illustrated according to an exemplary embodiment. Figure 3 Flowchart of step S202;

[0049] Figure 6 This is a block diagram illustrating a text detection device according to an exemplary embodiment;

[0050] Figure 7 This is a block diagram illustrating an electronic device according to an exemplary embodiment.

[0051] Explanation of reference numerals in the attached figures

[0052] 110-Backbone network layer; 120-Feature fusion layer; 130-Output layer; 400-Text detection device; 401-Acquisition module; 402-Text region detection module; 403-Text recognition module; 700-Electronic device; 701-Processor; 702-Memory; 703-Multimedia component; 704-I / O interface; 705-Communication component. Detailed Implementation

[0053] The specific embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit this disclosure.

[0054] In the following description, the words "first" and "second" are used only to distinguish the purpose of the description and should not be interpreted as indicating or implying relative importance or order.

[0055] This disclosure provides a text detection method. Please refer to the embodiments provided. Figure 1 This text detection method can be applied to electronic devices, which can be implemented in various forms. For example, the electronic devices described in this disclosure may include mobile phones, tablets, laptops, desktop computers, handheld computers, personal digital assistants (PDAs), portable media players (PMPs), navigation devices, wearable devices, smart bracelets, etc. The text detection method includes:

[0056] Step S101: Obtain the image to be detected containing the text to be detected.

[0057] The to-be-detected image can be an image containing at least one to-be-detected text. It can be understood that the to-be-detected image can include one to-be-detected text, two to-be-detected texts, three to-be-detected texts, or more to-be-detected texts. The to-be-detected image can be obtained by real-time shooting through a camera, or can be pre-stored on a storage, which is not limited in the embodiments of the present disclosure.

[0058] In step S102, the to-be-detected image is input into the pre-trained text region detection model to obtain a target text region output by the text region detection model. The text region detection model is determined from at least two student models. The text region detection model includes a backbone network layer, a feature fusion layer, and an output layer. The backbone network layer is used to extract features of the to-be-detected image to obtain a feature vector. The feature fusion network is used to fuse the feature vector to obtain a fusion result. The output layer is used to output the target text region according to the fusion result.

[0059] The pre-trained text region detection model is obtained through machine learning. Specifically, first, a training data set is collected, and then the collected training data set is used to train a neural network according to a preset algorithm to model, thereby summarizing a rule based on the training data set to obtain a trained text region detection model.

[0060] It can be understood that the text region detection model can be stored locally on an electronic device after being trained. Please refer to Figure 2 The text region detection model includes a backbone network layer 110, a feature fusion layer 120, and an output layer 130. The backbone network layer 110, the feature fusion layer 120, and the output layer 130 are connected in sequence. An input (Input) is connected to the backbone network layer 110, and an output (Output) is connected to the output layer 130.

[0061] The to-be-detected image is input to the backbone network layer 110 through the Input. The backbone network layer 110 extracts features of the to-be-detected image through a series of convolution calculations and down-sampling to obtain a feature vector, which is transmitted to the feature fusion layer 120. The feature fusion layer 120 fuses the feature vector and can also perform feature enhancement to obtain a fusion result, which is transmitted to the output layer 130. The output layer 130 calculates a text region probability map according to the fusion result and outputs a text region in which the to-be-detected text is located in the to-be-detected image, i.e., a target text region.

[0062] For example, the text region detection model can be a DBNet network model, and the feature fusion layer 120 can be a feature pyramid network (FPN).

[0063] In step S103, text recognition is performed on the target text region in the image to be detected to obtain a text detection result.

[0064] The content corresponding to the target text region in the image to be detected is segmented and then input into the pre-trained text recognition model for text recognition, and then the text information in the image to be detected, i.e., the character detection result, is obtained.

[0065] According to the above technical solution, first, an image to be detected containing text to be detected is obtained; then, the image to be detected is input into a pre-trained text region detection model to obtain a target text region output by the text region detection model, wherein the text region detection model is determined from at least two student models, the text region detection model includes a backbone network layer, a feature fusion layer, and an output layer, the backbone network layer is used for feature extraction on the image to be detected to obtain a feature vector, the feature fusion network is used for fusion on the feature vector to obtain a fusion result, and the output layer is used for outputting the target text region according to the fusion result; finally, text recognition is performed on the target text region in the image to be detected to obtain a text detection result. The accurate target text region in the image to be detected is recognized by the text region detection model determined from the at least two student models, so as to perform text recognition thereon to obtain an accurate text detection result, thereby improving the accuracy of character detection.

[0066] In a possible implementation, referring to Figure 3 , the training method of the text region detection model includes steps S201-S202:

[0067] In step S201, at least two teacher models are trained by using an online knowledge distillation strategy of mutual learning, and a target teacher model is determined therefrom.

[0068] At least two untrained teacher models are obtained, mutual supervision learning is performed between the at least two untrained teacher models, and a teacher model with good learning generalization ability is taken as the target teacher model.

[0069] In step S202, at least two student models are trained by using an offline knowledge distillation strategy based on the target teacher model and the online knowledge distillation strategy of mutual learning, and a target student model is determined therefrom as the text region detection model.

[0070] At least two untrained student models are obtained, mutual supervision learning is performed between the at least two untrained student models, the at least two student models learn from the target teacher model respectively, and a student model with good learning generalization ability is taken as the target student model, i.e., the text region detection model in the present disclosure.

[0071] It should be noted that the learning generalization ability of the model can be estimated by cross-validation, and the learning generalization ability of the model can also be estimated by generalization error, for example, bias, variance and noise, and the present embodiment is not limited thereto.

[0072] Through mutual supervision learning between the teacher models, the target teacher model with good generalization ability is obtained, and then through mutual supervision learning between the student models and learning of the student models to the target teacher model, the target student model with good generalization ability, i.e., the text region detection model, is obtained, so that the target text region detected by the text region detection model is accurate, and thus an accurate text detection result is obtained, and the accuracy of the text detection is improved.

[0073] In a possible implementation, please refer to Figure 4 The online knowledge distillation strategy of mutual learning is used to train the at least two teacher models, which can include steps S211-S214:

[0074] In step S211, at least two teacher models are obtained, and a first training sample is input into each teacher model to obtain a first result output by each teacher model.

[0075] The first training sample is a training sample input into the teacher model, at least two untrained teacher models are obtained, and a first training sample is input into each teacher model, and each teacher model outputs a corresponding first result.

[0076] In step S212, a first loss value is calculated according to the first result output by each teacher model through a first loss function.

[0077] The first loss function can be a mimic loss KL div that can make the class probability of one model consistent with the class probability of another model. Two first results output by each two teacher models are taken as a group, and the first loss value between the two teacher models in the group is calculated through the first loss function, and in this way, all first loss values between the teacher models are obtained.

[0078] In step S213, a second loss value is calculated according to the first result output by each teacher model and the first sample label through a second loss function.

[0079] The second loss function can be a traditional supervised learning loss GT loss. The first result output by each teacher model is calculated through the second loss function together with the first sample label to obtain a second loss value corresponding to each teacher model.

[0080] In step S214, the parameters of the at least two teacher models are adjusted according to the teacher model loss value, and the teacher model loss value includes the first loss value and the second loss value.

[0081] The parameters of each teacher model are adjusted according to the teacher model loss value, and the first loss value and the second loss value contained in the teacher model loss value corresponding to different teacher models are different.

[0082] For example, the at least two teacher models specifically include three teacher models, namely a first teacher model, a second teacher model and a third teacher model. The first training sample is input into the first teacher model, the second teacher model and the third teacher model respectively, and the first result corresponding to the first teacher model, the first result corresponding to the second teacher model and the first result corresponding to the third teacher model are obtained respectively.

[0083] The first result corresponding to the first teacher model and the first result corresponding to the second teacher model are calculated by the first loss function to obtain a first first loss value; the first result corresponding to the first teacher model and the first result corresponding to the third teacher model are calculated by the first loss function to obtain a second first loss value; and the first result corresponding to the second teacher model and the first result corresponding to the third teacher model are calculated by the first loss function to obtain a third first loss value.

[0084] The first result corresponding to the first teacher model and the first sample label are calculated by the second loss function to obtain a first second loss value; the first result corresponding to the second teacher model and the first sample label are calculated by the second loss function to obtain a second second loss value; and the first result corresponding to the third teacher model and the first sample label are calculated by the second loss function to obtain a third second loss value.

[0085] The parameters of the first teacher model can be adjusted according to the first first loss value, the second first loss value and the first second loss value; the parameters of the second teacher model can be adjusted according to the first first loss value, the third first loss value and the second second loss value; and the parameters of the third teacher model can be adjusted according to the second first loss value, the third first loss value and the third second loss value.

[0086] Through mutual supervision learning among the teacher models, a target teacher model with good generalization ability can be obtained, and then a text region detection model capable of accurately detecting a target text region is obtained, so that an accurate text detection result is obtained, and the accuracy of text detection is improved.

[0087] In a possible implementation, the convolution kernel size of the feature fusion layer of the teacher model is one of 3*3, 5*5, 7*7 and 9*9.

[0088] By increasing the convolution kernel in the feature fusion layer, the 1*1 convolution kernel is changed into a 3*3, 5*5, 7*7 or 9*9 convolution kernel, the receptive field covered by each position is improved, and better detection effect is obtained on the fonts of different sizes and the extreme aspect ratio text in the real scene.

[0089] In a possible implementation, referring to Figure 5 , the at least two student models are trained by using an offline knowledge distillation strategy based on a target teacher model and an online knowledge distillation strategy of mutual learning, which can include steps S221-S226.

[0090] In step S221, at least two student models are obtained, and a second training sample is input into each student model to obtain a second result output by each student model.

[0091] The second training sample is a training sample input into the student model, at least two untrained student models are obtained, and the second training sample is input into each student model, and each student model outputs a corresponding second result.

[0092] It should be noted that the first training sample and the second training sample can be the same or different, and the embodiments of the present disclosure do not limit the same.

[0093] In step S222, a third loss value is calculated by using a first loss function according to the second result output by each student model.

[0094] The first loss function can be a mimic loss KL div that can make the class probability of one model consistent with the class probability of another model. Two second results output by each two student models are taken as a group, and the third loss value between the two student models in the group is calculated by using the first loss function, and all third loss values between the student models are obtained in this way.

[0095] In step S223, a fourth loss value is calculated by using a second loss function according to the second result output by each student model and a second sample label.

[0096] The second loss function can be a traditional supervised learning loss GT loss. The second result output by each student model is calculated by using the second loss function together with the second sample label to obtain the fourth loss value corresponding to each student model.

[0097] In step S224, the second training sample is input into the target teacher model to obtain a third result output by the target teacher model.

[0098] In step S225, a fifth loss value is calculated by using a third loss function according to the second result output by each student model and the third result output by the target teacher model.

[0099] The third loss function can be an offline distillation loss. The second result output by each student model and the third result output by the target teacher model are input into the third loss function to obtain a fifth loss value corresponding to each student model.

[0100] In step S226, the parameters of the at least two student models are adjusted according to the student model loss values, which include the third loss value, the fourth loss value, and the fifth loss value.

[0101] The parameters of each student model are adjusted according to the student model loss value, and the third loss value, the fourth loss value, and the fifth loss value included in the student model loss value corresponding to different student models are different.

[0102] For example, the at least two student models specifically include three student models, i.e., a first student model, a second student model, and a third student model. The second training sample is input into the first student model, the second student model, and the third student model to obtain a second result corresponding to the first student model, a second result corresponding to the second student model, and a second result corresponding to the third student model, respectively.

[0103] The first third loss value is calculated by inputting the second result corresponding to the first student model and the second result corresponding to the second student model into the first loss function; the second third loss value is calculated by inputting the second result corresponding to the first student model and the second result corresponding to the third student model into the first loss function; and the third third loss value is calculated by inputting the second result corresponding to the second student model and the second result corresponding to the third student model into the first loss function.

[0104] The first fourth loss value is calculated by inputting the second result corresponding to the first student model and the second sample label into the second loss function; the second fourth loss value is calculated by inputting the second result corresponding to the second student model and the second sample label into the second loss function; and the third fourth loss value is calculated by inputting the second result corresponding to the third student model and the second sample label into the second loss function.

[0105] The second training sample is input into the target teacher model to obtain a third result output by the target teacher model.

[0106] The first fifth loss value is calculated by inputting the second result corresponding to the first student model and the third result output by the target teacher model into the third loss function; the second fifth loss value is calculated by inputting the second result corresponding to the second student model and the third result output by the target teacher model into the third loss function; and the third fifth loss value is calculated by inputting the second result corresponding to the third student model and the third result output by the target teacher model into the third loss function.

[0107] The parameters of the first student model can be adjusted according to the first third loss value, the second third loss value, the first fourth loss value and the first fifth loss value; the parameters of the second student model can be adjusted according to the first third loss value, the third third loss value, the second fourth loss value and the second fifth loss value; and the parameters of the third student model can be adjusted according to the second third loss value, the third third loss value, the third fourth loss value and the third fifth loss value.

[0108] Through mutual supervision learning among the student models and learning of the student models by the target teacher model with good generalization ability, the target student model with good generalization ability can be obtained, that is, the text region detection model that can accurately detect the target text region is obtained, so that accurate text detection results are obtained, and the accuracy of text detection is improved.

[0109] To implement the above method embodiment, the embodiment provides a text detection apparatus 400, as shown in Figure 6 , which is a block diagram of the text detection apparatus 400 according to an example embodiment. The text detection apparatus 400 can include an acquisition module 401, a text region detection module 402 and a text recognition module 403. Figure 6 The acquisition module 401 is configured to acquire a to-be-detected image containing text to be detected.

[0110] The text region detection module 402 is configured to input the to-be-detected image into a pre-trained text region detection model to obtain a target text region output by the text region detection model, wherein the text region detection model is determined from at least two student models, and the text region detection model includes a backbone network layer, a feature fusion layer and an output layer. The backbone network layer is configured to perform feature extraction on the to-be-detected image to obtain a feature vector, the feature fusion network is configured to fuse the feature vector to obtain a fusion result, and the output layer is configured to output the target text region according to the fusion result.

[0111] The text recognition module 403 is configured to perform text recognition on the target text region in the to-be-detected image to obtain a text detection result.

[0112] Optionally, the text detection apparatus 400 further includes:

[0113] The teacher model training module is configured to train at least two teacher models by using an online knowledge distillation strategy of mutual learning, and determine a target teacher model therefrom.

[0114] The student model training module is configured to train at least two student models by using the target teacher model, and determine a target student model therefrom.

[0115] The student model training module is configured to train at least two student models by using an offline knowledge distillation strategy based on the target teacher model and an online knowledge distillation strategy of mutual learning, and determine a target student model as the text region detection model.

[0116] Optionally, the teacher model training module is specifically configured to:

[0117] The at least two teacher models are obtained, and the first training sample is input into each teacher model to obtain a first result output by each teacher model;

[0118] The first loss value is calculated by using the first loss function according to the first result output by each teacher model;

[0119] The second loss value is calculated by using the second loss function according to the first result output by each teacher model and the first sample label;

[0120] The parameters of the at least two teacher models are adjusted according to the teacher model loss value, and the teacher model loss value includes the first loss value and the second loss value.

[0121] Optionally, the convolution kernel size of the feature fusion layer of the teacher model is one of 3*3, 5*5, 7*7 and 9*9.

[0122] Optionally, the student model training module is specifically configured to:

[0123] The at least two student models are obtained, and the second training sample is input into each student model to obtain a second result output by each student model;

[0124] The third loss value is calculated by using the first loss function according to the second result output by each student model,

[0125] The fourth loss value is calculated by using the second loss function according to the second result output by each student model and the second sample label;

[0126] The second training sample is input into the target teacher model to obtain a third result output by the target teacher model;

[0127] The fifth loss value is calculated by using the third loss function according to the second result output by each student model and the third result output by the target teacher model;

[0128] The parameters of the at least two student models are adjusted according to the student model loss value, and the student model loss value includes the third loss value, the fourth loss value and the fifth loss value.

[0129] As to the apparatus in the above-mentioned embodiments, the specific manners in which various modules perform operations have been described in details in the embodiments of the method, and thus will not be described in details here.

[0130] Figure 7 is a block diagram of an electronic device 700 according to an example embodiment. As shown, the electronic device 700 can include a processor 701, a memory 702. The electronic device 700 can also include one or more of a multimedia component 703, an input / output (I / O) interface 704, and a communication component 705. Figure 7

[0131] ​The processor 701 is configured to control overall operations of the electronic device 700 to complete all or part of the steps of the text detection method described above. The memory 702 is configured to store various types of data to support operations of the electronic device 700, which can include, for example, instructions of any application or method operating on the electronic device 700, and application-related data such as contact data, transmitted and received messages, pictures, audio, video, and the like. The memory 702 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic storage, a flash memory, a magnetic disk, or an optical disk. The multimedia component 703 can include a screen and an audio component. The screen can be, for example, a touch screen, and the audio component is configured to output and / or input audio signals. For example, the audio component can include a microphone configured to receive external audio signals. The received audio signals can be further stored in the memory 702 or transmitted through the communication component 705. The audio component further includes at least one speaker configured to output audio signals. The I / O interface 704 provides an interface between the processor 701 and other interface modules, which can be a keyboard, a mouse, a button, and the like. The buttons can be virtual buttons or physical buttons. The communication component 705 is configured to perform wired or wireless communication between the electronic device 700 and other devices. The wireless communication, such as Wi-Fi, Bluetooth, near field communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, and the like, or a combination of one or more of them, is not limited herein. Therefore, the communication component 705 can include, for example, a Wi-Fi module, a Bluetooth module, an NFC module, and the like.

[0132] In an exemplary embodiment, the electronic device 700 can be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors, or other electronic elements for performing the above-described text detection method.

[0133] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, which, when executed by a processor, implement the steps of the above-described text detection method. For example, the computer-readable storage medium can be the above-described memory 702 including program instructions, which can be executed by the processor 701 of the electronic device 700 to complete the above-described text detection method.

[0134] In another exemplary embodiment, a computer program product is also provided, which contains a computer program capable of being executed by a programmable device, and the computer program has code portions for executing the above-described text detection method when executed by the programmable device.

[0135] The preferred embodiments of the present disclosure are described in detail above with reference to the accompanying drawings, but the present disclosure is not limited to the specific details of the above-described embodiments. Within the technical concept range of the present disclosure, various simple modifications can be made to the technical solutions of the present disclosure, and these simple modifications all belong to the protection scope of the present disclosure.

[0136] In addition, it should be noted that each specific technical feature described in the above specific embodiments can be combined in any appropriate manner without contradiction. In order to avoid unnecessary repetition, various possible combinations are not described again by the present disclosure.

[0137] Furthermore, any combination of the various different embodiments of the present disclosure can also be made, as long as it does not deviate from the idea of the present disclosure, and it should also be considered as disclosed by the present disclosure.

Claims

1. A text detection method, characterized in that, The method includes: Obtain the image to be detected that contains the text to be detected; The image to be detected is input into a pre-trained text region detection model to obtain the target text region output by the text region detection model. The text region detection model is determined from at least two student models. The text region detection model includes a backbone network layer, a feature fusion layer, and an output layer. The backbone network layer is used to extract features from the image to be detected to obtain feature vectors. The feature fusion layer is used to fuse the feature vectors to obtain a fusion result. The output layer is used to output the target text region based on the fusion result. Text recognition is performed on the target text region in the image to be detected to obtain the text detection result; the training method of the text region detection model includes: At least two teacher models are trained using a mutual learning online knowledge distillation strategy, and the target teacher model is determined from them. At least two student models are trained using an offline knowledge distillation strategy based on the target teacher model and an online knowledge distillation strategy based on mutual learning, and the target student model is determined as the text region detection model.

2. The method according to claim 1, characterized in that, The online knowledge distillation strategy employing mutual learning is used to train at least two teacher models, including: Obtain at least two teacher models, and input the first training sample into each teacher model to obtain the first result output by each teacher model; The first loss value is calculated based on the first result output by each teacher model and the first loss function. Based on the first result output by each teacher model and the first sample label, the second loss value is calculated using the second loss function; The parameters of the at least two teacher models are adjusted based on the teacher model loss value, wherein the teacher model loss value includes the first loss value and the second loss value.

3. The method according to claim 2, characterized in that, The kernel size of the feature fusion layer of the teacher model is one of 3*3, 5*5, 7*7, or 9*9.

4. The method according to claim 1, characterized in that, The process of training at least two student models using an offline knowledge distillation strategy based on the target teacher model and a mutually learning online knowledge distillation strategy includes: Obtain at least two student models, and input the second training sample into each student model to obtain the second result output by each student model; Based on the second result output by each student model, a third loss value is calculated using the first loss function. Based on the second result and second sample label output by each student model, a fourth loss value is calculated using a second loss function; The second training sample is input into the target teacher model to obtain the third result output by the target teacher model; Based on the second result output by each student model and the third result output by the target teacher model, a fifth loss value is calculated using a third loss function; The parameters for training the at least two student models are adjusted based on the student model loss values, which include the third loss value, the fourth loss value, and the fifth loss value.

5. A text detection device, characterized in that, The text detection device includes: The acquisition module is used to acquire the image to be detected, which contains the text to be detected. A text region detection module is used to input the image to be detected into a pre-trained text region detection model to obtain the target text region output by the text region detection model. The text region detection model is determined from at least two student models. The text region detection model includes a backbone network layer, a feature fusion layer, and an output layer. The backbone network layer is used to extract features from the image to be detected to obtain feature vectors. The feature fusion layer is used to fuse the feature vectors to obtain a fusion result. The output layer is used to output the target text region based on the fusion result. The text recognition module is used to perform text recognition on the target text region in the image to be detected, and obtain the text detection result; The text detection device further includes: The teacher model training module is used to train at least two teacher models using an online knowledge distillation strategy of mutual learning, and to determine the target teacher model from them. The student model training module is used to train at least two student models using an offline knowledge distillation strategy based on the target teacher model and an online knowledge distillation strategy based on mutual learning, and to determine the target student model as the text region detection model.

6. The apparatus according to claim 5, characterized in that, The teacher model training module is specifically used for: Obtain at least two teacher models, and input the first training sample into each teacher model to obtain the first result output by each teacher model; The first loss value is calculated based on the first result output by each teacher model and the first loss function. Based on the first result output by each teacher model and the first sample label, the second loss value is calculated using the second loss function; The parameters of the at least two teacher models are adjusted based on the teacher model loss value, wherein the teacher model loss value includes the first loss value and the second loss value.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method described in any one of claims 1-4.

8. An electronic device, characterized in that, include: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of the method according to any one of claims 1-4.

Images