A text image detection method, device, medium and equipment

Abstract

Embodiments of the present application disclose a text image detection method, device, medium and equipment. The method comprises: obtaining a text image to be detected; using a pre-trained text detection model to detect the text image to be detected to obtain a text detection result; the text detection model comprises a backbone network, a region detection branch and a text segmentation branch; the region detection branch and the text segmentation branch are both connected after the backbone network. The technical solution can enhance the generalization ability of the text detection model by building a multi-branch text detection model, realize accurate region detection frame regression and text segmentation, and effectively improve the robustness and accuracy of text image detection and reduce the detection time.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of image processing, and in particular to a text image detection method, device, medium and equipment. BACKGROUND

[0002] With the continuous development of deep learning technology, as an important part of optical character recognition technology, text detection plays a very important role. Text detection has a wide range of application scenarios, such as ID card, bank card, invoice and express delivery slip text detection scenarios.

[0003] In the prior art, text detection in a text image mainly uses a target segmentation algorithm to first locate a text region in the text image, then performs text segmentation on the text region, and thus achieves the purpose of text detection.

[0004] The method based on text segmentation can effectively solve the problem of diverse text direction distribution, but existing text detection algorithms are prone to false detection, missed detection, and detection frame offset for complex background and blurred text. SUMMARY

[0005] Embodiments of the present application provide a text image detection method, device, medium and equipment, which can build a multi-branch text detection model, enhance the generalization ability of the text detection model, and achieve accurate region detection frame regression and text segmentation.

[0006] In a first aspect, embodiments of the present application provide a text image detection method, which comprises:

[0007] obtaining a text image to be detected;

[0008] performing text detection on the text image to be detected using a pre-trained text detection model to obtain a text detection result; the text detection model comprises a backbone network, a region detection branch and a text segmentation branch; the region detection branch and the text segmentation branch are both connected after the backbone network.

[0009] In a second aspect, embodiments of the present application provide a text image detection device, which comprises:

[0010] a text image to be detected obtaining module for obtaining a text image to be detected;

[0011] a text detection result determining module for performing text detection on the text image to be detected using a pre-trained text detection model to obtain a text detection result; the text detection model comprises a backbone network, a region detection branch and a text segmentation branch; the region detection branch and the text segmentation branch are both connected after the backbone network.

[0012] In a third aspect, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the text image detection method according to the embodiments of the present application.

[0013] In a fourth aspect, an electronic device is provided, and the electronic device includes a memory, a processor, and a computer program stored in the memory and executable by the processor. The processor implements the text image detection method according to the embodiments of the present application when executing the computer program.

[0014] The technical solution provided by the embodiments of the present application obtains a text image to be detected, and uses a pre-trained text detection model to perform text detection on the text image to be detected to obtain a text detection result. The text detection model includes a backbone network, a region detection branch, and a text segmentation branch. The region detection branch and the text segmentation branch are both connected after the backbone network. The present solution can enhance the generalization ability of the text detection model by building a multi-branch text detection model, accurately realize text region detection frame regression and text segmentation, and effectively improve the robustness and accuracy of text image detection and reduce the detection time. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1A is a flowchart of the text image detection method provided by the first embodiment of the present application;

[0016] Figure 1A is a structural schematic diagram of the text detection model provided by the first embodiment of the present application;

[0017] Figure 2 is a flowchart of the text image detection method in the second embodiment of the present application;

[0018] Figure 3 is a structural schematic diagram of the text image detection device provided by the third embodiment of the present application;

[0019] Figure 4 is a structural schematic diagram of the electronic device provided by the fifth embodiment of the present application. DETAILED DESCRIPTION

[0020] The present application will be further described in detail below with reference to the accompanying drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application. In addition, it should be noted that, for the convenience of description, only the parts related to the present application are shown in the drawings, but not all the structures.

[0021] Before any example embodiments are described in further detail, it should be noted that some example embodiments are described as processes or methods depicted as flowcharts. Although the processes are described in a particular sequential order, many of the processes described can be performed concurrently, in parallel, or simultaneously. In addition, the order of individual processes can be re-arranged. The processes can terminate when their functions have been fulfilled, but the processes can also terminate due to other process. The processes can correspond to methods, functions, procedures, subroutines, subprograms, etc.

[0022] Embodiment One

[0023] Figure 1A is a flowchart of a text image detection method provided by Embodiment One of the present application. The embodiment can be applicable to any text image detection scenario. The method can be executed by a text image detection device provided by Embodiments of the present application. The device can be implemented in software and / or hardware, and can be integrated into an electronic device.

[0024] As shown in Figure 1A , the text image detection method comprises:

[0025] S110, obtaining a text image to be detected.

[0026] The scheme can be executed by a computer, a server, a workstation, and other electronic devices. On this basis, the electronic device can be further configured with an image acquisition device. The electronic device can autonomously obtain a text image to be detected. The text image can be picture data such as an ID card, a bank card, a text screenshot, an invoice, and a courier receipt, and can also be video data such as monitoring, movies, and documentaries. The way to obtain the text image to be detected can be to read the picture data collected by the image receiving device in real time, for example, in an ID card recognition scenario. The electronic device can also selectively obtain the text image to be detected under the control of a detection instruction, for example, in a video understanding and video search scenario.

[0027] After obtaining the text image to be detected, if the text image to be detected has blur, inappropriate target size, or the like, the electronic device can preprocess the text image to be detected by means of image enhancement, image cropping, image scaling, and the like, to improve the image quality.

[0028] S120, performing text detection on the text image to be detected by using a pre-trained text detection model to obtain a text detection result; wherein the text detection model comprises a backbone network, a region detection branch, and a text segmentation branch; the region detection branch and the text segmentation branch are both connected after the backbone network.

[0029] The electronic device can perform text detection on the to-be-detected text image by using the pre-trained text detection model, and obtain a detection result. Figure 1B is a structural diagram of the text detection model provided in Embodiment I of the present application, as shown in Figure 1B The text detection model includes a backbone network, a region detection branch, and a text segmentation branch, and the region detection branch and the text segmentation branch are both connected after the backbone network. The region detection branch can be used to locate the region of text in the to-be-detected text image, and mark the text region with a regular-shaped detection box. The text segmentation branch can be used to further segment the text in the text region, and determine the inner boundary and the outer boundary of the text region. The text detection model can be a deep learning model, for example, a convolutional neural network. The backbone network can be any network structure with feature extraction function, for example, a classic convolutional neural network structure such as AlexNet, VGG-Net, and ResNet, or a self-built network structure that meets the scene requirements. The region detection branch can have a network structure with feature extraction function, or only a structure with region detection function. Similarly, the text segmentation branch can have a structure with feature extraction function, or only a structure with text segmentation function. If the region detection branch and the text segmentation branch both have a network structure with feature extraction function, their network structures can be the same or different.

[0030] In this scheme, optionally, the text segmentation branch includes a segmentation shared network, a first text segmentation branch, and a second text segmentation branch; the first text segmentation branch and the second text segmentation branch are both connected after the segmentation shared network.

[0031] To achieve a targeted text segmentation goal, the text segmentation branch can further branch the text segmentation branch according to the focus of text segmentation. As shown in Figure 1B The text segmentation branch can include a segmentation shared network, a first text segmentation branch, and a second text segmentation branch. The first text segmentation branch and the second text segmentation branch are both connected after the segmentation shared network. The first text segmentation branch can be used to focus on the irregular region composed of the inner boundary and the outer boundary in the text region. The second text segmentation branch can be used to focus on the difference between the text and the background in the text region.

[0032] This scheme can focus on different features in the text segmentation process by branching the text segmentation branch again, so as to achieve more distinctive text detection.

[0033] In order to realize the extraction of the target features of each branch, the electronic device can set different loss functions for each branch. In one possible solution, optionally, the region detection branch is provided with a first loss function; the first character segmentation branch is provided with a second loss function; and the second character segmentation branch is provided with a third loss function.

[0034] On the basis of the above solution, optionally, the first loss function expression is:

[0035] L1 = λ1l1;

[0036] l1 = l A + l B ;

[0037]

[0038]

[0039] wherein L1 represents the first loss function, λ1 is a preset coefficient, l A represents regression on the length and width of the character region, l B represents regression on the character region, x and y represent the pixel point coordinate positions of the text image, c represents the number of extracted feature channels, Y xyc represents the probability of the existence of a character at the coordinates x and y, represents the probability of the existence of a character at the coordinates x and y predicted by the character detection model, and α and β are hyperparameters. N represents the number of key points of the text image.

[0040] The second loss function expression is:

[0041] L2 = λ2l2;

[0042]

[0043] wherein L2 represents the second loss function, λ2 is a preset coefficient, (i, j) represents the coordinate position of the feature map, S l represents the feature map, Y (i，j) represents the true result of the character at the position (i, j) of the feature map, represents the predicted result of the character at the position (i, j) of the feature map.

[0044] The third loss function expression is:

[0045] L3 = λ3l3;

[0046]

[0047] Wherein, L3 represents a second loss function, λ3 is a preset coefficient, P represents a probability map of a character region, T represents a segmentation map of the character region, and γ is a hyperparameter.

[0048] Wherein, λ1+λ2+λ3=1.

[0049] The above scheme can make the character detection model learn different distribution characteristics of text by designing a multi-branch structure. Meanwhile, different loss functions are designed for different branch tasks, aiming to improve the robustness of the character detection model for character detection and improve the generalization ability. This scheme can effectively solve the problems of character mis-detection, missed detection and detection frame deviation caused by image blur and complex background.

[0050] In addition, in the above scheme, the preset coefficients λ1, λ2 and λ3 can be used to set the influence degree of each branch on the backbone network and the segmentation shared network of the character detection model, which is beneficial to realize flexible and effective network adjustment, accelerate the convergence speed of the character detection model, and improve the detection effect.

[0051] Optionally, the training process of the character detection model comprises:

[0052] Obtaining text image training data;

[0053] Inputting the text image training data into the backbone network of the character detection model to obtain intermediate features, and transmitting the intermediate features to the region detection branch and the character segmentation branch; wherein the intermediate features pass through the region detection branch to obtain region detection features; and the intermediate features pass through the character segmentation branch to obtain character segmentation features;

[0054] According to the region detection features and the character segmentation features, the backbone network is optimized until a preset condition is met to obtain a training result of the character detection model.

[0055] When training the character detection model, first, text image training data needs to be obtained, which can be a large number of original text images. In the case of less text image training data, the electronic device can also use a small amount of original text images to expand through rotation, mirroring and scaling, and then obtain diversified text image training data, which is beneficial to train a robust character detection model. In the case of difficult text image acquisition, the electronic device can also construct and generate text image training data by using other types of text image templates and text data.

[0056] After obtaining a sufficient amount of text image training data, the electronic device can input the text image training data in batches to the character detection model for subsequent training. Under the premise that the hardware environment condition allows, the electronic device can also input all the text image training data to the character detection model for training. After the backbone network, the character detection model can learn the intermediate features in the text image training data and pass the intermediate features to the region detection branch and the character segmentation branch.

[0057] The intermediate features pass through the region detection branch to obtain region detection features, and feed the region detection features back to the backbone network, so that the backbone network can be continuously optimized to extract features that are more in line with the requirements of region detection. The intermediate features pass through the character segmentation branch to obtain character segmentation features, and feed the character segmentation features back to the backbone network, so that the backbone network can be continuously optimized to extract features that are more in line with the requirements of character segmentation. Through the continuous optimization of the two branches to the backbone network, the backbone network is continuously learned. After the training reaches stability, the backbone network can extract features that meet the requirements of region detection, and can also extract features that meet the requirements of character segmentation.

[0058] It should be noted that the preset condition can be that the loss rates of the region detection branch and the character segmentation branch both meet the respective preset threshold values. The preset condition can also be that the loss curves of the region detection branch and the character segmentation branch drawn according to the training iteration numbers both conform to the preset shape. In addition, the electronic device can also calculate the overall loss rate of the character detection model or draw the loss curve according to the loss rates of the respective branches and the respective proportion coefficients. Whether the training requirement is met can be determined through the overall loss rate or the loss curve. In addition to the loss rate and the loss curve, the determination basis of the preset condition can also be the accuracy, the recall rate, and the precision, etc. The determination basis of the preset condition can be one or more.

[0059] The present scheme can optimize the backbone network using region detection features and character segmentation features. This scheme can enhance the robustness of the backbone network, enable the backbone network to learn in a targeted manner, thereby accelerating the training speed and ensuring the stability of the character detection model training.

[0060] The technical scheme provided in the embodiments of the present application comprises the following steps: obtaining a to-be-detected text image, performing text detection on the to-be-detected text image by using a pre-trained text detection model, and obtaining a text detection result. The text detection model comprises a backbone network, a region detection branch, and a text segmentation branch. The region detection branch and the text segmentation branch are both connected after the backbone network. The present scheme can enhance the generalization ability of the text detection model by building a multi-branch text detection model, realize accurate text region detection frame regression and text segmentation, and thus effectively improve the robustness and accuracy of text image detection and reduce the detection time.

[0061] Embodiment Two

[0062] Figure 2 For the flowchart of the text image detection method in the second embodiment of the present application, the present embodiment is optimized based on the above-mentioned embodiments.

[0063] As Figure 2 shown, the method of the present embodiment specifically comprises the following steps:

[0064] S210, obtaining text image training data.

[0065] S220, inputting the text image training data into a backbone network of a text detection model to obtain intermediate features, and transmitting the intermediate features to a region detection branch and a text segmentation branch; wherein the intermediate features obtain region detection features through the region detection branch; and the intermediate features obtain segmentation shared features through the segmentation shared network.

[0066] As Figure 1B can be understood, the structure of the text detection model shown in the present embodiment, the intermediate features obtain segmentation shared features through the segmentation shared network. The electronic device can continue to transmit the segmentation shared features.

[0067] S230, transmitting the segmentation shared features to a first text segmentation branch and a second text segmentation branch; wherein the segmentation shared features obtain first text segmentation features through the first text segmentation branch; and the segmentation shared features obtain second text segmentation features through the second text segmentation branch.

[0068] The electronic device can transmit the segmentation shared features to the first text segmentation branch and the second text segmentation branch, and obtain the corresponding segmentation features.

[0069] S240, optimizing the segmentation shared network according to the first character segmentation feature and the second character segmentation feature; and optimizing the backbone network according to the region detection feature, the first character segmentation feature and the second character segmentation feature, until a preset condition is met to obtain a training result of the character detection model.

[0070] According to the first character segmentation feature and the second character segmentation feature, the electronic device can optimize the segmentation shared network by constantly feeding back different segmentation features fed back by the first character segmentation branch and the second character segmentation branch to the segmentation shared network. Meanwhile, the electronic device can also optimize the backbone network by constantly feeding back respective features of the first character segmentation branch, the second character segmentation branch and the region detection branch to the backbone network. When the training result of the character detection model reaches the preset condition, it is considered that the character detection model has been successfully trained.

[0071] S250, obtaining a text image to be detected.

[0072] S260, performing character detection on the text image to be detected by using the pre-trained character detection model; the character detection model is a deep learning model.

[0073] S270, determining a character detection result according to the character segmentation branch.

[0074] When performing character detection in an actual scene, the electronic device does not need to determine a final detection result according to the output results of multiple branches. It only needs to determine a character detection result according to the output result of the character segmentation branch. This is because the character detection model finally implements character detection, and the region detection branch only marks the regular region where the character is located and cannot implement detection of the character boundary. Furthermore, the electronic device can determine the character detection result only according to the output result of the first character segmentation branch or the second character segmentation branch. It should be noted that due to the robustness of the character detection model, the character detection result determined according to a single branch is still accurate and has universal adaptability.

[0075] The present scheme can determine the character detection result according to the character segmentation branch, which is beneficial to speed up the character detection speed and save time.

[0076] The technical scheme provided by the embodiment of the application optimizes the shared network by using the first character segmentation feature and the second character segmentation feature; the backbone network is optimized by using the region detection feature, the first character segmentation feature and the second character segmentation feature; and then the training result of the character detection model meeting the preset condition is obtained. The trained deep learning model is used to determine the character detection result according to the first character segmentation branch. The multi-branch character detection model is built, the generalization ability of the character detection model is enhanced, accurate character region detection frame regression and character segmentation are realized, and the robustness and accuracy of the text image detection are effectively improved, and the detection time is reduced.

[0077] Embodiment three

[0078] Figure 3 A structure diagram of a text image detection device provided by the third embodiment of the application is shown in the figure. The device can perform the text image detection method provided by any embodiment of the application, has the function modules and beneficial effects corresponding to the execution method. As shown in the figure, the device can include: Figure 3

[0079] The text image detection device further includes a text image detection result determination module 320, configured to use the pre-trained character detection model to perform character detection on the to-be-detected text image to obtain a character detection result.

[0080] The text image detection device further includes a text image detection result determination module 320, configured to use the pre-trained character detection model to perform character detection on the to-be-detected text image to obtain a character detection result.

[0081] In the scheme, optionally, the character segmentation branch includes a segmentation shared network, a first character segmentation branch and a second character segmentation branch; the first character segmentation branch and the second character segmentation branch are connected after the segmentation shared network.

[0082] In a feasible scheme, optionally, the region detection branch is provided with a first loss function; the first character segmentation branch is provided with a second loss function; and the second character segmentation branch is provided with a third loss function.

[0083] On the basis of the above scheme, optionally, the first loss function expression is:

[0084] The first loss function expression is:

[0085] L1=λ1l1;

[0086] l1=l A +l B ; ​

[0087]

[0088]

[0089] wherein, L1 represents a first loss function, λ1 is a preset coefficient, l A represents regression on the length and width of the text region, l B represents regression on the text region, x, y represent the pixel coordinate position of the text image, c represents the number of extracted feature channels, Y xyc represents the probability of the existence of text at coordinates x, y, represents the probability of the existence of text at coordinates x, y predicted by the text detection model, α and β are hyperparameters, and N represents the number of key points of the text image.

[0090] and the second loss function expression is:

[0091] L2 = λ2l2;

[0092]

[0093] wherein, L2 represents a second loss function, λ2 is a preset coefficient, (i, j) represents the coordinate position of the feature map, S l represents the feature map, Y (i，j) represents the text real result of the feature map (i, j) position, represents the text prediction result of the feature map (i, j) position.

[0094] and the third loss function expression is:

[0095] L3 = λ3l3;

[0096]

[0097] wherein, L3 represents a second loss function, λ3 is a preset coefficient, P represents a probability map of the text region, T represents a segmentation map of the text region, and γ is a hyperparameter.

[0098] wherein, λ1 + λ2 + λ3 = 1.

[0099] Optionally, the device further comprises a text detection model training module, configured to:

[0100] obtain text image training data;

[0101] input the text image training data into a backbone network of the text detection model to obtain intermediate features, and pass the intermediate features to a region detection branch and a text segmentation branch; wherein the intermediate features pass through the region detection branch to obtain region detection features; and the intermediate features pass through the text segmentation branch to obtain text segmentation features;

[0102] According to the region detection features and the text segmentation features, the backbone network is optimized until a preset condition is met to obtain a training result of the text detection model.

[0103] In a preferred scheme, optionally, the text detection model training module is specifically configured to:

[0104] obtain text image training data;

[0105] input the text image training data into a backbone network of the text detection model to obtain intermediate features, and pass the intermediate features to a region detection branch and a text segmentation branch; wherein the intermediate features pass through the region detection branch to obtain region detection features; and the intermediate features pass through the segmentation shared network to obtain segmentation shared features;

[0106] pass the segmentation shared features to a first text segmentation branch and a second text segmentation branch; wherein the segmentation shared features pass through the first text segmentation branch to obtain first text segmentation features; and the segmentation shared features pass through the second text segmentation branch to obtain second text segmentation features;

[0107] According to the first text segmentation features and the second text segmentation features, the segmentation shared network is optimized; according to the region detection features, the first text segmentation features and the second text segmentation features, the backbone network is optimized; and until a preset condition is met to obtain a training result of the text detection model.

[0108] In another preferred embodiment, optionally, the text detection result determination module 320 is specifically configured to:

[0109] perform text detection on the text image to be detected by using a pre-trained text detection model; the text detection model is a deep learning model;

[0110] determine a text detection result according to the text segmentation branch.

[0111] The above product can perform the text image detection method provided in the embodiments of the present application, and has the corresponding function modules and beneficial effects of the execution method.

[0112] Embodiment Four

[0113] The embodiment four of the present application provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to realize the text image detection method provided by all the embodiments of the present application.

[0114] Obtaining a text image to be detected;

[0115] Performing text detection on the text image to be detected by using a pre-trained text detection model to obtain a text detection result; wherein the text detection model comprises a backbone network, a region detection branch and a text segmentation branch; the region detection branch and the text segmentation branch are both connected after the backbone network.

[0116] Any combination of one or more computer readable medium can be employed. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination of the above. More specific examples (a non-exhaustive list) of the computer readable storage medium include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus or device.

[0117] The computer readable signal medium can include a data signal propagated in a baseband or as a carrier wave in a propagated data signal, which bears computer readable program code. Such propagated data signal can take various forms, including but not limited to, an electromagnetic signal, an optical signal or any suitable combination of the above. The computer readable signal medium can also be any computer readable medium that can send, propagate or transfer a program for use by or in connection with an instruction execution system, apparatus or device.

[0118] The program code contained in the computer readable medium can be transmitted by any suitable medium, including but not limited to wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0119] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0120] Embodiment five

[0121] An electronic device is provided in the embodiment five of the present application. Figure 4 As shown in the embodiment, the electronic device 400 includes one or more processors 420, a storage 410 configured to store one or more programs, and the one or more programs, when executed by the one or more processors 420, cause the one or more processors 420 to implement the method for detecting a text image provided in the embodiments of the present application, which includes: Figure 4

[0122] obtaining a text image to be detected;

[0123] performing text detection on the text image to be detected by using a pre-trained text detection model to obtain a text detection result, wherein the text detection model includes a backbone network, a region detection branch, and a text segmentation branch, and the region detection branch and the text segmentation branch are both connected after the backbone network.

[0124] Of course, those skilled in the art can understand that the processor 420 also implements the technical solutions of the method for detecting a text image provided in any of the embodiments of the present application.

[0125] Figure 4 The electronic device 400 shown is merely an example and should not impose any limitations on the functions and use range of the embodiments of the present application.

[0126] As shown in the embodiment, the electronic device 400 includes one or more processors 420, a storage 410 configured to store one or more programs, and the one or more programs, when executed by the one or more processors 420, cause the one or more processors 420 to implement the method for detecting a text image provided in the embodiments of the present application, which includes: Figure 4 Figure 4 ​​The processor 420 in the electronic device is taken as an example; the processor 420, the storage device 410, the input device 430 and the output device 440 in the electronic device can be connected through a bus or other means, Figure 4 The processor 420 in the electronic device is taken as an example; the processor 420, the storage device 410, the input device 430 and the output device 440 in the electronic device can be connected through a bus or other means,

[0127] The storage device 410 can be used to store software programs, computer executable programs and module units, such as program instructions corresponding to the text image detection method in the embodiments of the present application.

[0128] The storage device 410 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system and application programs required by at least one function; the data storage area can store data created according to the use of the terminal and the like. In addition, the storage device 410 can include a high-speed random access memory, and can also include a non-volatile memory, for example, at least one magnetic disk storage device, a flash memory device or other non-volatile solid-state storage device. In some examples, the storage device 410 can further include a memory remotely arranged with respect to the processor 420, and these remote memories can be connected through a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0129] The input device 430 can be used to receive input digital, character information or voice information, and generate key signal input related to user settings and function control of the electronic device. The output device 440 can include a display screen, a loudspeaker and other electronic devices.

[0130] The electronic device provided in the embodiments of the present application can build a multi-branch text detection model, which can enhance the generalization ability of the text detection model, and realize accurate text region detection frame regression and text segmentation.

[0131] The text image detection device, medium and electronic device provided in the above embodiments can execute the text image detection method provided in any embodiment of the present application, and have the corresponding function modules and beneficial effects of executing the method. Technical details not described in detail in the above embodiments can be referred to the text image detection method provided in any embodiment of the present application.

[0132] Note that the above merely describes preferred embodiments of the present application and the principles of the technology applied. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and that various obvious changes, modifications and substitutions can be made without departing from the scope of the present application. Therefore, although the present application has been described in detail through the above embodiments, the present application is not limited to the above embodiments, and can include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the claims.

Claims

1. A method for detecting text images, characterized in that, The method includes: Obtain the image of the text to be detected; A pre-trained text detection model is used to detect text in the text image to be detected, and the text detection result is obtained. The text detection model includes a backbone network, a region detection branch, and a text segmentation branch. The region detection branch and the text segmentation branch are both connected after the backbone network. The region detection branch is used to locate the region where the text is located in the text image to be detected, and to label the text region with a regular-shaped detection box. The text segmentation branch includes a segmentation sharing network, a first text segmentation branch, and a second text segmentation branch; both the first text segmentation branch and the second text segmentation branch are connected after the segmentation sharing network; the first text segmentation branch is used to focus on irregular regions composed of inner and outer boundaries in the text region, and the second text segmentation branch is used to focus on the differences between text and background in the text region; The region detection branch is equipped with a first loss function; the first text segmentation branch is equipped with a second loss function; the second text segmentation branch is equipped with a third loss function. The first loss function expression is: ; ; ; ; in, Represents the first loss function. For preset coefficients, This indicates a regression of the length and width of the text area. This indicates a regression to the text region. , This represents the pixel coordinates of the text image. Indicates the number of feature channels extracted. Indicates coordinates , The probability that text exists at a given location. This indicates that the text detection model predicts the coordinates... , The probability that text exists at a given location. and For hyperparameters, This represents the number of key points in the text image; And, the expression for the second loss function is: ; ; in, Represents the second loss function. For preset coefficients, Indicates the coordinate position of the feature map. Representing feature maps, Representation of feature map The actual result of the location text. Representation of feature map The text prediction results for the location; And, the expression for the third loss function is: ; ; in, Represents the third loss function. For preset coefficients, A probability graph representing a text region. A segmentation diagram representing a text region. For hyperparameters; in, .

2. The method according to claim 1, characterized in that, The training process of the text detection model includes: Obtain text image training data; The text image training data is input into the backbone network of the text detection model to obtain intermediate features, and the intermediate features are then passed to the region detection branch and the text segmentation branch; wherein, the intermediate features are processed by the region detection branch to obtain region detection features; and the intermediate features are processed by the text segmentation branch to obtain text segmentation features. Based on the region detection features and the text segmentation features, the backbone network is optimized until it meets the preset conditions to obtain the training results of the text detection model.

3. The method according to claim 1, characterized in that, The training process of the text detection model includes: Obtain text image training data; The text image training data is input into the backbone network of the text detection model to obtain intermediate features, and the intermediate features are then passed to the region detection branch and the text segmentation branch; wherein, the intermediate features are processed by the region detection branch to obtain region detection features; and the intermediate features are processed by the segmentation sharing network to obtain segmentation sharing features. The segmentation shared features are passed to the first text segmentation branch and the second text segmentation branch; wherein, the segmentation shared features are passed through the first text segmentation branch to obtain the first text segmentation feature; and the segmentation shared features are passed through the second text segmentation branch to obtain the second text segmentation feature. The segmentation sharing network is optimized based on the first and second text segmentation features; the backbone network is optimized based on the region detection features, the first and second text segmentation features; until the preset conditions are met to obtain the training results of the text detection model.

4. The method according to claim 1, characterized in that, The step of using a pre-trained text detection model to perform text detection on the text image to be detected, and obtaining text detection results, includes: The text detection model is a deep learning model used to detect text in the image to be detected, which is a pre-trained text detection model. Based on the text segmentation branches, the text detection results are determined.

5. A text image detection device, characterized in that, The device includes: The text image acquisition module is used to acquire the text image to be detected; The text detection result determination module is used to perform text detection on the text image to be detected using a pre-trained text detection model to obtain the text detection result; the text detection model includes a backbone network, a region detection branch, and a text segmentation branch; the region detection branch and the text segmentation branch are both connected after the backbone network; the region detection branch is used to locate the region where the text is located in the text image to be detected, and to label the text region with a regular-shaped detection box; The text segmentation branch includes a segmentation sharing network, a first text segmentation branch, and a second text segmentation branch; both the first text segmentation branch and the second text segmentation branch are connected after the segmentation sharing network; the first text segmentation branch is used to focus on irregular regions composed of inner and outer boundaries in the text region, and the second text segmentation branch is used to focus on the differences between text and background in the text region; The region detection branch is equipped with a first loss function; the first text segmentation branch is equipped with a second loss function; the second text segmentation branch is equipped with a third loss function. The first loss function expression is: ; ; ; ; in, Represents the first loss function. For preset coefficients, This indicates a regression of the length and width of the text area. This indicates a regression to the text region. , This represents the pixel coordinates of the text image. Indicates the number of feature channels extracted. Indicates coordinates , The probability that text exists at a given location. This indicates that the text detection model predicts the coordinates... , The probability that text exists at a given location. and For hyperparameters, This represents the number of key points in the text image; And, the expression for the second loss function is: ; ; in, Represents the second loss function. For preset coefficients, Indicates the coordinate position of the feature map. Representing feature maps, Representation of feature map The actual result of the location text. Representation of feature map The text prediction results for the location; And, the expression for the third loss function is: ; ; in, Represents the third loss function. For preset coefficients, A probability graph representing a text region. A segmentation diagram representing a text region. For hyperparameters; in, .

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the text image detection method as described in any one of claims 1-4.

7. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the text image detection method as described in any one of claims 1-4.

Images