Natural scene text detection model training method and device, server and storage medium

Abstract

Embodiments of the present application disclose a natural scene text detection model training method and device, a server and a storage medium. The method comprises: fusing features of an image to form a feature fusion image; inputting the feature fusion image into a probability prediction network to obtain a text probability prediction image output by the probability prediction network; calculating a pixel difficulty value according to a pixel value and a pixel prediction value; setting a progressive difficulty coefficient according to a progressive difficulty adjustment rule, calculating a training difficulty of a corresponding pixel according to the progressive difficulty coefficient and the pixel difficulty value, calculating a loss value according to the training difficulty, the corresponding pixel value and the pixel prediction value, returning to the step of setting the progressive difficulty coefficient according to the progressive difficulty adjustment rule when the loss value is less than a preset loss threshold, and repeating until the loss value is greater than the preset loss threshold; and training the natural scene text detection model according to the pixel prediction value corresponding to the training difficulty.

Description

Technical Field

[0001] This invention relates to the field of text detection technology, and in particular to a method, apparatus, server, and storage medium for training a text detection model in natural scenes. Background Technology

[0002] Text, as the most expressive form of information, records the diverse technological and cultural heritage of humankind and can be embedded in documents or scenes as communication information. Text in scene images can be broadly categorized into two types: artificial text and scene text. With the development of internet technology, text localization and recognition technology has been widely applied in daily life, such as license plate recognition and ID card recognition. Traditional ORC recognition technology can only recognize printed text with a single background and fixed font. However, text in scene images exhibits diversity, such as irregular arrangement and inconsistent font sizes. Furthermore, factors such as lighting intensity or shooting angle can cause blurry or incomplete fonts, which strongly interfere with text detection and severely affect its accuracy.

[0003] Scene text detection can be categorized into two main approaches: those based on manually designed features and those based on deep learning. Currently, manually designed feature methods suffer from drawbacks such as complex design processes, poor robustness, and limited applicability. Deep learning-based scene text detection can be further divided into regression-based and segmentation-based methods. Regression-based methods typically directly regress the coordinates of text regions to obtain the final text region, while segmentation-based methods usually predict a pixel-level probability map and then obtain the coordinates of the text region through post-processing.

[0004] However, the complexity of natural scenes, the diversity of text appearances, and the frequent presence of degradation and interference in images make text recognition in natural scene images quite challenging. The methods mentioned above do not specifically address the image itself to improve recognition conditions, thus often failing to further enhance recognition performance. Summary of the Invention

[0005] This invention provides a method, apparatus, server, and storage medium for training a natural scene text detection model, in order to solve the technical problem that it is difficult to improve the performance of scene text detection technology in the prior art.

[0006] In a first aspect, embodiments of the present invention provide a method for training a natural scene text detection model, including:

[0007] The images in the natural scene text image training set are fused to form a feature fusion map;

[0008] The feature fusion map is input into the probability prediction network to obtain the text probability prediction map output by the probability prediction network. Each pixel value of the text probability prediction map represents the probability that the corresponding position is a text region, and the pixel prediction value of that pixel is obtained.

[0009] Calculate the pixel difficulty value based on the pixel value and the pixel prediction value;

[0010] The progressive difficulty coefficient is set according to the progressive difficulty adjustment rules, and the training difficulty of the corresponding pixel is calculated based on the progressive difficulty coefficient and the pixel difficulty value. The progressive difficulty adjustment rules include: calculating the progressive difficulty coefficient based on the preset maximum difficulty value, the starting difficulty value and the maximum number of training rounds.

[0011] The loss value is calculated based on the training difficulty, the corresponding pixel value, and the pixel prediction value. When the loss value is less than the preset loss threshold, the process returns to the step of setting the progressive difficulty coefficient according to the progressive difficulty adjustment rule until the loss value is greater than the preset loss threshold.

[0012] The natural scene text detection model is trained according to the pixel prediction value corresponding to the training difficulty to obtain the trained natural scene text detection model.

[0013] Secondly, embodiments of the present invention also provide an apparatus for training a natural scene text detection model, comprising:

[0014] The fusion module is used to fuse features from images in the natural scene text image training set to form a feature fusion map.

[0015] The input module is used to input the feature fusion map into the probability prediction network to obtain the text probability prediction map output by the probability prediction network. Each pixel value of the text probability prediction map represents the probability that the corresponding position is a text region, and the pixel prediction value of that pixel is obtained accordingly.

[0016] The calculation module is used to calculate the pixel difficulty value based on the pixel value and the pixel prediction value;

[0017] The training difficulty calculation module is used to set the progressive difficulty coefficient according to the progressive difficulty adjustment rules, and calculate the training difficulty of the corresponding pixel based on the progressive difficulty coefficient and the pixel difficulty value. The progressive difficulty adjustment rules include: calculating the progressive difficulty coefficient based on the preset maximum difficulty value, the starting difficulty value and the maximum number of training rounds.

[0018] The loss value calculation module is used to calculate the loss value based on the training difficulty, the corresponding pixel value and the pixel prediction value. When the loss value is less than the preset loss threshold, it returns to the step of setting the progressive difficulty coefficient according to the progressive difficulty adjustment rule until the loss value is greater than the preset loss threshold.

[0019] The training module is used to train the natural scene text detection model according to the pixel prediction value corresponding to the training difficulty, so as to obtain the trained natural scene text detection model.

[0020] Thirdly, embodiments of the present invention also provide a server, comprising:

[0021] One or more processors;

[0022] Storage device for storing one or more programs;

[0023] When the one or more programs are executed by the one or more processors, the one or more processors implement the natural scene text detection model training method provided in the above embodiments.

[0024] Fourthly, embodiments of the present invention also provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the natural scene text detection model training method provided in the above embodiments.

[0025] The natural scene text detection model training method, apparatus, server, and storage medium provided in this invention embodiment fuses features of images in a natural scene text image training set to form a feature fusion map; inputs the feature fusion map into a probabilistic prediction network to obtain a text probability prediction map output by the probabilistic prediction network, where each pixel value of the text probability prediction map represents the probability that the corresponding position is a text region, and a pixel prediction value is obtained for that pixel; a pixel difficulty value is calculated based on the pixel value and the pixel prediction value; a progressive difficulty coefficient is set according to a progressive difficulty adjustment rule, and the training difficulty of the corresponding pixel is calculated based on the progressive difficulty coefficient and the pixel difficulty value, wherein the progressive difficulty adjustment rule includes: calculating the progressive difficulty coefficient based on a preset maximum difficulty value, a starting difficulty value, and a maximum number of training rounds; a loss value is calculated based on the training difficulty, the corresponding pixel value, and the pixel prediction value; when the loss value is less than a preset loss threshold, the step of setting the progressive difficulty coefficient according to the progressive difficulty adjustment rule is returned until the loss value is greater than the preset loss threshold; the natural scene text detection model is trained according to the pixel prediction value corresponding to the training difficulty to obtain a trained natural scene text detection model. A high-performing scene text detection model can be obtained through a progressively more difficult learning process. To address the issues of overly simplified supervision and positive sample noise, a difficulty value is assigned to each pixel sample using a probabilistic map, with noise treated as a positive sample with a higher difficulty coefficient. By applying different penalties to pixels of varying difficulty, the goal of hard sample mining is achieved. Furthermore, by controlling the overall difficulty value of pixels one step at a time through different difficulty coefficients at different training stages, the network gradually learns samples of varying difficulty levels. Attached Figure Description

[0026] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0027] Figure 1 This is a flowchart of the natural scene text detection model training method provided in Embodiment 1 of the present invention;

[0028] Figure 2 This is a schematic diagram of the receptive field extension network in the natural scene text detection model training method provided in Embodiment 1 of the present invention;

[0029] Figure 3 This is a structural diagram of the apparatus for training a natural scene text detection model according to Embodiment 2 of the present invention;

[0030] Figure 4 This is a structural diagram of the server provided in Embodiment 3 of the present invention. Detailed Implementation

[0031] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0032] Example 1

[0033] Figure 1 This is a flowchart of a natural scene text detection model training method provided in Embodiment 1 of the present invention. This embodiment is applicable to the training of a detection model for recognizing natural scene text. The method can be executed by a natural scene text detection model training device and can be set in a server. Specifically, it includes the following steps:

[0034] Step 110: Perform feature fusion on the images in the natural scene text image training set to form a feature fusion map.

[0035] In this embodiment, a large number of natural scene text images can be pre-collected as a natural scene text image training set. Natural scene text images are obtained from the natural scene text image training set and processed to form a feature fusion map, which facilitates the training of the natural scene text detection model.

[0036] Optionally, the step of fusing features from images in the natural scene text image training set to form a feature fusion map may include: acquiring images from the natural scene text image training set; performing image enhancement operations such as random flipping, random rotation, random mirror flipping, or random cropping on the images respectively; sequentially inputting multiple enhanced images into a residual network model to obtain three first feature maps at different scales for each image; and inputting all first feature maps into a BiFPN feature fusion network to obtain three feature fusion maps at different scales.

[0037] Data augmentation enriches sample data, helping to prevent overfitting in the network model. The augmented image is then input into the ResNet backbone to obtain feature maps at three different scales. These three feature maps are then fused using a BiFPN feature fusion network to obtain a fused feature map. This fused feature map facilitates subsequent network learning of scene text at different scales.

[0038] Furthermore, the aforementioned feature fusion maps can be further processed using deformable convolutions with strides of 3x3 (2 strides), 3x3 (3 strides), and 5x5 (5 strides) to obtain maps with scales corresponding to the original maps. Figure 1 The 2, 1 / 3, and 1 / 5 size channels are eigenmaps representing 1 / 3 of the input channel. These eigenmaps are resized to the input size using dilated convolutions and stacked with the input channel. Finally, a 1x1 convolution is used to make the channels identical to the input, resulting in the final output feature map. Since long texts exist in scene text detection tasks, requiring a large receptive field, this network was added. The goal of this network is to increase the receptive field of each pixel in the output feature map. First, deformable convolutions are used to obtain the original... Figure 1 Feature maps of sizes 2, 1 / 3, and 1 / 5 are generated, and then dilated convolutions are used to resize these feature maps to the same size as the original image. This expands the receptive field of the three feature maps to 2, 3, and 5 times their original size, respectively. However, this expansion comes at the cost of finer granularity. Therefore, the original input feature map and the three feature maps are stacked together by channel, and a single convolution is used to further fuse the original image containing fine-grained features with the three images with expanded receptive fields. Figure 2 This is a schematic diagram of the receptive field extension network in the natural scene text detection model training method provided in Embodiment 1 of the present invention. Figure 2 As can be seen, it can include deformable convolution and dilated convolution. This will output the final feature map.

[0039] Step 120: Input the feature fusion map into the probability prediction network to obtain the text probability prediction map output by the probability prediction network. Each pixel value of the text probability prediction map represents the probability that the corresponding position is a text region, and the pixel prediction value of that pixel is obtained.

[0040] Optionally, the feature fusion map can be input into a semantic segmentation neural network; the output of the deconvolution layer of the semantic segmentation neural network can be obtained; and a text probability prediction map can be output according to the pixel position of the feature fusion map corresponding to the output result.

[0041] Semantic segmentation neural networks first perform convolution -> convolution -> pooling on the image, then convolution -> convolution -> pooling again, until the image is reduced to a sufficiently small size. Then, upsampling can be performed to restore the image size. The deconvolution layer (deconv) of the semantic segmentation neural network can output image features. The output probabilities are used to obtain a text probability prediction map, where each pixel value is the probability value of the corresponding pixel in the original image.

[0042] Step 130: Calculate the pixel difficulty value based on the pixel value and the pixel prediction value.

[0043] Typically, segmentation-based methods are used to supervise the learning of input text using pixel-level masks. However, conventional segmentation-based text detection simply converts text regions into pixel masks for supervision. Furthermore, different samples have varying learning difficulties, making it unreasonable to apply the same loss constraint to positive samples of different difficulties. Therefore, in this embodiment, a progressively harder learning process is used to obtain a high-performing scene text detection model. A difficulty value is assigned to each pixel sample using a probability map.

[0044] The pixel difficulty value is calculated based on the pixel value and the pixel prediction value in the following manner:

[0045]

[0046] Where y i For pixel values, The predicted value for each pixel. This difficulty is a tensor of size H*W, representing the difficulty at each location in the predicted image. The height and width correspond to the image's dimensions.

[0047] Step 140: Set the progressive difficulty coefficient according to the progressive difficulty adjustment rules, and calculate the training difficulty of the corresponding pixel based on the progressive difficulty coefficient and the pixel difficulty value. The progressive difficulty adjustment rules include: calculating the progressive difficulty coefficient based on the preset maximum difficulty value, the starting difficulty value and the maximum number of training rounds.

[0048] After calculating the pixel difficulty value, different penalties can be applied to pixels of different difficulty to achieve the goal of hard sample mining. Furthermore, by controlling the overall difficulty value of pixels one step at a time through different difficulty coefficients at different training stages, the network gradually learns to handle hard samples of varying degrees.

[0049] First, an initial difficulty coefficient is set based on the initial state, and then the difficulty coefficient is continuously increased. The training difficulty of each pixel is then calculated. The progressive difficulty coefficient is set according to progressive difficulty adjustment rules.

[0050] The rules for adjusting the gradual difficulty are as follows:

[0051]

[0052] Where E i This is the current training round. (D) max and D min α represents the maximum and minimum difficulty values. i To achieve a progressive difficulty level, E max This represents the maximum number of training rounds.

[0053] Step 150: Calculate the loss value based on the training difficulty, the corresponding pixel value, and the pixel prediction value. When the loss value is less than the preset loss threshold, return to the step of setting the progressive difficulty coefficient according to the progressive difficulty adjustment rule until the loss value is greater than the preset loss threshold.

[0054] The difficulty of the training process is allocated according to the current training round, using the following formula: This is a numerical value. Calculate the difficulty graph for the current step, i.e., the numerical value α. i With difficulty map Multiply them. This gives us the difficulty map and the predicted image for the current step. Input both images into the loss function to calculate the loss, guiding the model training to complete the progressively harder learning. Progressive difficulty is controlled by α, which gradually increases with training. The formula for determining α is:

[0055]

[0056] Where E i This is the current training round. (D) max and D min These are the maximum and minimum difficulty values, typically set to 10 and 0.7.

[0057] E max This represents the maximum number of training epochs. Therefore, the training difficulty of pixels at different difficulty levels is:

[0058]

[0059] The system loss function is:

[0060]

[0061] Where k is an empirical coefficient, and the value of this system is 10.

[0062] Step 160: Train the natural scene text detection model according to the pixel prediction value corresponding to the training difficulty to obtain the trained natural scene text detection model.

[0063] The goal is to gradually increase the training difficulty of the network model to prioritize optimizing easier positions and then gradually learn more difficult positions. The difficulty-controlled loss function is based on the difficulty map and probability prediction map generated by the difficulty control strategy, and the final loss of the network is used to avoid overfitting and other issues.

[0064] This embodiment fuses features from images in a natural scene text image training set to form a feature fusion map. The feature fusion map is then input into a probabilistic prediction network to obtain a text probability prediction map output by the network. Each pixel value in the text probability prediction map represents the probability that the corresponding location is a text region, and a pixel prediction value is obtained for that pixel. A pixel difficulty value is calculated based on the pixel value and the pixel prediction value. A progressive difficulty coefficient is set according to a progressive difficulty adjustment rule. The training difficulty of the corresponding pixel is calculated based on the progressive difficulty coefficient and the pixel difficulty value. The progressive difficulty adjustment rule includes: calculating the progressive difficulty coefficient based on a preset maximum difficulty value, a starting difficulty value, and a maximum number of training rounds. A loss value is calculated based on the training difficulty, the corresponding pixel value, and the pixel prediction value. If the loss value is less than a preset loss threshold, the step of setting the progressive difficulty coefficient according to the progressive difficulty adjustment rule is returned until the loss value is greater than the preset loss threshold. The natural scene text detection model is trained according to the pixel prediction value corresponding to the training difficulty to obtain a trained natural scene text detection model. A high-performing scene text detection model can be obtained through a progressively difficult learning process. To address the issues of overly simplistic supervision and positive sample noise, a difficulty value is assigned to each pixel sample using a probabilistic map, with noise treated as a positive sample with a higher difficulty coefficient. By applying different penalties to pixels of varying difficulty, the goal of hard sample mining is achieved. Furthermore, by controlling the overall difficulty value of pixels step-by-step at different training stages using different difficulty coefficients, the network gradually learns samples of varying difficulty levels.

[0065] Example 2

[0066] Figure 3 This is a structural diagram of the apparatus for training a natural scene text detection model according to Embodiment 4 of the present invention, as shown below. Figure 3 As shown, the device includes:

[0067] The fusion module 210 is used to fuse features of images in the natural scene text image training set to form a feature fusion map;

[0068] The input module 220 is used to input the feature fusion map into the probability prediction network to obtain the text probability prediction map output by the probability prediction network. Each pixel value of the text probability prediction map represents the probability that the corresponding position is a text region, and the pixel prediction value of that pixel is obtained accordingly.

[0069] Calculation module 230 is used to calculate pixel difficulty value based on pixel value and pixel prediction value;

[0070] The training difficulty calculation module 240 is used to set the progressive difficulty coefficient according to the progressive difficulty adjustment rules, and calculate the training difficulty of the corresponding pixel according to the progressive difficulty coefficient and the pixel difficulty value. The progressive difficulty adjustment rules include: calculating the progressive difficulty coefficient according to the preset maximum difficulty value, the starting difficulty value and the maximum number of training rounds.

[0071] The loss value calculation module 250 is used to calculate the loss value based on the training difficulty, the corresponding pixel value and the pixel prediction value. When the loss value is less than the preset loss threshold, it returns to the step of setting the progressive difficulty coefficient according to the progressive difficulty adjustment rule until the loss value is greater than the preset loss threshold.

[0072] The training module 260 is used to train the natural scene text detection model according to the pixel prediction value corresponding to the training difficulty, so as to obtain the trained natural scene text detection model.

[0073] The natural scene text detection model training device provided in this embodiment fuses features from images in a natural scene text image training set to form a feature fusion map; the feature fusion map is input into a probabilistic prediction network to obtain a text probability prediction map output by the probabilistic prediction network, where each pixel value represents the probability that the corresponding position is a text region, and a pixel prediction value is obtained for that pixel; a pixel difficulty value is calculated based on the pixel value and the pixel prediction value; a progressive difficulty coefficient is set according to a progressive difficulty adjustment rule, and the training difficulty of the corresponding pixel is calculated based on the progressive difficulty coefficient and the pixel difficulty value, the progressive difficulty adjustment rule including: calculating the progressive difficulty coefficient based on a preset maximum difficulty value, a starting difficulty value, and a maximum number of training rounds; a loss value is calculated based on the training difficulty, the corresponding pixel value, and the pixel prediction value; when the loss value is less than a preset loss threshold, the step of setting the progressive difficulty coefficient according to the progressive difficulty adjustment rule is returned until the loss value is greater than the preset loss threshold; the natural scene text detection model is trained according to the pixel prediction value corresponding to the training difficulty to obtain a trained natural scene text detection model. A high-performing scene text detection model can be obtained through a progressively difficult learning process. To address the issues of overly simplistic supervision and positive sample noise, a difficulty value is assigned to each pixel sample using a probabilistic map, with noise treated as a positive sample with a higher difficulty coefficient. By applying different penalties to pixels of varying difficulty, the goal of hard sample mining is achieved. Furthermore, by controlling the overall difficulty value of pixels step-by-step at different training stages using different difficulty coefficients, the network gradually learns samples of varying difficulty levels.

[0074] Based on the above embodiments, the calculation module calculates in the following manner:

[0075]

[0076] Where y i For pixel values, The predicted value for a pixel.

[0077] Based on the above embodiments, the progressive difficulty adjustment rule is as follows:

[0078]

[0079] Where E i This is the current training round. (D) max and D min α represents the maximum and minimum difficulty values. i To achieve a progressive difficulty level, E max This represents the maximum number of training rounds.

[0080] Based on the above embodiments, the training difficulty calculation module includes:

[0081] The pixel difficulty calculation unit is implemented in the following way:

[0082]

[0083] Based on the above embodiments, the fusion module includes:

[0084] The acquisition unit is used to acquire images from the natural scene text image training set;

[0085] The enhancement unit is used to perform image enhancement operations on the image, such as random reversal, random rotation, random mirror reversal, or random cropping.

[0086] The first input unit is used to sequentially input multiple enhanced images into the residual network model to obtain three first feature maps at different scales for each image.

[0087] The second input unit inputs all the first feature maps into the BiFPN feature fusion network to obtain feature fusion maps at three different scales.

[0088] Based on the above embodiments, the fusion module further includes:

[0089] The obtaining unit is used to obtain the feature fusion map by performing deformable convolutions with strides of 3*3 (2 strides), 3*3 (3 strides), and 5*5 (5 strides) to obtain the original feature maps with different scales. Figure 1 The 2, 1 / 3, and 1 / 5 size channels are feature maps representing 1 / 3 of the input channel;

[0090] The adjustment unit is used to adjust the size of the three feature maps of different scales to the input size through dilated convolution and stack them with the input channel. Finally, the channel is adjusted to be the same as the input through 1*1 convolution to obtain the final output feature map.

[0091] Based on the above embodiments, the input module includes:

[0092] The input unit is used to input the feature fusion map into the semantic segmentation neural network;

[0093] The acquisition unit is used to acquire the output of the deconvolution layer of the semantic segmentation neural network;

[0094] The output unit is used to output a text probability prediction map based on the pixel position of the feature fusion map corresponding to the output result.

[0095] The natural scene text detection model training method apparatus provided in the embodiments of the present invention can execute the natural scene text detection model training method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0096] Example 3

[0097] Figure 4 This is a structural diagram of the server provided in Embodiment 3 of the present invention. Figure 4 A block diagram of an exemplary server 12 suitable for implementing embodiments of the present invention is shown. Figure 4 The server 12 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0098] like Figure 4 As shown, server 12 is presented as a general-purpose computing device. The components of server 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0099] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0100] Server 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by server 12, including volatile and non-volatile media, removable and non-removable media.

[0101] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache 32. Server 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (… Figure 4 Not shown; usually referred to as a "hard drive"). Although Figure 4 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0102] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of the present invention.

[0103] Server 12 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable users to interact with server 12, and / or with any device that enables server 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 22. Furthermore, server 12 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20. As shown, network adapter 20 communicates with other modules of server 12 via bus 18. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with server 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0104] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the natural scene text detection model training method provided in the embodiments of the present invention.

[0105] Example 5

[0106] Embodiment 5 of the present invention also provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the natural scene text detection model training method as described in any of the above embodiments.

[0107] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0108] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0109] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including—but not limited to—wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0110] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0111] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A method for training a text detection model in natural scenes, characterized in that, include: The images in the natural scene text image training set are fused to form a feature fusion map; The feature fusion map is input into the probability prediction network to obtain the text probability prediction map output by the probability prediction network. Each pixel value of the text probability prediction map represents the probability that the corresponding position is a text region, and the pixel prediction value of that pixel is obtained. Calculate the pixel difficulty value based on the pixel value and the pixel prediction value; The progressive difficulty coefficient is set according to the progressive difficulty adjustment rules, and the training difficulty of the corresponding pixel is calculated based on the progressive difficulty coefficient and the pixel difficulty value. The progressive difficulty adjustment rules include: calculating the progressive difficulty coefficient based on the preset maximum difficulty value, the starting difficulty value and the maximum number of training rounds. The loss value is calculated based on the training difficulty, the corresponding pixel value, and the pixel prediction value. When the loss value is less than the preset loss threshold, the process returns to the step of setting the progressive difficulty coefficient according to the progressive difficulty adjustment rule until the loss value is greater than the preset loss threshold. The natural scene text detection model is trained according to the pixel prediction value corresponding to the training difficulty to obtain the trained natural scene text detection model. The pixel difficulty value is calculated based on the pixel value and the pixel prediction value in the following manner: Where y i For pixel values, The predicted value for a pixel; The rules for adjusting the gradual difficulty are as follows: Where E i For the current training round, D max and D min α represents the maximum and minimum difficulty values. i To achieve a progressive difficulty level, E max This represents the maximum number of training rounds.

2. The method according to claim 1, characterized in that, The calculation of the training difficulty of the corresponding pixel based on the progressive difficulty coefficient and the pixel difficulty value is achieved in the following way:

3. The method according to claim 1, characterized in that, The step of fusing features from images in the natural scene text image training set to form a feature fusion map includes: Obtain images from the natural scene text image training set; The image is subjected to image enhancement operations such as random reversal, random rotation, random mirror reversal, or random cropping. Multiple enhanced images are sequentially input into the residual network model, and three first feature maps at different scales are obtained for each image. Input all the first feature maps into the bifpn feature fusion network to obtain feature fusion maps at three different scales.

4. The method according to claim 3, characterized in that, The step of fusing features from images in the natural scene text image training set to form a feature fusion map further includes: The fused feature map is processed by deformable convolutions with strides of 2, 3, 3, and 5, resulting in system feature maps with sizes of 1 / 2, 1 / 3, and 1 / 5 of the original image, and with each channel being 1 / 3 of the input channel. The system feature map is adjusted to the input size by dilated convolution of three feature maps of different scales and stacked with the input channels. Finally, a 1*1 convolution is used to adjust the channels to be equal to the input to obtain the final output feature map.

5. The method according to claim 4, characterized in that, The step of inputting the feature fusion map into the probabilistic prediction network to obtain the text probability prediction map output by the probabilistic prediction network includes: The feature fusion map is input into a semantic segmentation neural network; Obtain the output of the deconvolution layer of the semantic segmentation neural network; Based on the pixel positions of the corresponding feature fusion map, a text probability prediction map is output.

6. A training apparatus for a text detection model in natural scenes, characterized in that, include: The fusion module is used to fuse features from images in the natural scene text image training set to form a feature fusion map. The input module is used to input the feature fusion map into the probability prediction network to obtain the text probability prediction map output by the probability prediction network. Each pixel value of the text probability prediction map represents the probability that the corresponding position is a text region, and the pixel prediction value of that pixel is obtained accordingly. The calculation module is used to calculate the pixel difficulty value based on the pixel value and the pixel prediction value; The training difficulty calculation module is used to set the progressive difficulty coefficient according to the progressive difficulty adjustment rules, and calculate the training difficulty of the corresponding pixel based on the progressive difficulty coefficient and the pixel difficulty value. The progressive difficulty adjustment rules include: calculating the progressive difficulty coefficient based on the preset maximum difficulty value, the starting difficulty value and the maximum number of training rounds. The loss value calculation module is used to calculate the loss value based on the training difficulty, the corresponding pixel value and the pixel prediction value. When the loss value is less than the preset loss threshold, it returns to the step of setting the progressive difficulty coefficient according to the progressive difficulty adjustment rule until the loss value is greater than the preset loss threshold. The training module is used to train the natural scene text detection model according to the pixel prediction value corresponding to the training difficulty, so as to obtain the trained natural scene text detection model. The calculation module performs the calculation in the following manner: Where y i For pixel values, The predicted value for a pixel; The rules for adjusting the gradual difficulty are as follows: Where E i For the current training round, D max and D min α represents the maximum and minimum difficulty values. i To achieve a progressive difficulty level, E max This represents the maximum number of training rounds.

7. A server, characterized in that, The server includes: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the natural scene text detection model training method as described in any one of claims 1-5.

8. A storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the natural scene text detection model training method as described in any one of claims 1-5.

Images