Method and system for automatic recognition of special font on internet
By constructing a multiplexing model and combining Mask TextSpotter V3 and Language Prediction Network (LPN), the problems of large data volume and docking error in the recognition of mixed text with special fonts are solved, and efficient and accurate multilingual text recognition is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- INNER MONGOLIA AUTONOMOUS REGION ACAD OF SOCIAL SCI
- Filing Date
- 2023-04-24
- Publication Date
- 2026-05-22
AI Technical Summary
In existing technologies, the text detection, language identification, and text recognition tasks for special font mixed text recognition consume a large amount of data and are prone to docking errors, resulting in low recognition efficiency and poor performance.
A multi-path multiplexing model is constructed, which combines the Mask TextSpotter V3 recognition model and spatial attention module, and adds a language prediction network LPN. Through parallel training and fine-tuning techniques, efficient and accurate special font recognition is achieved. The multi-path multiplexing model is used to integrate text detection, text recognition and text recognition.
It improves the efficiency and accuracy of special font recognition, expands multilingual recognition capabilities, reduces entanglement loss, and achieves efficient and accurate multilingual text recognition.
Smart Images
Figure CN116403225B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of character recognition technology, and more specifically, to an automatic Internet recognition method and system for special fonts. Background Technology
[0002] Text recognition for special mixed-language texts (such as Mongolian script mixed with other languages) can generally be divided into three sub-tasks: text detection, language identification, and text recognition. Text detection is usually the first stage of understanding the text content in an image. Early methods typically used manually designed features or heuristics, such as connected components or sliding windows. Later, early deep learning models used convolutional networks in combination with the above methods. Text recognition in multilingual scenarios usually requires language identification to determine the language of the text. Once the text is located through detection, the region is identified as the text to be recognized and cropped, then fed into the text recognition system for recognition and reading as a sequence of characters or words.
[0003] In existing technologies, the three sub-tasks mentioned above are generally considered separately and then linked together. However, this approach has some drawbacks: the amount of data consumed between each task is too large, and connection errors are prone to occur; the recognition efficiency is low, and the recognition effect is poor. Summary of the Invention
[0004] To overcome or at least partially solve the above problems, this invention provides an automatic Internet font recognition method and system that uses a multiplexing model to efficiently and accurately recognize special fonts.
[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:
[0006] In a first aspect, the present invention provides an automatic Internet font recognition method, comprising the following steps:
[0007] An initial recognition model is constructed based on the detection and segmentation module and the spatial attention module of the Mask TextSpotter V3 recognition model. A new language prediction network (LPN) is added on top of the initial recognition model to obtain a multiplexing model.
[0008] To build a separate recognition head for the multiplexing model to process words from different languages, select or combine them for prediction;
[0009] We acquired and trained the Language Prediction Network (LPN) and the recognition head in parallel based on labeled datasets of multiple languages and target special fonts, and determined the loss function of the multiplexing model.
[0010] The fine-tuning technique in deep learning is used to annotate and acquire the layout data of the target special font, and the layout analysis model is trained to obtain the layout analysis model of the target special font.
[0011] The target special font is identified using a target special font layout analysis model.
[0012] This method is a further development based on the existing Mask TextSpotter V3 recognition model. It constructs a multi-channel model integrating text detection, text-to-text recognition, and text recognition for efficient and accurate special font recognition. This multi-channel model uses the same detection and segmentation modules as the Mask TextSpotter V3 model, and adds a new language prediction network to determine which script's recognition head the multiplexer selects. The model uses a spatial attention module for text recognition. To extend the multi-channel model from a single language to multiple languages, separate recognition heads are built to process words from different languages, and then selection or combination predictions are made, greatly increasing the scope of expansion. To further improve recognition accuracy, the loss function of the multi-channel model is determined based on the parallel training of the labeled dataset's information language prediction network (LPN) and the recognition head to eliminate entanglement loss. Furthermore, data annotation and training are involved. Fine-tuning techniques from deep learning are used to annotate and acquire target special font layout data, and then trained to obtain the corresponding target special font layout analysis model, which is then used for efficient and accurate recognition of special fonts.
[0013] Based on the first aspect, further, the input of the above-mentioned Language Prediction Network (LPN) is the mask set features of the detection and segmentation modules.
[0014] Based on the first aspect, furthermore, the output of the aforementioned Language Prediction Network (LPN) is used to determine which script's recognition head the multiplexing model selects.
[0015] Based on the first aspect, the aforementioned language prediction network LPN further includes two 2×2 convolutional layers with ReLU activation function and one 2×2 pooling layer.
[0016] Based on the first aspect, further, the method described above for annotating and acquiring target special font layout data using fine-tuning techniques in deep learning, and training a layout analysis model to obtain a target special font layout analysis model includes the following steps:
[0017] Input manually annotated target special font layout data and preset English layout annotation data, train the layout analysis model to obtain the target layout analysis model;
[0018] The target layout analysis model is adjusted using a preset initial special font layout model, and the target special font layout data is identified using the adjusted target layout analysis model to obtain the identification results.
[0019] The target special font layout annotation data is obtained and determined based on the manual review of the recognition results, and the model is trained to obtain the target special font layout analysis model.
[0020] Secondly, this invention provides an automatic internet font recognition system, comprising a model building module, a recognition head building module, a loss function determination module, a data annotation and training module, and a font recognition module, wherein:
[0021] The model building module is used to build an initial recognition model based on the detection and segmentation module and the spatial attention module of the Mask TextSpotter V3 recognition model. A new language prediction network (LPN) is added on top of the initial recognition model to obtain a multiplexed model.
[0022] The recognition head building module is used to build separate recognition heads for multiplexing models to process words from different languages and select or combine them for prediction.
[0023] The loss function determination module is used to acquire and perform parallel training of the Language Prediction Network (LPN) and the recognition head based on labeled datasets of multiple different languages and target special fonts, and to determine the loss function of the multiplexed model.
[0024] The data annotation and training module is used to annotate and acquire target special font layout data using fine-tuning techniques in deep learning, and to train the layout analysis model to obtain the target special font layout analysis model.
[0025] The font recognition module is used to identify target special fonts using a target special font layout analysis model.
[0026] This system, through the collaboration of multiple modules including model building, recognition head construction, loss function determination, data annotation and training, and font recognition, represents a further development based on the existing Mask TextSpotter V3 recognition model. It constructs a multi-way multiplexing model integrating text detection, text-to-text recognition, and text recognition for efficient and accurate special font recognition. This multi-way multiplexing model uses the same detection and segmentation modules as the Mask TextSpotter V3 model, and adds a new language prediction network to determine which script's recognition head the multiplexer selects. The model employs a spatial attention module for text recognition. To extend the multi-way multiplexing model from a single language to multiple languages, separate recognition heads are built to process words from different languages, and then selection or combination predictions are made, significantly improving the scalability. To further improve recognition accuracy, the loss function of the multiplexing model is determined based on the parallel training of the Language Prediction Network (LPN) and the recognition head on the labeled dataset, so as to eliminate entanglement loss. On this basis, data annotation and training are also involved. The fine-tuning technique in deep learning is used to annotate and acquire target special font layout data, and then train it to obtain the corresponding target special font layout analysis model, and based on this, the special font is efficiently and accurately recognized.
[0027] Thirdly, this application provides an electronic device including a memory for storing one or more programs; a processor; and, when the one or more programs are executed by the processor, implementing the method as described in any of the first aspects above.
[0028] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any of the first aspects above.
[0029] The present invention has at least the following advantages or beneficial effects:
[0030] This invention provides an automatic Internet font recognition method and system. It constructs a multiplexing model that integrates text detection, text recognition, and text identification to perform efficient and accurate recognition of special fonts. The multiplexing model uses the same detection and segmentation modules as the Mask TextSpotter V3 recognition model, and adds a new language prediction network on this basis. This network can determine which script's recognition head the multiplexer selects. Attached Figure Description
[0031] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0032] Figure 1 This is a flowchart illustrating an internet automatic font recognition method according to an embodiment of the present invention;
[0033] Figure 2 This is a schematic diagram of a multiplexing model in an embodiment of the present invention;
[0034] Figure 3 This is a schematic diagram of the Language Prediction Network (LPN) in an embodiment of the present invention;
[0035] Figure 4 This is a schematic diagram of an internet automatic font recognition system according to an embodiment of the present invention;
[0036] Figure 5 This is a structural block diagram of an electronic device provided in an embodiment of the present invention.
[0037] Figure labeling: 100, Model building module; 200, Recognition head building module; 300, Loss function determination module; 400, Data annotation and training module; 500, Font recognition module; 101, Memory; 102, Processor; 103, Communication interface. Detailed Implementation
[0038] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.
[0039] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.
[0040] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.
[0041] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the term "comprising" or any other variations thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0042] In the description of the embodiments of the present invention, "multiple" means at least two.
[0043] Example:
[0044] like Figures 1-3 As shown, in a first aspect, embodiments of the present invention provide an internet automatic font recognition method, comprising the following steps:
[0045] S1. Construct an initial recognition model based on the detection and segmentation module and spatial attention module of the Mask TextSpotter V3 recognition model. Add a new language prediction network (LPN) on top of the initial recognition model to obtain a multiplexing model.
[0046] Furthermore, the input to the language prediction network LPN is the mask set features of the detection and segmentation modules.
[0047] Furthermore, the output of the aforementioned Language Prediction Network (LPN) is used to determine which script's recognition head the multiplexing model selects.
[0048] Furthermore, the language prediction network LPN described above includes two 2×2 convolutional layers with ReLU activation function and one 2×2 pooling layer.
[0049] In some embodiments of the present invention, in order to automatically select a recognition module suitable for a given language, the project employs a Language Prediction Network (LPN), such as... Figure 3 As shown. The input to the LPN is a mask set of features from the detection and segmentation modules, with a size of 256×32×32. The project plans to use a standard classification network with two 2×2 convolutional layers with ReLU activation and one 2×2 pooling layer. This network will produce an output vector of size L, which can be converted into probabilities using Softmax, where L = Nlang is the number of language classes we want the model to support. Figure 3 In the middle, L = Nlan g=8 indicates that the default model in this paper supports a maximum of eight languages. The number L of different recognition heads in the model does not necessarily have to be equal to the number of supported languages N. lang The number of characters, especially in the case of a shared alphabet, such as the Mongolian alphabet which can be used for Mongolian-based languages (e.g., Todo Mongolian, Manchu), means that LPN predictions for any of these languages can be routed to a single Mongolian-based recognition head. The special fonts described in this invention can be Mongolian, seal script, oracle bone script, etc.
[0050] S2. Build separate recognition heads for the multiplexing model to process words from different languages and select or combine them for prediction;
[0051] S3. Obtain and train the Language Prediction Network (LPN) and the recognition head in parallel based on labeled datasets of multiple different languages and target special fonts, and determine the loss function of the multiplexing model.
[0052] In some embodiments of the present invention, since partially labeled datasets are already available for languages such as English, Chinese, and Mongolian, the LPN and recognition head can be trained in parallel, while eliminating entanglement loss. That is, the loss terms for each head and LPN are calculated in parallel, and then they are directly added together:
[0053]
[0054] Where L lang It is the loss of LPN, R is the set of recognition heads, and L is the loss of LPN. seq(r) It is the loss of the recognition head r.
[0055] Language prediction is a standard multi-class classification problem, so the language prediction loss in equation (1) can be calculated using cross-entropy loss:
[0056]
[0057] Whether the language matches the annotation language is determined by I(l=l) gt The calculation is performed to represent true or false, and p(l) is the probability that the word inferred by LPN belongs to language l.
[0058] In this invention, negative log-likelihood can be used as the text recognition loss L. seq(r) :
[0059]
[0060] Where p(y) t =c t ) is the predicted probability of the character at position t in the sequence, where t is the length of the character label sequence.
[0061] To calculate L seq(r) That is, L supports different recognition heads for different character sets. seq Unsupported characters need to be ignored in the loss calculation:
[0062]
[0063] Where C r It identifies the character set supported by the header r, c t It is the ground truth character of step t, if c is supported. t ,I(c t ∈C r ) = 1, if c is not supported t Then I(c t ∈C r ) = 0.
[0064] S4. Use fine-tuning techniques in deep learning to annotate and acquire the target special font layout data, and train the layout analysis model to obtain the target special font layout analysis model.
[0065] Furthermore, this includes: inputting manually annotated target special font layout data and preset English layout annotation data, training a layout analysis model to obtain a target layout analysis model; adjusting the target layout analysis model using a preset initial special font layout model, and using the adjusted target layout analysis model to identify the target special font layout data to obtain identification results; obtaining and determining the target special font layout annotation data based on the manual review information of the identification results, and training the model to obtain the target special font layout analysis model.
[0066] In some embodiments of the present invention, the fine-tuning technique in deep learning is used to complete the annotation and acquisition of Mongolian page layout data. Taking Mongolian as a special font as an example, the specific method includes: first, manually annotating a small amount of Mongolian page layout data, while simultaneously using a large amount of English page layout annotation data to train a page layout analysis model to obtain a page layout analysis model; then, using the Mongolian page layout model to fine-tune this model, and after obtaining a model with a certain recognition rate, using this model to recognize a larger amount of Mongolian page layout data; then, manually reviewing and correcting errors to obtain a larger amount of Mongolian page layout annotation data; and then training again using the fine-tuning method executed in the previous step, repeating the process until the ideal Mongolian page layout analysis model is finally obtained.
[0067] S5. Use the target special font layout analysis model to identify the target special font.
[0068] This method is a further development based on the existing Mask TextSpotter V3 recognition model. It constructs a multi-channel model integrating text detection, text-to-text recognition, and text recognition for efficient and accurate special font recognition. This multi-channel model uses the same detection and segmentation modules as the Mask TextSpotter V3 model, and adds a new language prediction network to determine which script's recognition head the multiplexer selects. The model uses a spatial attention module for text recognition. To extend the multi-channel model from a single language to multiple languages, separate recognition heads are built to process words from different languages, and then selection or combination predictions are made, greatly increasing the scope of expansion. To further improve recognition accuracy, the loss function of the multi-channel model is determined based on the parallel training of the labeled dataset's information language prediction network (LPN) and the recognition head to eliminate entanglement loss. Furthermore, data annotation and training are involved. Fine-tuning techniques from deep learning are used to annotate and acquire target special font layout data, and then trained to obtain the corresponding target special font layout analysis model, which is then used for efficient and accurate recognition of special fonts.
[0069] like Figure 4 As shown, in a second aspect, embodiments of the present invention provide an automatic internet font recognition system, comprising a model building module 100, a recognition head building module 200, a loss function determination module 300, a data annotation and training module 400, and a font recognition module 500, wherein:
[0070] The model building module 100 is used to build an initial recognition model based on the detection and segmentation module and the spatial attention module of the Mask TextSpotter V3 recognition model. A new language prediction network LPN is added on top of the initial recognition model to obtain a multiplexed model.
[0071] Recognition head building module 200 is used to build separate recognition heads for multiplexing models to process words from different languages and select or combine them for prediction;
[0072] The loss function determination module 300 is used to acquire and perform parallel training of the language prediction network LPN and the recognition head based on labeled datasets of multiple different languages and target special fonts, and to determine the loss function of the multiplexing model.
[0073] The data annotation and training module 400 is used to annotate and acquire target special font layout data using fine-tuning techniques in deep learning, and to train the layout analysis model to obtain the target special font layout analysis model.
[0074] The font recognition module 500 is used to recognize target special fonts using a target special font layout analysis model.
[0075] This system, through the collaboration of multiple modules including model building module 100, recognition head building module 200, loss function determination module 300, data annotation and training module 400, and font recognition module 500, further develops upon the existing Mask TextSpotterV3 recognition model. It constructs a multi-channel multiplexing model integrating text detection, text-to-text recognition, and text recognition for efficient and accurate special font recognition. This multi-channel multiplexing model uses the same detection and segmentation modules as the Mask TextSpotterV3 recognition model, and adds a new language prediction network to determine which script's recognition head the multiplexer selects. The model employs a spatial attention module for text recognition. To extend the multi-channel multiplexing model from a single language to multiple languages, separate recognition heads are built to process words from different languages, and then selection or combination predictions are made, greatly improving the scalability. To further improve recognition accuracy, the loss function of the multiplexing model is determined based on the parallel training of the Language Prediction Network (LPN) and the recognition head on the labeled dataset, so as to eliminate entanglement loss. On this basis, data annotation and training are also involved. The fine-tuning technique in deep learning is used to annotate and acquire target special font layout data, and then train it to obtain the corresponding target special font layout analysis model, and based on this, the special font is efficiently and accurately recognized.
[0076] like Figure 5 As shown, in a third aspect, embodiments of this application provide an electronic device including a memory 101 for storing one or more programs; and a processor 102. When the one or more programs are executed by the processor 102, they implement the methods described in any of the first aspects above.
[0077] The system also includes a communication interface 103. The memory 101, processor 102, and communication interface 103 are electrically connected directly or indirectly to each other to enable data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines. The memory 101 can be used to store software programs and modules, and the processor 102 executes various functional applications and data processing by executing the software programs and modules stored in the memory 101. The communication interface 103 can be used for signaling or data communication with other node devices.
[0078] The memory 101 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.
[0079] The processor 102 can be an integrated circuit chip with signal processing capabilities. The processor 102 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.
[0080] In the embodiments provided in this application, it should be understood that the disclosed methods and systems can also be implemented in other ways. The method and system embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of methods and systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.
[0081] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.
[0082] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program thereon. When executed by processor 102, the computer program implements the methods described in any of the first aspects above. If the functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0083] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
[0084] It will be apparent to those skilled in the art that this application is not limited to the details of the exemplary embodiments described above, and that this application can be implemented in other specific forms without departing from the spirit or essential characteristics of this application. Therefore, the embodiments should be considered illustrative and non-limiting in all respects, and the scope of this application is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within this application. No reference numerals in the claims should be construed as limiting the scope of the claims.
Claims
1. A method for automatic recognition of special fonts on the Internet, characterized in that, Includes the following steps: An initial recognition model is constructed based on the detection and segmentation module and the spatial attention module of the Mask TextSpotter V3 recognition model. A new language prediction network (LPN) is added on top of the initial recognition model to obtain a multiplexing model. To build a separate recognition head for the multiplexing model to process words from different languages, select or combine them for prediction; We acquired and trained the Language Prediction Network (LPN) and the recognition head in parallel using labeled datasets of multiple languages and target-specific fonts, and determined the loss function of the multiplexing model. Specifically, we trained the LPN and the recognition head in parallel while eliminating entanglement loss, i.e., we calculated the loss terms for each head and the LPN in parallel and then directly added them together. ; where L lang It is the loss of LPN, R is the set of recognition heads, and L is the loss of LPN. seq(r) The loss is for recognizing the head r; language prediction is a standard multi-type classification problem, so the language prediction loss in equation (1) can be calculated using cross-entropy loss: Whether the language matches the annotation language is determined by I(l=l) gt The calculation is performed to represent true or false, and p(l) is the probability that the word inferred by LPN belongs to language l; Using negative log-likelihood as the text recognition loss L seq(r) : ;where p(y t =c t L is the predicted probability of the character at position t in the sequence, where t is the length of the character label sequence; to calculate L seq(r) That is, L supports different recognition heads for different character sets. seq Unsupported characters need to be ignored in the loss calculation: ; where C r It identifies the character set supported by the header r, c t It is the ground truth character of step t, if c is supported. t ,I(c t ∈C r )=1, if c is not supported t Then I(c t ∈C r )=0; The fine-tuning technique in deep learning is used to annotate and acquire the layout data of the target special font, and the layout analysis model is trained to obtain the layout analysis model of the target special font. The target special font is identified using a target special font layout analysis model; The input to the Language Prediction Network (LPN) is the mask set features of the detection and segmentation modules; The output of the Language Prediction Network (LPN) is used to determine which script's recognition head the multiplexing model should select.
2. The method for automatic recognition of special fonts on the Internet according to claim 1, characterized in that, The Language Prediction Network (LPN) consists of two 2×2 convolutional layers with ReLU activation and one 2×2 pooling layer.
3. The method for automatic recognition of special fonts on the Internet according to claim 1, characterized in that, The method for annotating and acquiring target special font layout data using fine-tuning techniques in deep learning, and training a layout analysis model to obtain a target special font layout analysis model includes the following steps: Input manually annotated target special font layout data and preset English layout annotation data, train the layout analysis model to obtain the target layout analysis model; The target layout analysis model is adjusted using a preset initial special font layout model, and the target special font layout data is identified using the adjusted target layout analysis model to obtain the identification results. The target special font layout annotation data is obtained and determined based on the manual review of the recognition results, and the model is trained to obtain the target special font layout analysis model.
4. An internet-based automatic font recognition system, characterized in that, It includes a model building module, a recognition head building module, a loss function determination module, a data annotation and training module, and a font recognition module, among which: The model building module is used to build an initial recognition model based on the detection and segmentation module and the spatial attention module of the Mask TextSpotter V3 recognition model. A new language prediction network (LPN) is added on top of the initial recognition model to obtain a multiplexed model. The recognition head building module is used to build separate recognition heads for multiplexing models to process words from different languages and select or combine them for prediction. The loss function determination module is used to acquire and perform parallel training of the Language Prediction Network (LPN) and the recognition head based on labeled datasets of multiple different languages and target-specific fonts, and to determine the loss function of the multiplexed model. Specifically, it involves parallel training of the LPN and the recognition head while eliminating entanglement loss, i.e., calculating the loss terms for each head and the LPN in parallel, and then directly summing them. ; where L lang It is the loss of LPN, R is the set of recognition heads, and L is the loss of LPN. seq(r) The loss is for recognizing the head r; language prediction is a standard multi-type classification problem, so the language prediction loss in equation (1) can be calculated using cross-entropy loss: Whether the language matches the annotation language is determined by I(l=l) gt The calculation is performed to represent true or false, and p(l) is the probability that the word inferred by LPN belongs to language l; negative log-likelihood is used as the text recognition loss L. seq(r) : ;where p(y t =c t L is the predicted probability of the character at position t in the sequence, where t is the length of the character label sequence; to calculate L seq(r) That is, L supports different recognition heads for different character sets. seq Unsupported characters need to be ignored in the loss calculation: ; where C r It identifies the character set supported by the header r, c t It is the ground truth character of step t, if c is supported. t ,I(c t ∈C r )=1, if c is not supported t Then I(c t ∈C r )=0; The data annotation and training module is used to annotate and acquire target special font layout data using fine-tuning techniques in deep learning, and to train the layout analysis model to obtain the target special font layout analysis model. The font recognition module is used to identify target special fonts using a target special font layout analysis model; The input to the Language Prediction Network (LPN) is the mask set features of the detection and segmentation modules; The output of the Language Prediction Network (LPN) is used to determine which script's recognition head the multiplexing model selects.
5. An electronic device, characterized in that, include: Memory, used to store one or more programs; processor; When the one or more programs are executed by the processor, the method as described in any one of claims 1-3 is implemented.
6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the method as described in any one of claims 1-3.