Open set character recognition method and device, electronic equipment and storage medium

Abstract

The invention relates to an open set character recognition method and device, electronic equipment and a storage medium. The method comprises the steps of performing character stroke analysis on a text image through a character writing sequence model to obtain a stroke image sequence corresponding to the text image; according to a stroke image sequence corresponding to the text image, performing stroke correlation analysis on each single character image set in the stroke image sequence through a writing sequence adjustment model, and adjusting the stroke image sequence to obtain an adjusted stroke image sequence; and according to the adjusted stroke image sequence, performing multi-language character recognition through a classification recognition model to obtain a character recognition result of the text image. According to the method provided by the embodiment of the invention, classification and recognition can be carried out by taking the writing stroke sequence as representation, and the recognition precision in a multi-language complex scene is improved, so that the character recognition accuracy in an open set scene is improved.

Description

Technical Field

[0001] This application relates to the field of character recognition technology, and more specifically, to an open set character recognition method, apparatus, electronic device and storage medium. Background Technology

[0002] Optical Character Recognition (OCR), a key branch of computer vision, initially relied primarily on template matching or statistical feature extraction methods, suitable for recognizing regular fonts such as printed characters. However, it faces significant challenges in recognizing handwritten characters. Due to the large differences in writing styles, connected strokes, and uneven character spacing in handwritten characters, traditional methods struggle to effectively adapt to this diversity, resulting in limited recognition accuracy.

[0003] With the development of deep learning technology, the introduction of Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) has significantly improved the performance of OCR. For example, by combining the feature extraction capabilities of CNNs with the sequence modeling advantages of RNNs, the model can automatically learn the local and global features of an image and capture the contextual dependencies between characters, thereby achieving high-precision recognition of handwritten text on large-scale data. However, such methods typically rely on the assumption of a closed character set, making it difficult to meet the recognition needs of multilingual and multi-symbol systems in open environments.

[0004] In recent years, with the expansion of global application scenarios, open-set character recognition has become an important research direction. Traditional OCR systems have insufficient generalization ability when processing unknown characters outside the training set, especially for minority languages ​​or rare symbols. Existing methods often rely too much on contextual information or language models, while ignoring the visual features of the characters themselves, resulting in a high false recognition rate. Summary of the Invention

[0005] This application provides an open-set character recognition method, apparatus, electronic device, and storage medium. The method of this application can classify and recognize characters by using the stroke order of writing as a representation, improving recognition accuracy in complex multilingual scenarios, thereby enhancing the accuracy of character recognition in open-set scenarios.

[0006] Firstly, this embodiment provides an open set character recognition method, including: For text images, character stroke analysis is performed using a text writing sequence model to obtain the stroke image sequence corresponding to the text image; wherein, the stroke image sequence includes multiple single character image sets arranged in writing order, and each single character image set includes the stroke images corresponding to that single character arranged in writing order; Based on the stroke image sequence corresponding to the text image, the stroke association analysis of each single character image set in the stroke image sequence is performed and the stroke image order is adjusted by the writing sequence adjustment model to obtain the adjusted stroke image sequence. Based on the adjusted stroke image sequence, multilingual character recognition is performed using a classification and recognition model to obtain the character recognition results of the text image.

[0007] In some embodiments, before performing character stroke analysis using a text writing sequence model to obtain the stroke image sequence corresponding to the text image, the method further includes: For the initial image, image preprocessing is performed using an image processing model to obtain the processed initial image; the image preprocessing includes grayscale conversion, denoising, binarization, and perspective correction; The processed initial image is segmented into characters to obtain a text image.

[0008] In some embodiments, it also includes: Based on different languages ​​and writing styles, multiple combinations of language and script styles are obtained; For each combination of language and text styles, corresponding image enhancement parameters are preset to obtain an enhancement parameter mapping table; The image processing model is dynamically adjusted based on the enhanced parameter mapping table.

[0009] In some embodiments, for a text image, character stroke analysis is performed using a text writing sequence model to obtain a stroke image sequence corresponding to the text image, including: Multi-scale convolution is performed on the text image to obtain a two-dimensional feature map of the text image; the two-dimensional feature map includes global and local features of the handwriting of the characters in the text image; Based on the potential paths of the strokes, the text in the two-dimensional feature map is decomposed into a set of stroke images; By using a long short-term memory network to predict the stroke trajectory of a set of stroke images, a sequence of stroke images corresponding to a text image is obtained.

[0010] In some embodiments, based on the stroke image sequence corresponding to the text image, a writing sequence adjustment model is used to perform stroke association analysis on each single character image set in the stroke image sequence to adjust the stroke image order, resulting in an adjusted stroke image sequence, including: Convolutional encoding is performed on each character image set in the stroke image sequence to obtain the local morphology and adjacency relationship features of each stroke in each character; Based on the local shape and adjacency relationship features of each stroke in each character, a bidirectional temporal modeling network is used to perform stroke association analysis to obtain a stroke adjustment scheme for each character. The stroke adjustment scheme includes stroke position, stroke combination, and stroke decomposition. Based on the stroke adjustment scheme for each character, the stroke image sequence corresponding to the text image is adjusted to obtain the adjusted stroke image sequence.

[0011] In some embodiments, based on the adjusted stroke image sequence, multilingual character recognition is performed using a classification and recognition model to obtain the character recognition result of the text image, including: For each single character image set in the adjusted stroke image sequence, a similarity match is performed with the stroke sequence of the standard character in the standard dictionary to obtain the character recognition result of that single character image set; The character recognition results of all single-character image sets are arranged according to the stroke image sequence to obtain the character recognition results of the text image.

[0012] In some embodiments, for each single-character image set in the adjusted stroke image sequence, a similarity match is performed with the stroke sequence of standard characters in a standard dictionary to obtain the character recognition result for that single-character image set, further comprising: If the similarity matching result of any single character image set is lower than the preset threshold, then the single character image set is marked as a character to be confirmed. Before arranging the character recognition results of all single-character image sets according to the stroke image sequence, the method also includes: For each character to be confirmed, context-based auxiliary recognition is performed to obtain the text recognition result of the character to be confirmed.

[0013] Secondly, this embodiment provides an open set character recognition device, including: The analysis module is used to perform character stroke analysis on text images using a text writing sequence model to obtain the stroke image sequence corresponding to the text image; wherein, the stroke image sequence includes multiple single character image sets arranged in writing order, and each single character image set includes the stroke images corresponding to that single character arranged in writing order; The adjustment module is used to perform stroke association analysis on each single character image set in the stroke image sequence according to the stroke image sequence corresponding to the text image, and adjust the stroke image sorting to obtain the adjusted stroke image sequence. The recognition module is used to perform multilingual character recognition based on the adjusted stroke image sequence through a classification recognition model, and obtain the character recognition result of the text image.

[0014] Thirdly, this embodiment provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any one of the first aspects.

[0015] Fourthly, this embodiment provides an electronic device, including a memory and a processor. The memory is used to store computer instructions, and the processor is used to retrieve the computer instructions from the memory to perform the method as described in any one of the first aspects.

[0016] This application's embodiments can analyze the stroke order of characters in a text image and perform classification and recognition by comparing and matching the relationships between strokes. This effectively improves recognition accuracy in complex multilingual scenarios and significantly enhances text recognition accuracy in open-set scenarios.

[0017] Other features and advantages of this application will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments of the present application and, together with their description, serve to explain the principles of the present application.

[0019] Figure 1 A schematic diagram of the open set character recognition method provided in an embodiment of this application is shown.

[0020] Figure 2 A schematic diagram of another open set character recognition method provided in an embodiment of this application is shown.

[0021] Figure 3 A schematic diagram of an open set character recognition device provided in an embodiment of this application is shown.

[0022] Figure 4 A schematic diagram of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0023] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present application.

[0024] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the scope of this application and its application or use.

[0025] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0026] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0027] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0028] Currently, most existing technologies focus on mainstream languages ​​such as Chinese and English, and still have limitations in recognizing unknown characters in multilingual mixed scenarios.

[0029] In related technologies, weakly supervised character masking methods are typically used to mask the context regions in word-level images, forcing the recognition network to rely solely on the visual features of the characters themselves for discrimination. Simultaneously, by constructing synthetic single-character samples and performing single-character classification tasks, the model strengthens its character-level discriminative representation even without context. While this approach removes the influence of context on open-set OCR, relying solely on character shape features for template matching, its over-reliance on the accuracy of the character mask can lead to insufficient accuracy in weakly supervised mask generation during the initial training phase or in complex scenarios. This can cause context isolation to fail, interfering with character feature learning and thus affecting recognition robustness. Furthermore, while isolating contextual influences, easily confused or similar-looking characters cannot be accurately identified using only visual features, resulting in a high error rate.

[0030] Therefore, providing a solution that can dynamically adapt to changes in character sets while taking into account both visual details and semantic context is of great practical significance.

[0031] Based on the above analysis, such as Figure 1 As shown, this application provides an open set character recognition method, including: Step S101: For the text image, character stroke analysis is performed using a text writing sequence model to obtain the stroke image sequence corresponding to the text image; wherein, the stroke image sequence includes multiple single character image sets arranged in writing order, and each single character image set includes the stroke images corresponding to that single character arranged in writing order.

[0032] Specifically, this step mainly involves simulating the rewriting of characters in the text image to break down the characters into a continuous stroke sequence. Then, the stroke images corresponding to each stroke are arranged according to the stroke sequence and stored in units of characters to obtain a stroke image sequence.

[0033] The text writing sequence model uses a convolutional neural network as the feature extraction backbone. To cope with various complex open-set scenarios and adapt to different languages ​​and writing systems, Step S102: Based on the stroke image sequence corresponding to the text image, perform stroke association analysis on each single character image set in the stroke image sequence through the writing sequence adjustment model and adjust the stroke image sorting to obtain the adjusted stroke image sequence.

[0034] Specifically, for different writing samples of the same character, the decomposed strokes may be the same, but the writing order may be different. This application embodiment learns the spatial and temporal features of the input sample image sequence to realize the sorting adjustment of stroke images and the automatic mapping of stroke combination or decomposition operations. The purpose is to standardize the stroke image sequence decomposed in step S101 so as to better perform character recognition through stroke sequence matching.

[0035] Step S103: Based on the adjusted stroke image sequence, multilingual character recognition is performed using a classification and recognition model to obtain the character recognition result of the text image.

[0036] Specifically, to adapt to multilingual character recognition scenarios, this application embodiment pre-constructs a standard dictionary for the classification and recognition model. In the standard dictionary, all characters are stored in the form of standard handwritten stroke image sequences. The standard dictionary may include multiple languages ​​and scripts, as well as mathematical symbols, chemical structural formulas, logical symbols, musical scores, etc.

[0037] In some embodiments, such as Figure 2 As shown, before performing character stroke analysis using a text writing sequence model to obtain the stroke image sequence corresponding to the text image, the following steps are also included: Step S104: For the initial image, perform image preprocessing using an image processing model to obtain the processed initial image; wherein, image preprocessing includes grayscale conversion, denoising, binarization, and perspective correction; Step S105: Perform character segmentation on the processed initial image to obtain a text image.

[0038] Specifically, before performing character stroke analysis on the text image, the embodiments of this application preprocess the acquired initial image by using mean filtering, Gaussian filtering and wavelet denoising methods to perform image processing such as grayscale conversion, denoising, binarization and perspective correction on the initial image to ensure the clarity of the stroke edges and details.

[0039] Next, based on the preprocessed initial image and the characters in the image, the image is segmented into continuous character images, which are then arranged in order to form a text image.

[0040] In some embodiments, it also includes: Based on different languages ​​and writing styles, multiple combinations of language and script styles are obtained; For each combination of language and text styles, corresponding image enhancement parameters are preset to obtain an enhancement parameter mapping table; The image processing model is dynamically adjusted based on the enhanced parameter mapping table.

[0041] Specifically, the open set character recognition method provided in this application provides image enhancement parameters for different combinations of languages ​​and writing styles in order to adapt to different languages ​​and writing styles. The image enhancement parameters include the stretching, rotation and other operations and magnitudes of the image. By configuring the image enhancement parameters, the preprocessed text can be more easily recognized.

[0042] If an initial image contains multiple language styles, the image enhancement parameters for each language style combination will be called simultaneously to comprehensively adjust the image processing model.

[0043] For example, if an initial image contains multiple language style combinations, the average value of each parameter in the image enhancement parameters of all language style combinations can be used as the comprehensive enhancement parameter.

[0044] In some embodiments, for a text image, character stroke analysis is performed using a text writing sequence model to obtain a stroke image sequence corresponding to the text image, including: Multi-scale convolution is performed on the text image to obtain a two-dimensional feature map of the text image; the two-dimensional feature map includes global and local features of the handwriting of the characters in the text image; Based on the potential paths of the strokes, the text in the two-dimensional feature map is decomposed into a set of stroke images; By using a long short-term memory network to predict the stroke trajectory of a set of stroke images, a sequence of stroke images corresponding to a text image is obtained.

[0045] Specifically, an attention mechanism is incorporated into the text writing sequence model. By weighted integration of features from different regions in the character image, the text writing sequence model can generate potential stroke paths by capturing long-distance dependencies and continuity features between strokes.

[0046] The global and local features of the handwriting in the text image are extracted to form a two-dimensional feature map. Then, the potential path of the generated strokes is determined by the text writing sequence model, and the two-dimensional feature map is expanded into a stroke image set. Finally, the serialization information such as the start point, end point, trajectory direction and length of the strokes in the stroke image set is predicted by the Long Short-Term Memory Network and added to the image information to obtain the stroke image sequence corresponding to the text image.

[0047] In particular, during the training phase of the text writing sequence model, unsupervised reconstruction loss and structural consistency constraints were adopted, and an angle loss function was designed to simulate the angle of human writing, making the stroke path rewritten by the model smoother. A stroke number loss function was also designed to constrain the number of strokes, guide the model to generate a stroke image sequence that is consistent with the shape of the original image, and maintain the rationality of the writing order.

[0048] This stroke image sequence not only preserves the visual morphological features of the characters, but also preserves the spatial positional relationship between the strokes in the characters, thus realizing the stroke image sequence as a character representation.

[0049] In some embodiments, based on the stroke image sequence corresponding to the text image, a writing sequence adjustment model is used to perform stroke association analysis on each single character image set in the stroke image sequence to adjust the stroke image order, resulting in an adjusted stroke image sequence, including: Convolutional encoding is performed on each character image set in the stroke image sequence to obtain the local morphology and adjacency relationship features of each stroke in each character; Based on the local shape and adjacency relationship features of each stroke in each character, a bidirectional temporal modeling network is used to perform stroke association analysis to obtain a stroke adjustment scheme for each character. The stroke adjustment scheme includes stroke position, stroke combination, and stroke decomposition. Based on the stroke adjustment scheme for each character, the stroke image sequence corresponding to the text image is adjusted to obtain the adjusted stroke image sequence.

[0050] Specifically, for different writing styles of the same character, the number of strokes and the stroke order obtained by the character writing sequence model are different. In order to match more accurately in subsequent classification and recognition, the stroke image sequence obtained after the character writing sequence model is decomposed needs to be standardized.

[0051] This application embodiment designs a writing sequence adjustment model, which performs convolutional encoding on the stroke image sequence to extract the local morphology and adjacency relationship features of each stroke, wherein the adjacency relationship features include the continuous relationship between strokes of a single character.

[0052] The system integrates global stroke dependencies through a bidirectional temporal modeling network. It uses an attention mechanism to identify whether the positional relationship between adjacent strokes in each character image set conforms to the potential stroke path. If not, it generates a rearrangement scheme according to the potential stroke path. It also identifies whether adjacent strokes in each character image set have continuous trajectories and consistent trajectories. If two adjacent strokes that meet the conditions are identified, a stroke combination scheme is generated. Finally, it identifies whether there are continuity problems such as excessively long stroke trajectories or too many directional changes in each character image set. If strokes with continuity problems are identified, a stroke decomposition scheme is generated.

[0053] The above-mentioned order rearrangement scheme, stroke combination scheme, and stroke decomposition scheme are integrated into a stroke adjustment scheme. The stroke adjustment scheme is then applied to the learnable mapping function on a character-by-character basis to perform standardization adjustment of the character image set. This ensures that the stroke image sequence adjusted by the writing sequence adjustment model achieves a standardized order, and that the stroke length, stroke direction, and stroke spacing of each character in the stroke image sequence remain consistent.

[0054] In particular, during the training of the writing sequence adjustment model, self-supervised reconstruction loss and sequence consistency constraints were introduced. This enabled the writing sequence adjustment model to autonomously adjust stroke image sequences with different writing styles and stroke granularities into a consistent standard stroke image sequence without relying on manual annotation. This provides standardized data input for subsequent classification and recognition, thereby improving the recognition accuracy and adaptability to multilingual open set environments.

[0055] In some embodiments, based on the adjusted stroke image sequence, multilingual character recognition is performed using a classification and recognition model to obtain the character recognition result of the text image, including: For each single character image set in the adjusted stroke image sequence, a similarity match is performed with the stroke sequence of the standard character in the standard dictionary to obtain the character recognition result of that single character image set; The character recognition results of all single-character image sets are arranged according to the stroke image sequence to obtain the character recognition results of the text image.

[0056] Specifically, for the standardized stroke image sequence, the classification and recognition model performs similarity matching between the single character image set and the standard characters in the standard dictionary based on the stroke sequence, selects the standard character with the highest similarity matching degree, and determines whether the similarity matching degree reaches the preset matching degree threshold. If it does, the character corresponding to the single image set is determined to be the standard character.

[0057] After all the single-character image sets have undergone similarity matching by the classification and recognition model, the characters corresponding to all the single-character image sets are concatenated in sequence to obtain the text recognition result of the text image.

[0058] Furthermore, during the similarity matching process, the classification and recognition model comprehensively considers information such as the global shape, local shape, writing order, and structure of each stroke, ensuring that even when there are differences in writing style, multilingual characters, or slight deformation of strokes after image enhancement, the classification and recognition model can still accurately perform similarity matching and output the corresponding characters.

[0059] The aforementioned standard dictionary is a pre-built language knowledge base, storing standard characters and their corresponding stroke orders according to the existing language system. Each entry contains the standard stroke order of the character and its corresponding category label. In particular, before classification and recognition, data from a portion of the language knowledge base can be selected to construct the standard dictionary through a language system judgment model or manual input. This reduces the workload of similarity matching in the classification and recognition model, thereby improving recognition efficiency and accuracy.

[0060] In some embodiments, for each single-character image set in the adjusted stroke image sequence, a similarity match is performed with the stroke sequence of standard characters in a standard dictionary to obtain the character recognition result for that single-character image set, further comprising: If the similarity matching result of any single character image set is lower than the preset threshold, then the single character image set is marked as a character to be confirmed. Before arranging the character recognition results of all single-character image sets according to the stroke image sequence, the method also includes: For each character to be confirmed, context-based auxiliary recognition is performed to obtain the text recognition result of the character to be confirmed.

[0061] Specifically, in order to reduce the false recognition rate of open set recognition, when the similarity matching degree between a single character image set and the standard characters in the standard dictionary does not reach the preset matching degree threshold, the rejection mechanism is triggered, the single character image set is marked as a character to be confirmed, and the recognition of the single character image set is skipped, so as to ensure the smooth flow of the overall recognition sequence.

[0062] After recognizing all the single-character image sets, the character to be confirmed can be identified with the help of context to estimate the corresponding character, or it can be edited by manual input. This application's embodiments can analyze the stroke order of characters in a text image and perform classification and recognition by comparing and matching the relationships between strokes. This effectively improves recognition accuracy in complex multilingual scenarios and significantly enhances text recognition accuracy in open-set scenarios.

[0063] This application also provides an open set character recognition device. For example... Figure 3 As shown, the open set character recognition device 300 includes: Analysis module 301 is used to perform character stroke analysis on text images using a text writing sequence model to obtain a stroke image sequence corresponding to the text image; wherein, the stroke image sequence includes multiple single character image sets arranged in writing order, and each single character image set includes the stroke images corresponding to that single character arranged in writing order; The adjustment module 302 is used to perform stroke association analysis on each single character image set in the stroke image sequence according to the stroke image sequence corresponding to the text image, and adjust the stroke image sorting to obtain the adjusted stroke image sequence. The recognition module 303 is used to perform multilingual character recognition based on the adjusted stroke image sequence through a classification recognition model to obtain the character recognition result of the text image.

[0064] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0065] This application also provides an electronic device, such as... Figure 4 As shown, the electronic device 400 includes a memory 401 and a processor 402. The memory 401 is used to store computer instructions, and the processor 402 is used to retrieve the computer instructions from the memory 401 to execute the method as described in any of the above method embodiments.

[0066] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the method as described in any of the above method embodiments.

[0067] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device and apparatus embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0068] The foregoing has described specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0069] This application may be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this application.

[0070] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0071] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0072] The computer program instructions used to perform the operations of this application may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuits, such as programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), are personalized by utilizing the status information of the computer-readable program instructions. These electronic circuits can execute the computer-readable program instructions to implement various aspects of this application.

[0073] Various aspects of this application are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0074] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0075] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0076] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It will be well known to those skilled in the art that implementation in hardware, implementation in software, and implementation using a combination of software and hardware are equivalent.

[0077] The various embodiments of this application have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of this application is defined by the appended claims.

Claims

1. An open set method of recognizing text, characterized by, Comprise: For a text image, character stroke analysis is performed on the text image by a character writing sequence model to obtain a stroke image sequence corresponding to the text image; wherein the stroke image sequence comprises a plurality of single character image sets arranged in writing order, and each single character image set comprises stroke images corresponding to the single character arranged in writing order; According to the stroke image sequence corresponding to the text image, stroke association analysis is performed on each single character image set in the stroke image sequence by a writing sequence adjustment model, and the stroke image sequence is adjusted to obtain an adjusted stroke image sequence; According to the adjusted stroke image sequence, multi-language character recognition is performed by a classification recognition model to obtain a character recognition result of the text image.

2. The method of claim 1, wherein, Before the character stroke analysis is performed on the text image by the character writing sequence model to obtain the stroke image sequence corresponding to the text image, it further comprises: For an initial image, image preprocessing is performed on the initial image by an image processing model to obtain a processed initial image; wherein the image preprocessing comprises grayscale, denoising, binarization and perspective correction; Text image is obtained by character segmentation on the processed initial image.

3. The method of claim 2, wherein, Further comprise: According to different languages and writing styles, a plurality of language and style combinations are obtained; For each language and style combination, a corresponding image enhancement parameter is preset to obtain an enhancement parameter mapping table; According to the enhancement parameter mapping table, the image processing model is dynamically adjusted.

4. The method of claim 1, wherein, The character stroke analysis is performed on the text image by the character writing sequence model to obtain the stroke image sequence corresponding to the text image, comprising: Multi-scale convolution is performed on the text image to obtain a two-dimensional feature map of the text image; wherein the two-dimensional feature map comprises global features and local features of character handwriting in the text image; According to the stroke latent path, the text in the two-dimensional feature map is decomposed into a stroke image set; Stroke trajectory prediction is performed on the stroke image set by a long short-term memory network to obtain the stroke image sequence corresponding to the text image.

5. The method of claim 1, wherein, The stroke association analysis is performed on each single character image set in the stroke image sequence by the writing sequence adjustment model according to the stroke image sequence corresponding to the text image to adjust the stroke image sequence, comprising: Convolution coding is performed on each single character image set in the stroke image sequence to obtain local morphology and adjacency relationship features of each stroke in each single character; According to the local morphology and adjacency relationship features of each stroke in each single character, stroke association analysis is performed by a bidirectional time series modeling network to obtain a stroke adjustment scheme of each single character; wherein the stroke adjustment operation in the stroke adjustment scheme comprises stroke position, stroke combination and stroke decomposition; According to the stroke adjustment scheme of each single character, the stroke image sequence corresponding to the text image is adjusted to obtain an adjusted stroke image sequence.

6. The method of claim 1, wherein, The multi-language character recognition is performed by the classification recognition model according to the adjusted stroke image sequence to obtain the character recognition result of the text image, comprising: For each single character image set in the adjusted stroke image sequence, similarity matching is performed with the stroke sequence of the standard character in the standard dictionary, to obtain the character recognition result of the single character image set; The character recognition results of all single character image sets are arranged according to the stroke image sequence, to obtain the character recognition result of the text image.

7. The method of claim 6, wherein, The similarity matching of the stroke sequence of the standard character in the standard dictionary for each single character image set in the adjusted stroke image sequence, to obtain the character recognition result of the single character image set, further includes: When the similarity matching result of any single character image set is lower than a preset threshold, the single character image set is marked as a to-be-confirmed character; Before arranging the character recognition results of all single character image sets according to the stroke image sequence, the method further includes: For each to-be-confirmed character, auxiliary recognition is performed based on context, to obtain the character recognition result of the to-be-confirmed character.

8. An open set character recognition apparatus, characterized by, Comprise: An analysis module is configured to perform stroke analysis on a text image by a character writing sequence model, to obtain a stroke image sequence corresponding to the text image; wherein the stroke image sequence comprises a plurality of single character image sets arranged in writing order, and each single character image set comprises stroke images corresponding to the single character and arranged in writing order; An adjustment module is configured to perform stroke correlation analysis on each single character image set in the stroke image sequence and adjust the stroke image sequence according to the stroke image sequence corresponding to the text image by a writing sequence adjustment model, to obtain an adjusted stroke image sequence; An identification module is configured to perform multi-language character recognition by a classification recognition model according to the adjusted stroke image sequence, to obtain a character recognition result of the text image.

9. An electronic device, comprising: comprise a memory and a processor, The memory is configured to store computer instructions, and the processor is configured to call the computer instructions from the memory to execute the method of any one of claims 1 to 7.

10. A computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the method of any one of claims 1 to 7.

Images