A hieroglyphic recognition method, device, equipment and medium

Abstract

A pictograph recognition method, device, equipment and medium are disclosed. The method comprises: inputting a character picture into a recognition model; performing feature extraction on the character picture through at least two feature extraction units connected in series of the recognition model to obtain target features; wherein the feature extraction unit comprises a mixed perception mapping unit and / or a spatial perception aggregation unit; the mixed perception mapping unit is used to obtain global effective features and global key position features; the spatial perception aggregation unit is used to obtain local key detail features; different feature extraction units are used to extract features of different sizes; and the target features are classified through a classification layer of the recognition model to determine a character recognition result corresponding to the character picture. Through the embodiment of the present application, the richness of the feature extraction result can be improved after the feature extraction on the character picture, thereby improving the accuracy of the recognition model in recognizing the character picture.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a method, apparatus, device, and medium for recognizing pictographic characters. Background Technology

[0002] Pictographs have obvious pictorial characteristics, and there are highly similar pictographs that are difficult to distinguish, making them more difficult to recognize than other character recognition tasks.

[0003] Most commonly used deep learning recognition models improve the model from the depth or width of the network, which greatly enhances the network's ability to extract semantic features. However, when dealing with the recognition task of pictographic characters with high similarity, the recognition rate is low. Summary of the Invention

[0004] This invention provides a method, apparatus, device, and medium for pictographic character recognition, which improves the richness of features extracted from text images and enhances the accuracy of character recognition models.

[0005] According to one aspect of the present invention, a method for recognizing pictographs is provided, the method comprising:

[0006] Input the text image into the recognition model;

[0007] The text image is processed by at least two concatenated feature extraction units of the recognition model to obtain target features; wherein, the feature extraction unit includes a hybrid sensing mapping unit and / or a spatial sensing aggregation unit; the hybrid sensing mapping unit is used to obtain global effective features and global key location features; the spatial sensing aggregation unit is used to obtain local key detail features; different feature extraction units are used to extract features of different sizes;

[0008] The target features are classified using the classification layer of the recognition model to determine the text recognition result corresponding to the text image.

[0009] According to another aspect of the present invention, a pictographic character recognition device is provided, the device comprising:

[0010] The input module is used to input text images into the recognition model;

[0011] The target feature acquisition module is used to extract features from text images through at least two concatenated feature extraction units of the recognition model to obtain target features; wherein, the feature extraction unit includes a hybrid perception mapping unit and / or a spatial perception aggregation unit; the hybrid perception mapping unit is used to acquire local effective features and local key detail features; the spatial perception aggregation unit is used to acquire global key location features; different feature extraction units are used to extract features of different sizes;

[0012] The result determination module is used to classify the target features through the classification layer of the recognition model and determine the text recognition result corresponding to the text image.

[0013] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0014] At least one processor; and

[0015] A memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the pictographic character recognition method according to any embodiment of the present invention.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the pictographic character recognition method according to any embodiment of the present invention.

[0018] According to another aspect of the present invention, a computer program product is provided, the computer program product comprising a computer program that, when executed by a processor, implements the pictographic character recognition method according to any embodiment of the present invention.

[0019] The technical solution of this invention involves inputting a text image into a recognition model. At least two feature extraction units connected in series within the recognition model extract features from the text image to obtain target features. Different feature extraction units are used to extract features of different sizes, enabling the acquisition of text image features at different sizes and increasing the richness of the extracted text image features. Simultaneously, the feature extraction units include a hybrid perception mapping unit and / or a spatial perception aggregation unit. The hybrid perception mapping unit is used to acquire global effective features and global key position features, while the spatial perception aggregation unit is used to acquire local key detail features. This allows the target features to focus on both global effective features and global key position features, while also paying attention to local key detail position features. This enriches feature information from both global and local detail perspectives, improving feature richness. Compared to global key position features and global effective features, local key detail features retain more detailed information. This allows for the effective extraction of detail features from images where the text outlines are nearly identical but there are minor differences in the details of the character shapes. Therefore, when classifying the target features through the classification layer of the recognition model to determine the text recognition result corresponding to the text image, the accuracy of text recognition can be improved.

[0020] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1a This is a flowchart of a pictographic character recognition method provided in Embodiment 1 of the present invention;

[0023] Figure 1b This is a schematic diagram of the Dongba pictographic script and its corresponding character recognition results according to Embodiment 1 of the present invention;

[0024] Figure 2 This is a flowchart of a pictographic character recognition method provided in Embodiment 2 of the present invention;

[0025] Figure 3a This is a flowchart of a pictographic character recognition method provided in Embodiment 3 of the present invention;

[0026] Figure 3b This is a flowchart of a pictographic character recognition method provided in Embodiment 3 of the present invention;

[0027] Figure 4a This is a flowchart of a pictographic character recognition method provided in Embodiment 4 of the present invention;

[0028] Figure 4b This is a flowchart of a pictographic character recognition method provided in Embodiment 4 of the present invention;

[0029] Figure 5 This is a schematic diagram illustrating an application scenario of a pictographic character recognition method provided in Embodiment 5 of the present invention;

[0030] Figure 6 This is a schematic diagram of the structure of a pictographic character recognition device according to Embodiment Six of the present invention;

[0031] Figure 7 This is a schematic diagram of the structure of an electronic device that implements the pictographic character recognition method of this invention. Detailed Implementation

[0032] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0033] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the term "comprising" and any variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0034] Example 1

[0035] Figure 1a This is a flowchart of a pictographic character recognition method provided in Embodiment 1 of the present invention. This embodiment is applicable to the recognition of pictographic characters. The method can be executed by a pictographic character recognition device, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1a As shown, the method includes:

[0036] S110. Input the text image into the recognition model.

[0037] A text image refers to an image containing text content. In this application, the text in the text image is hieroglyphic; for example, the text in the text image may be Dongba hieroglyphs, Egyptian hieroglyphs, or oracle bone script. A recognition model refers to a model that identifies the text in the text image to obtain the corresponding modern standard script. The recognition model can be a network model constructed using deep learning technology.

[0038] S120. The text image is subjected to feature extraction through at least two concatenated feature extraction units of the recognition model to obtain target features; wherein, the feature extraction unit includes a hybrid perception mapping unit and / or a spatial perception aggregation unit; the hybrid perception mapping unit is used to obtain global effective features and global key location features; the spatial perception aggregation unit is used to obtain local key detail features; different feature extraction units are used to extract features of different sizes.

[0039] The feature extraction unit is used to extract image features from text images. The recognition model includes at least two feature extraction units, which are connected in series, allowing the text image to pass through at least two feature extraction units sequentially for feature extraction. The feature extraction unit includes at least one of a hybrid sensing mapping unit and a spatial sensing aggregation unit. The target feature refers to the result obtained after feature extraction from the text image. Different feature extraction units extract image features of different sizes.

[0040] A hybrid perceptual mapping unit (HPC) is a unit that extracts features from text images from different dimensions. For example, an HPC can extract features from text images from both channel and spatial dimensions. The channel dimension allows for the acquisition of foreground or background features, while the spatial dimension allows for the acquisition of features representing the location of the text. Specifically, the HPC is used to acquire globally effective features and globally key location features. Globally effective features refer to foreground features in the text image. Generally, for text images, foreground features are features related to the text and represent the effective information that needs attention, while background features are other features unrelated to the text and represent ineffective information that can be ignored. Globally key location features refer to the features representing the location of the text in the text image. Generally, besides the location of the text, there are also locations where the text does not exist in the text image. For pictographic character recognition methods, the key location features are those representing the locations where the text is located.

[0041] A spatially aware aggregation unit (SAU) is a unit that aggregates input and extracts features from the aggregation results. Specifically, SAU is used to acquire local key detail features. Local key detail features refer to the features obtained after aggregating the input and extracting features from the aggregation results. Compared to global key location features, which focus on the overall positional features of the text, local key detail features, in addition to global key location features, focus on the positional features of each component of the text, making the information in local key detail features richer. Specifically, compared to global key location features, local key detail features retain more detailed information in the text image, making the information in local key detail features richer.

[0042] Specifically, by using at least two feature extraction units connected in series in the recognition model, features are extracted from the text image sequentially to obtain the target features.

[0043] S130. The target features are classified through the classification layer of the recognition model to determine the text recognition result corresponding to the text image.

[0044] The classification layer is used to classify target features to determine the semantics of the modern characters corresponding to the text in the text image. In this application, the classification layer includes a global average pooling layer and a fully connected layer. The text recognition result refers to the semantics of the modern characters corresponding to the text in the text image. Specifically, the text recognition result can be described in Chinese, English, or other modern languages. For example, the text recognition result for the first text image is "water tank," and the text recognition result for the second text image is "foot of a mountain," etc.

[0045] Specifically, by using the classification layer of the recognition model, the target features are sequentially passed through a global average pooling layer and a fully connected layer to achieve the classification of the target features and determine the text recognition result corresponding to the text image. Reducing the dimensionality of the target features through the global average pooling layer before inputting them into the fully connected layer for classification reduces the computational load of the classification layer and improves the computational speed.

[0046] The technical solution of this invention involves inputting a text image into a recognition model. At least two feature extraction units connected in series within the recognition model extract features from the text image to obtain target features. Different feature extraction units are used to extract features of different sizes, enabling the acquisition of text image features at different sizes and increasing the richness of the extracted text image features. Simultaneously, the feature extraction units include a hybrid perception mapping unit and / or a spatial perception aggregation unit. The hybrid perception mapping unit is used to acquire global effective features and global key position features, while the spatial perception aggregation unit is used to acquire local key detail features. This allows the target features to focus on both global effective features and global key position features while also paying attention to local key detail features, thus improving feature richness. Local key detail features retain more detailed information, effectively extracting detail features from images where the text outlines are nearly identical but there are slight differences in the details of the character shapes. Therefore, when classifying the target features through the classification layer of the recognition model to determine the text recognition result corresponding to the text image, the accuracy of text recognition can be improved.

[0047] Optionally, the text image is an image of Dongba pictographs, and a text image includes at least one Dongba pictograph.

[0048] Among them, Dongba pictographic script is a primitive pictographic writing system mainly taught and used by the Naxi people to write Dongba scriptures; it is also called Dongba script. A single image of the text contains at least one Dongba pictographic character. Figure 1b This is a schematic diagram illustrating the Dongba pictographic script and its corresponding character recognition results provided in Embodiment 1 of the present invention. For example... Figure 1b As shown, the first row contains eight images of eight Dongba pictographic characters. After recognition using the pictographic character recognition method, the resulting characters are the corresponding modern standard characters below each image in the second row. The third row contains eight images of eight Dongba pictographic characters. After recognition using the pictographic character recognition method, the resulting characters are the corresponding modern standard characters below each image in the fourth row.

[0049] like Figure 1b As shown, the Dongba pictographs in the images are grouped in pairs from left to right. The outlines of the characters in each pair are very similar, with only minor differences in line curvature and length. Handwriting can easily reduce the differences between the two Dongba pictographs in each pair, increasing the difficulty of recognizing highly similar characters and leading to a decrease in accuracy. Therefore, it is necessary to extract a large amount of local detail information to determine the subtle differences in the Dongba pictographs. Current deep learning recognition models mostly improve the model at the depth or width level of the network, significantly enhancing its ability to extract semantic features. However, when dealing with the recognition task of highly similar Dongba pictographs, they still face the problem of insufficient detail feature extraction, resulting in a low recognition rate.

[0050] By inputting an image of Dongba pictographs into the recognition model, features can be extracted from the image through at least two concatenated feature extraction units. Different feature extraction units are used to extract features of different sizes, thus acquiring features of the image at different dimensions and increasing the richness of the extracted features. Simultaneously, the feature extraction units include a hybrid perceptual mapping unit and / or a spatial perceptual aggregation unit. The hybrid perceptual mapping unit is used to acquire global effective features and global key location features, while the spatial perceptual aggregation unit is used to acquire local key detail features. This allows the target features to focus on both global effective features and global key location features, while also paying attention to local key detail features, improving feature richness. Local key detail features retain more detailed information, improving the acquisition of detailed features even when the character outlines are nearly identical, with only minor differences in the details of the character shape. Therefore, when classifying the target features through the classification layer of the recognition model to determine the corresponding character recognition result, the accuracy of character recognition can be improved.

[0051] Example 2

[0052] Figure 2 This is a flowchart of a pictographic character recognition method provided in Embodiment 2 of the present invention. Based on the above embodiments, this embodiment refines the feature extraction of the text image by at least two concatenated feature extraction units of the recognition model to obtain target features. For example... Figure 2 As shown, the method includes:

[0053] S210. Input the text image into the recognition model, wherein the recognition model includes at least two feature extraction units connected in series; the first feature extraction unit of the at least two feature extraction units includes a hybrid sensing mapping unit, and the other feature extraction units of the at least two feature extraction units include a hybrid sensing mapping unit and a spatial sensing aggregation unit.

[0054] The recognition model includes concatenated feature extraction units, which are divided into two categories: the first feature extraction unit and the remaining feature extraction units.

[0055] S220. The text image is subjected to feature extraction by the hybrid perception mapping unit in the first feature extraction unit to obtain the intermediate global features of the first feature extraction unit.

[0056] The first feature extraction unit refers to the feature extraction unit in the recognition model that directly extracts features from the text image. The first feature extraction unit includes at least one cascaded hybrid sensing mapping unit. Specifically, the first feature extraction unit includes at least one hybrid sensing mapping unit, and these hybrid sensing mapping units are cascaded. Other feature extraction units refer to the feature extraction units in the recognition model other than the first feature extraction unit.

[0057] Other feature extraction units include a spatial sensing aggregation unit and at least one cascaded hybrid sensing mapping unit, wherein the spatial sensing aggregation unit and the cascaded hybrid sensing mapping unit are connected in parallel. For example, the recognition model includes three feature extraction units: a first feature extraction unit, a second feature extraction unit, and a third feature extraction unit. The first feature extraction unit is the primary feature extraction unit, and the second and third feature extraction units are the other feature extraction units. Specifically, the number of hybrid sensing mapping units in the other feature extraction units is at least one, and these hybrid sensing mapping units are connected in series.

[0058] Intermediate global features refer to the features obtained after feature extraction by at least one hybrid sensing mapping unit connected in series in the feature extraction unit. Specifically, they are the features output by the last hybrid sensing mapping unit in the series of hybrid sensing mapping units.

[0059] Specifically, for the first feature extraction unit, intermediate global features are calculated through the hybrid sensing mapping unit in the first feature extraction unit. The input of the hybrid sensing mapping unit in the first feature extraction unit is the text image, and the output is the intermediate global features of the first feature extraction unit.

[0060] S230. The intermediate global features of the previous feature extraction unit are extracted by the hybrid sensing mapping unit in the other feature extraction unit to obtain the intermediate global features of the other feature extraction unit.

[0061] The preceding feature extraction unit in a concatenation refers to the feature extraction unit preceding the current feature extraction unit. The output of this preceding feature extraction unit serves as the input to the current feature extraction unit, and the output of the current feature extraction unit serves as the input to the following feature extraction unit. Specifically, for each other feature extraction unit, the intermediate global features of those other feature extraction units are calculated through the hybrid sensing mapping unit within each other feature extraction unit. The input of the hybrid sensing mapping unit in each other feature extraction unit is the intermediate global feature of the preceding feature extraction unit, and the output is the intermediate global feature of that feature extraction unit.

[0062] For example, the recognition model includes three feature extraction units: a first feature extraction unit, a second feature extraction unit, and a third feature extraction unit. The first feature extraction unit is the primary feature extraction unit, while the second and third feature extraction units are the other feature extraction units. When the second feature extraction unit calculates intermediate global features, the input of the hybrid sensing mapping unit in the second feature extraction unit is the intermediate global features of the first feature extraction unit, and the output is the intermediate global features of the second feature extraction unit.

[0063] S240. The intermediate global features of the first feature extraction unit are extracted by the spatial perception aggregation unit in the first other feature extraction unit to obtain the intermediate local features of the first other feature extraction unit.

[0064] The other feature extraction units in the series are divided into two categories: the first other feature extraction unit and the remaining feature extraction units excluding the first other feature extraction unit. Furthermore, among the other feature extraction units excluding the first other feature extraction unit, the last other feature extraction unit is further distinguished as the final other feature extraction unit.

[0065] The first feature extraction unit refers to the first feature extraction unit among the other feature extraction units. Intermediate local features refer to the features obtained by the spatial awareness aggregation unit among the other feature extraction units after extracting features from the input.

[0066] Specifically, for the first other feature extraction unit, intermediate local features are calculated through the spatial perception aggregation unit in the first other feature extraction unit. The input of the spatial perception aggregation unit in the first other feature extraction unit is the intermediate global feature of the first feature extraction unit, and the output is the intermediate local feature of the first other feature extraction unit.

[0067] S250. The intermediate local features of the first other feature extraction unit are fused with the intermediate global features of the first other feature extraction unit to obtain the intermediate fused features of the first other feature extraction unit.

[0068] Fusion refers to the process of combining intermediate local features from other feature extraction units with intermediate global features from other feature extraction units into a single feature. Specifically, fusion can be achieved by adding, multiplying, or weighting the intermediate local features from other feature extraction units with the intermediate global features from other feature extraction units. The intermediate fused feature is the feature obtained after fusing the intermediate global features from other feature extraction units with the intermediate local features from other feature extraction units.

[0069] By fusing intermediate local features and intermediate global features, both detailed and overall information can be added to the feature information. The fused features contain both detailed and overall information, thereby improving the richness of the fused features.

[0070] Specifically, the intermediate local features of the first other feature extraction unit are fused with the intermediate global features of the first other feature extraction unit, and the fusion result is determined as the intermediate fused feature of the first other feature extraction unit.

[0071] S260. Through the spatial perception aggregation unit in the other feature extraction units besides the first other feature extraction unit, the intermediate fusion features of the previous feature extraction unit are extracted to obtain the intermediate local features of the other feature extraction units besides the first other feature extraction unit.

[0072] "Other feature extraction units besides the first other feature extraction unit" refers to feature extraction units other than the first other feature extraction unit among the other feature extraction units. For example, the recognition module includes four feature extraction units: a first feature extraction unit, a second feature extraction unit, a third feature extraction unit, and a fourth feature extraction unit. The first feature extraction unit is the first feature extraction unit, the second feature extraction unit is the first other feature extraction unit, and the third and fourth feature extraction units are other feature extraction units besides the first other feature extraction unit.

[0073] Specifically, for feature extraction units other than the first feature extraction unit, intermediate local features are calculated through spatial awareness aggregation units in these feature extraction units. The input of the spatial awareness aggregation units in these feature extraction units is the intermediate fused features of the previous feature extraction unit, and the output is the intermediate local features of these feature extraction units.

[0074] S270. The intermediate local features of the feature extraction units other than the first feature extraction unit are fused with the intermediate global features of the feature extraction units other than the first feature extraction unit to obtain the intermediate fused features of the feature extraction units other than the first feature extraction unit.

[0075] Specifically, the intermediate local features of all feature extraction units other than the first feature extraction unit are fused with the intermediate global features of all feature extraction units other than the first feature extraction unit, and the fusion result is determined as the intermediate fused feature of all feature extraction units other than the first feature extraction unit.

[0076] S280. The intermediate fusion features of the last other feature extraction unit are determined as the target features.

[0077] The last other feature extraction unit refers to the last feature extraction unit among at least two feature extraction units connected in series in the recognition model. For example, the recognition model includes three feature extraction units, namely the first feature extraction unit, the second feature extraction unit, and the third feature extraction unit, with the third feature extraction unit being the last feature extraction unit.

[0078] Specifically, the intermediate fusion features of the last other feature extraction unit are directly determined as the target feature. The spatial perception aggregation unit in each other feature extraction unit extracts detailed features from the previous fusion result and fuses the global features of the hybrid perception mapping unit in the same other feature extraction unit, realizing the fusion of local and global features of the same size layer by layer. Finally, the intermediate fusion features of the last other feature extraction unit are directly determined as the target feature, making the feature information of the target feature richer.

[0079] S290. The target features are classified through the classification layer of the recognition model to determine the text recognition result corresponding to the text image.

[0080] The technical solution of this invention extracts features from text images through a hybrid perceptual mapping unit in the first feature extraction unit, obtaining intermediate global features of the first feature extraction unit. Then, hybrid perceptual mapping units in other feature extraction units extract features from the intermediate global features of the previous feature extraction unit, obtaining intermediate global features for other feature extraction units. This allows for the acquisition of intermediate global features of different sizes, improving the richness of intermediate global feature extraction and laying the foundation for the accuracy of target feature acquisition. When calculating the intermediate global features of each feature extraction unit, as the number of feature extractions increases, the detailed features in the feature extraction results, i.e., local key details, gradually decrease. Therefore, the intermediate global features, or intermediate fused features obtained by fusing intermediate global features with intermediate local features, are used as input to the spatial perceptual aggregation unit in other feature extraction units to obtain intermediate fused features for other feature extraction units. This achieves the transfer of intermediate global features or intermediate fused features from the previous feature extraction unit to the next feature extraction unit, preserving the detailed features in the feature results of the previous feature extraction unit. This increases the detailed features in the target features, improves the accuracy of target feature acquisition, and thus improves the accuracy of the recognition model.

[0081] Example 3

[0082] Figure 3a and Figure 3b This is a flowchart of a pictographic character recognition method provided in Embodiment 3 of the present invention. Based on the above embodiments, this embodiment refines the step of extracting features from the text image using the hybrid perceptual mapping unit in the first feature extraction unit to obtain the intermediate global features of the first feature extraction unit. For example... Figure 3a and Figure 3b As shown, the method includes:

[0083] S301. Input the text image into the recognition model.

[0084] S302. The text image is subjected to feature extraction through at least one feature extraction layer of the hybrid sensing mapping unit in the first feature extraction unit to obtain a first feature map; the first feature extraction unit in the at least two feature extraction units includes a hybrid sensing mapping unit, and the other feature extraction units in the at least two feature extraction units include a hybrid sensing mapping unit and a spatial sensing aggregation unit.

[0085] A feature extraction layer refers to the module in a hybrid sensing mapping unit (HRM) used for feature extraction. A HRM includes at least two feature extraction layers, which are connected in series, and the feature extraction results from each layer have the same size. Each feature extraction layer includes at least convolution and normalization to achieve feature extraction from text images. Of the at least two concatenated feature extraction layers, all except the last one include activation functions to learn the non-linear relationships in the feature extraction results of the other layers. The last feature extraction layer refers to the last feature extraction layer in the series-connected HRM. The first feature map refers to the output of the last feature extraction layer after the text image has passed through at least two feature extraction layers.

[0086] Specifically, in the first feature extraction unit, features are extracted from the text image through at least two feature extraction layers in the hybrid sensing mapping unit, and the resulting feature extraction result is determined as the first feature map.

[0087] S303. Based on the channel attention mechanism, feature extraction is performed on the first feature map to obtain the global effective features of the first feature map, wherein the channel attention mechanism is used to focus on the foreground information in the text image and ignore the background information in the text image.

[0088] Attention, simply put, is the ability to focus attention on important points amidst a multitude of information, selecting key information while ignoring less important details. Channel attention mechanisms can be simply understood as assigning different weights to different channels, thereby enhancing attention to key channels. In this application, key channels are those corresponding to foreground information, which have higher weights than those corresponding to background information. Therefore, the attention mechanism can focus on foreground information in text images while ignoring background information. For example, in a deep learning network, each text image is initially represented by three channels (R, G, B). By assigning different weights to these three channels, foreground and background information in the text image can be distinguished, thus increasing attention to the foreground information.

[0089] Specifically, based on the channel attention mechanism, the first feature map is reduced to 1×1 dimension in the spatial dimension using max pooling. The reduced feature map is then subjected to convolution to extract features. The extracted feature map has the same size as the reduced feature map. An activation function is used to learn the non-linear relationship in the channel dimension, obtaining the globally effective features of the first feature map. Max pooling for dimensionality reduction in the spatial dimension, compared to average pooling, better preserves the differences between foreground and background information, providing a foundation for text image recognition. The activation function can be set according to the specific requirements.

[0090] S304. Based on the spatial attention mechanism, feature extraction is performed on the first feature map to obtain the global key position features of the first feature map, wherein the spatial attention mechanism is used to focus on the position of the text in the text image and ignore the position where the text is not in the text image.

[0091] Spatial attention mechanisms can be understood as transforming spatial information in a text image into another space while retaining key spatial information. Alternatively, they can be understood as performing corresponding spatial transformations on the spatial information in a text image to identify the areas requiring attention and extract crucial spatial information. In this application, the areas requiring attention in the text image are those containing text; areas without text can be ignored. Therefore, the spatial attention mechanism focuses on the positions of text within the text image, ignoring areas where text is not present.

[0092] Specifically, based on the spatial attention mechanism, the first feature map is reduced to 1 dimension in the channel dimension using max pooling. The sigmoid activation function is then used to normalize the dimensionality-reduced feature map, yielding the global key location features of the first feature map. Max pooling, compared to average pooling, better preserves the differences between features at locations containing text and those at locations without text, providing a foundation for text image recognition.

[0093] S305. The first feature map, the global effective features of the first feature map, and the global key location features of the first feature map are fused to obtain the first fused feature of the first feature extraction unit.

[0094] The first fused feature of the first feature extraction unit refers to the feature obtained by fusing the first feature map, the global effective features of the first feature map, and the global key position features of the first feature map.

[0095] Specifically, the first feature map, the global effective features of the first feature map, and the global key location features of the first feature map are fused together, and the fusion result is determined as the first fused feature of the first feature extraction unit.

[0096] For example, the feature elements in the same position in the first feature map, the global effective features of the first feature map, and the global key position features of the first feature map can be multiplied together, and the result can be determined as the first fused feature of the first feature extraction unit.

[0097] S306. Determine the intermediate global features of the first feature extraction unit based on the first fusion feature, and execute step S307.

[0098] Specifically, the first fused feature is fused with the input features of the hybrid sensing mapping unit to obtain the fused result. The non-linear relationship of the summation result is learned through an activation function to obtain the intermediate global features of the first feature extraction unit. Here, the input feature of the first hybrid sensing mapping unit in the first feature extraction unit is the text image, and the input features of other hybrid sensing mapping units are the output results of the previous hybrid sensing mapping unit. For example, the first fused feature can be added to the input features of the hybrid sensing mapping unit to achieve fusion.

[0099] S307. The intermediate global features of the previous feature extraction unit are extracted by the hybrid sensing mapping unit in the other feature extraction unit to obtain the intermediate global features of the other feature extraction unit.

[0100] S308. The intermediate global features of the first feature extraction unit are extracted by the spatial perception aggregation unit in the first other feature extraction unit to obtain the intermediate local features of the first other feature extraction unit.

[0101] S309. The intermediate local features of the first other feature extraction unit are fused with the intermediate global features of the first other feature extraction unit to obtain the intermediate fused features of the first other feature extraction unit.

[0102] S310. Through the spatial perception aggregation unit in the other feature extraction units besides the first other feature extraction unit, the intermediate fusion features of the previous feature extraction unit are extracted to obtain the intermediate local features of the other feature extraction units besides the first other feature extraction unit.

[0103] S311. The intermediate local features of the feature extraction units other than the first feature extraction unit are fused with the intermediate global features of the feature extraction units other than the first feature extraction unit to obtain the intermediate fused features of the feature extraction units other than the first feature extraction unit.

[0104] S312. The intermediate fusion features of the last other feature extraction unit are determined as the target features.

[0105] S313. The target features are classified through the classification layer of the recognition model to determine the text recognition result corresponding to the text image.

[0106] The technical solution of this invention obtains a first feature map through at least one feature extraction layer of the hybrid perception mapping unit in the first feature extraction unit. Features are then extracted from the first feature map based on channel attention and spatial attention mechanisms, and the feature extraction results are fused to obtain the first fused feature of the first feature extraction unit. Fusing features obtained from different attention mechanisms increases the difference between important and redundant features, thus strengthening the features of interest. This allows the intermediate global features obtained by the hybrid perception mapping unit to contain more important information, thereby improving the accuracy of text recognition. In existing technologies, attention mechanisms typically use average pooling for dimensionality reduction. Average pooling is used to calculate the average of foreground and background information, which reduces the proportion of foreground information and is not conducive to recognizing text in images. Therefore, this application uses max pooling in both the channel attention and spatial attention mechanisms for dimensionality reduction, which can retain foreground information and features indicating the location of text to the greatest extent, thereby improving the accuracy of text recognition.

[0107] Based on the above embodiments, the method further includes: adjusting the size of the intermediate global features of the concatenated previous feature extraction unit through the pooling layer in the other feature extraction unit, and updating the intermediate global features so that the updated intermediate global features and the intermediate local features of the other feature extraction unit have the same size; the step of extracting features from the intermediate global features of the concatenated previous feature extraction unit through the hybrid sensing mapping unit in the other feature extraction unit to obtain the intermediate global features of the other feature extraction unit includes: extracting features from the updated intermediate global features through at least one feature extraction layer in the hybrid sensing mapping unit in the other feature extraction unit to obtain a second feature map; extracting features from the updated intermediate global features through at least one feature extraction layer to obtain a second feature map; extracting features from the second feature map based on a channel attention mechanism to obtain the global effective features of the second feature map; extracting features from the second feature map based on a spatial attention mechanism to obtain the global key position features of the second feature map; fusing the second feature map, the global effective features of the second feature map, and the global key position features of the second feature map to obtain the second fused feature of the other feature extraction unit; and determining the intermediate global features of the other feature extraction unit based on the second fused feature.

[0108] Size adjustment can be achieved by reducing the size of the intermediate global features of the preceding feature extraction unit in the sequence.

[0109] Specifically, in other feature extraction units, the intermediate global features of the preceding feature extraction unit are resized using max pooling, and the resized features are used as the updated intermediate global features. The updated intermediate global features have the same size as the intermediate local features of other feature units.

[0110] The second feature map refers to the result output by the last feature extraction layer after the updated intermediate global features have passed through at least one feature extraction layer.

[0111] In the hybrid sensing mapping unit of other feature extraction units, feature extraction is performed on the updated intermediate global features through at least two feature extraction layers, and the feature extraction result obtained by the last feature extraction layer is determined as the second feature map.

[0112] Based on the channel attention mechanism, the second feature map is reduced to 1×1 dimension in the spatial dimension by max pooling. The feature map after dimensionality reduction is then processed by convolution to extract features. The feature map after feature extraction has the same size as the feature map after dimensionality reduction. The non-linear relationship in the channel dimension is learned by the activation function to obtain the global effective features of the second feature map.

[0113] Based on the spatial attention mechanism, the second feature map is reduced to 1 dimension in the channel dimension by max pooling. The reduced feature map is then normalized by the sigmoid activation function to obtain the global key location features of the second feature map.

[0114] The second fusion feature of other feature extraction units refers to the feature obtained by fusing the second feature map, the global effective features of the second feature map, and the global key location features of the second feature map.

[0115] The second fusion feature is fused with the input features of the hybrid sensing mapping unit to obtain the fusion result. The non-linear relationship of the summation result is learned through an activation function to obtain intermediate global features for other feature extraction units. Specifically, the input features of the first hybrid sensing mapping unit in the other feature extraction units are the intermediate global features of the preceding feature extraction unit, and the input features of the other hybrid sensing mapping units are the output results of the previous hybrid sensing mapping unit. For example, the second fusion feature can be added to the input features of the hybrid sensing mapping unit to achieve fusion.

[0116] The intermediate global features of the preceding feature extraction unit are resized and updated by pooling layers in other feature extraction units. In the hybrid perceptual mapping unit, features are extracted from the updated intermediate global features through at least two feature extraction layers to obtain a second feature map. Features are then extracted from the second feature map based on channel attention and spatial attention mechanisms, and the feature extraction results are fused to obtain the second fused features of other feature extraction units. This approach enables the acquisition of intermediate global features of different sizes based on the intermediate global features of the preceding feature extraction unit, increasing the depth of feature extraction for text images and improving the recognition model's ability to extract features from text images, thereby improving the accuracy of text recognition.

[0117] Example 4

[0118] Figure 4a and Figure 4b This is a flowchart of a pictographic character recognition method provided in Embodiment 4 of the present invention. Based on the above embodiments, this embodiment refines the intermediate local features of the first feature extraction unit by extracting the intermediate global features of the first feature extraction unit through the spatial perception aggregation unit in the first other feature extraction unit. For example... Figure 4a and Figure 4b As shown, the method includes:

[0119] S401. Input the text image into the recognition model.

[0120] S402. The text image is subjected to feature extraction by the hybrid sensing mapping unit in the first feature extraction unit to obtain the intermediate global features of the first feature extraction unit; the first feature extraction unit in the at least two feature extraction units includes a hybrid sensing mapping unit, and the other feature extraction units in the at least two feature extraction units include a hybrid sensing mapping unit and a spatial sensing aggregation unit.

[0121] S403. The intermediate global features of the previous feature extraction unit are extracted by the hybrid sensing mapping unit in the other feature extraction unit to obtain the intermediate global features of the other feature extraction unit.

[0122] S404. The intermediate global features of the first feature extraction unit are extracted by the spatial perception aggregation unit in the first other feature extraction unit to obtain the intermediate local features of the first other feature extraction unit.

[0123] S405. The intermediate local features of the first other feature extraction unit are fused with the intermediate global features of the first other feature extraction unit to obtain the intermediate fused features of the first other feature extraction unit, and step S406 is executed.

[0124] S406. Using the spatial awareness aggregation unit in the first other feature extraction unit, the intermediate global features of the first feature extraction unit are extracted based on the spatial attention mechanism to obtain the first local key detail features.

[0125] The first local key detail feature refers to the feature obtained by the spatial perception aggregation unit in the first other feature extraction unit after extracting the intermediate global features based on the spatial attention mechanism.

[0126] Specifically, based on the spatial attention mechanism, the intermediate global features of the first feature extraction unit are reduced to 1 dimension in the channel dimension by max pooling. The reduced feature map is then normalized by the sigmoid activation function to obtain the global key position features of the first feature map.

[0127] S407. Adjust the size of the first local key detail feature to obtain the intermediate local feature of the first other feature extraction unit, so that the size of the intermediate local feature of the first other feature extraction unit is the same as that of the intermediate global feature of the first other feature extraction unit.

[0128] Specifically, the size of the first local key detail feature is adjusted so that the size of the intermediate local feature of the first other feature extraction unit is the same as the size of the intermediate global feature of the first other feature extraction unit, and the adjusted feature is determined as the intermediate local feature of the first other feature extraction unit. The size adjustment of the first local key detail feature can be achieved by reducing its size; specifically, this can be done through convolution or max pooling.

[0129] S408. The intermediate local features of the feature extraction units other than the first feature extraction unit are fused with the intermediate global features of the feature extraction units other than the first feature extraction unit to obtain the intermediate fused features of the feature extraction units other than the first feature extraction unit.

[0130] S409. The intermediate fusion features of the last other feature extraction unit are determined as the target features.

[0131] S410. The target features are classified through the classification layer of the recognition model to determine the text recognition result corresponding to the text image.

[0132] The technical solution of this invention uses a spatial awareness aggregation unit in the first other feature extraction unit to extract features from the intermediate global features of the first feature extraction unit based on a spatial attention mechanism, thereby obtaining the first local key detail features. The size of the first local key detail features is adjusted to obtain the intermediate local features of the first other feature extraction unit, thereby achieving feature size unification and feature fusion. It can focus on the position of the text in the intermediate global features of the first feature extraction unit and enhance the feature of the text position to improve the accuracy of text recognition.

[0133] Based on the above embodiments, the step of extracting intermediate fused features of the concatenated previous feature extraction unit through the spatial awareness aggregation unit among the other feature extraction units (excluding the first other feature extraction unit) to obtain intermediate local features of the other feature extraction units (excluding the first other feature extraction unit) includes: extracting features of the intermediate fused features of the concatenated previous feature extraction unit through the spatial awareness aggregation unit among the other feature extraction units (excluding the first other feature extraction unit) based on a spatial attention mechanism to obtain the second local key detail feature; adjusting the size of the second global key position feature to obtain the intermediate local features of the other feature extraction units (excluding the first other feature extraction unit), so that the size of the intermediate local features of the other feature extraction units (excluding the first other feature extraction unit) and the intermediate global features of the other feature extraction units (excluding the first other feature extraction unit) are the same.

[0134] The second local key detail feature refers to the feature obtained by the spatial perception aggregation unit in the feature extraction units other than the first other feature extraction unit, which extracts the intermediate fusion features of the previous feature extraction unit based on the spatial attention mechanism.

[0135] Specifically, based on the spatial attention mechanism, the spatial perception aggregation unit of the feature extraction units other than the first feature extraction unit reduces the intermediate fusion features of the concatenated previous feature extraction unit to 1 dimension in the channel dimension by max pooling. The sigmoid activation function is then used to normalize the dimensionality-reduced feature map to obtain the global key position features of the second feature map.

[0136] The size of the second local key detail feature is adjusted so that the size of the intermediate local features of all feature extraction units other than the first one is the same as the size of the intermediate global features of all feature extraction units other than the first one. This adjusted feature is then identified as the intermediate local feature of all feature extraction units other than the first one. This size adjustment of the second local key detail feature can be achieved by reducing its size; specifically, this can be done through convolution or max pooling.

[0137] By using the spatial awareness aggregation unit in the feature extraction units other than the first one, the intermediate fusion features of the concatenated previous feature extraction unit are extracted based on the spatial attention mechanism to obtain the second local key detail features. The size of the second local key detail features is adjusted to obtain the intermediate local features of the feature extraction units other than the first one, thus achieving feature size uniformity. This allows the intermediate fusion features of the concatenated previous feature extraction unit to be passed to the current feature extraction unit, preserving the information in the intermediate fusion features of the previous feature extraction unit to the greatest extent, thereby improving the accuracy of text recognition.

[0138] Example 5

[0139] Figure 5 This is a schematic diagram illustrating an application scenario of a pictographic character recognition method provided in Embodiment 5 of the present invention. For example... Figure 5 As shown, the implementation process of this method is as follows:

[0140] Input the text image into the recognition model.

[0141] The text images are images of Dongba pictographs, with each image containing at least one Dongba pictograph. The recognition model comprises at least two feature extraction units. The first feature extraction unit includes a hybrid sensing mapping unit, while other feature extraction units include a hybrid sensing mapping unit and a spatial sensing aggregation unit. The hybrid sensing mapping unit is used to acquire global effective features and global key location features; the spatial sensing aggregation unit is used to acquire local key detail features; and different feature extraction units are used to extract features of different sizes.

[0142] The text image is processed by extracting features through at least one feature extraction layer of the hybrid sensing mapping unit in the first feature extraction unit to obtain the first feature map.

[0143] Based on the channel attention mechanism, feature extraction is performed on the first feature map to obtain the global effective features of the first feature map. The channel attention mechanism is used to focus on the foreground information in the text image and ignore the background information in the text image.

[0144] Feature extraction is performed on the first feature map based on the spatial attention mechanism to obtain the global key position features of the first feature map. The spatial attention mechanism is used to focus on the position of the text in the text image and ignore the position where the text is not in the text image.

[0145] The first feature map, the global effective features of the first feature map, and the global key location features of the first feature map are multiplied together to obtain the first fused feature of the first feature extraction unit.

[0146] The intermediate global features of the first feature extraction unit are determined based on the first fusion feature.

[0147] Specifically, the first fused feature is added to the input feature of the hybrid sensing mapping unit to obtain the summed result. The non-linear relationship of the summed result is learned through an activation function to obtain the intermediate global features of the first feature extraction unit. The input feature of the first hybrid sensing mapping unit in the first feature extraction unit is the text image, and the input features of other hybrid sensing mapping units are the output of the previous hybrid sensing mapping unit.

[0148] The pooling layers in other feature extraction units use max pooling to reduce the size of the intermediate global features of the preceding feature extraction unit and update the intermediate global features so that the updated intermediate global features have the same size as the intermediate local features of other feature extraction units.

[0149] The updated intermediate global features are extracted through at least one feature extraction layer in the hybrid sensing mapping unit of other feature extraction units to obtain a second feature map.

[0150] Feature extraction is performed on the second feature map based on the channel attention mechanism to obtain the global effective features of the second feature map.

[0151] The second feature map, the global effective features of the second feature map, and the global key location features of the second feature map are added together to obtain the second fused features of other feature extraction units.

[0152] The intermediate global features of other feature extraction units are determined based on the second fusion feature.

[0153] The second fused feature is added to the input feature of the hybrid sensing mapping unit to obtain the summed result. The non-linear relationship of the summed result is learned through an activation function to obtain the intermediate global features of other feature extraction units. Among these, the input feature of the first hybrid sensing mapping unit in the other feature extraction units is the intermediate global feature of the preceding feature extraction unit, and the input feature of the other hybrid sensing mapping units is the output result of the previous hybrid sensing mapping unit.

[0154] By using the spatial awareness aggregation unit in the first other feature extraction unit, the intermediate global features of the first feature extraction unit are extracted based on the spatial attention mechanism to obtain the first local key detail features.

[0155] The size of the first local key detail feature is reduced to obtain the intermediate local feature of the first other feature extraction unit, so that the size of the intermediate local feature of the first other feature extraction unit is the same as that of the intermediate global feature of the first other feature extraction unit.

[0156] The intermediate local features of the first other feature extraction unit are added to the intermediate global features of the first other feature extraction unit to obtain the intermediate fusion features of the first other feature extraction unit;

[0157] By using the spatial awareness aggregation unit in the feature extraction units other than the first other feature extraction unit, the intermediate fused features of the concatenated previous feature extraction unit are extracted based on the spatial attention mechanism to obtain the second local key detail features.

[0158] The size of the second global key location feature is reduced to obtain the intermediate local features of the other feature extraction units excluding the first other feature extraction unit, so that the size of the intermediate local features of the other feature extraction units excluding the first other feature extraction unit is the same as that of the intermediate global features of the other feature extraction units excluding the first other feature extraction unit.

[0159] The intermediate local features of all feature extraction units other than the first feature extraction unit are added to the intermediate global features of all feature extraction units other than the first feature extraction unit to obtain the intermediate fused features of all feature extraction units other than the first feature extraction unit.

[0160] The intermediate fusion features of the last other feature extraction unit are determined as the target features.

[0161] The target features are classified using the classification layer of the recognition model to determine the text recognition result corresponding to the text image. The classification layer includes a global average pooling layer and a fully connected layer.

[0162] The technical solution of this invention involves inputting a text image into a recognition model. At least two feature extraction units connected in series within the recognition model extract features from the text image to obtain target features. Different feature extraction units are used to extract features of different sizes, enabling the acquisition of text image features at different sizes and increasing the richness of the extracted text image features. Simultaneously, the feature extraction units include a hybrid perception mapping unit and / or a spatial perception aggregation unit. The hybrid perception mapping unit is used to acquire global effective features and global key position features, while the spatial perception aggregation unit is used to acquire local key detail features. This allows the target features to focus on both global effective features and global key position features while also paying attention to local key detail features, thus improving feature richness. Local key detail features retain more detailed information, effectively extracting detail features from images where the text outlines are nearly identical but there are slight differences in the details of the character shapes. Therefore, when classifying the target features through the classification layer of the recognition model to determine the text recognition result corresponding to the text image, the accuracy of text recognition can be improved.

[0163] Example 6

[0164] Figure 6 Figure 3 is a schematic diagram of a pictographic character recognition device provided in Embodiment 6 of the present invention. As shown in Figure 3, the device includes: an input module 601, a target feature acquisition module 602, and a result determination module 603.

[0165] The input module 601 is used to input text images into the recognition model;

[0166] The target feature acquisition module 602 is used to extract features from a text image through at least two concatenated feature extraction units of the recognition model to obtain target features; wherein, the feature extraction unit includes a hybrid sensing mapping unit and / or a spatial sensing aggregation unit; the hybrid sensing mapping unit is used to acquire local effective features and local key detail features; the spatial sensing aggregation unit is used to acquire global key location features; different feature extraction units are used to extract features of different sizes;

[0167] The result determination module 603 is used to classify the target features through the classification layer of the recognition model and determine the text recognition result corresponding to the text image.

[0168] The technical solution of this invention involves inputting a text image into a recognition model. At least two feature extraction units connected in series within the recognition model extract features from the text image to obtain target features. Different feature extraction units are used to extract features of different sizes, enabling the acquisition of text image features at different sizes and increasing the richness of the extracted text image features. Simultaneously, the feature extraction units include a hybrid perception mapping unit and / or a spatial perception aggregation unit. The hybrid perception mapping unit is used to acquire global effective features and global key position features, while the spatial perception aggregation unit is used to acquire local key detail features. This allows the target features to focus on both global effective features and global key position features while also paying attention to local key detail features, thus improving feature richness. Local key detail features retain more detailed information, effectively extracting detail features from images where the text outlines are nearly identical but there are slight differences in the details of the character shapes. Therefore, when classifying the target features through the classification layer of the recognition model to determine the text recognition result corresponding to the text image, the accuracy of text recognition can be improved.

[0169] Optionally, the first feature extraction unit in the at least two feature extraction units includes a hybrid sensing mapping unit, and the other feature extraction units in the at least two feature extraction units include a hybrid sensing mapping unit and a spatial sensing aggregation unit;

[0170] Target feature acquisition module 602 includes:

[0171] The first intermediate global feature determination unit is used to extract features from the text image through the hybrid perception mapping unit in the first feature extraction unit to obtain the intermediate global features of the first feature extraction unit.

[0172] The second intermediate global feature determination unit is used to extract the intermediate global features of the previous feature extraction unit by using the hybrid sensing mapping unit in the other feature extraction units to obtain the intermediate global features of the other feature extraction units.

[0173] The first intermediate local feature determination unit is used to extract intermediate global features of the first feature extraction unit through the inter-perception aggregation unit in the first other feature extraction unit, so as to obtain intermediate local features of the first other feature extraction unit.

[0174] The first intermediate fusion feature determination unit is used to fuse the intermediate local features of the first other feature extraction unit with the intermediate global features of the first other feature extraction unit to obtain the intermediate fusion feature of the first other feature extraction unit;

[0175] The second intermediate local feature determination unit is used to extract the intermediate fusion features of the previous feature extraction unit by using the spatial perception aggregation unit in the other feature extraction units besides the first other feature extraction unit, so as to obtain the intermediate local features of the other feature extraction units besides the first other feature extraction unit.

[0176] The second intermediate fusion feature determination unit is used to fuse the intermediate local features of the feature extraction units other than the first other feature extraction unit with the intermediate global features of the feature extraction units other than the first other feature extraction unit to obtain the intermediate fusion features of the feature extraction units other than the first other feature extraction unit.

[0177] The target feature acquisition unit determines the intermediate fusion features of the last other feature extraction unit as the target feature.

[0178] Optionally, the first intermediate global feature determination unit includes:

[0179] The first feature map determination subunit is used to extract features from the text image through at least one feature extraction layer of the hybrid perceptual mapping unit in the first feature extraction unit to obtain the first feature map;

[0180] The first global effective feature acquisition subunit is used to extract features from the first feature map based on the channel attention mechanism to obtain the global effective features of the first feature map. The channel attention mechanism is used to focus on the foreground information in the text image and ignore the background information in the text image.

[0181] The first global key location feature acquisition subunit is used to extract features from the first feature map based on the spatial attention mechanism to obtain the global key location features of the first feature map. The spatial attention mechanism is used to focus on the position of the text in the text image and ignore the position where the text is not in the text image.

[0182] The first fusion feature acquisition subunit is used to fuse the first feature map, the global effective features of the first feature map, and the global key position features of the first feature map to obtain the first fusion feature of the first feature extraction unit;

[0183] The first intermediate global feature acquisition subunit is used to determine the intermediate global features of the first feature extraction unit based on the first fused features.

[0184] Optionally, the target feature acquisition module 602 further includes:

[0185] The size adjustment unit is used to adjust the size of the intermediate global feature of the concatenated previous feature extraction unit through the pooling layer in the other feature extraction unit, and update the intermediate global feature so that the size of the updated intermediate global feature is the same as that of the intermediate local feature of the other feature extraction unit.

[0186] The second intermediate global feature determination unit includes:

[0187] The second feature map acquisition subunit is used to extract features from the updated intermediate global features through at least one feature extraction layer to obtain a second feature map.

[0188] The second global effective feature acquisition subunit is used to extract features from the second feature map based on the channel attention mechanism and acquire the global effective features of the second feature map.

[0189] The second global key location feature acquisition subunit is used to extract features from the second feature map based on the spatial attention mechanism to obtain the global key location features of the second feature map;

[0190] The second fusion feature acquisition subunit is used to fuse the second feature map, the global effective features of the second feature map, and the global key position features of the second feature map to obtain the second fusion feature of the other feature extraction units;

[0191] The second global feature acquisition subunit is used to determine the intermediate global features of the other feature extraction units based on the second fusion feature.

[0192] Optionally, the first intermediate local feature determination unit includes:

[0193] The first local key location acquisition subunit is used to extract the intermediate global features of the first feature extraction unit based on the spatial attention mechanism through the spatial awareness aggregation unit in the first other feature extraction unit to obtain the first local key detail features;

[0194] The size adjustment subunit is used to adjust the size of the first local key detail feature to obtain the intermediate local feature of the first other feature extraction unit, so that the size of the intermediate local feature of the first other feature extraction unit is the same as that of the intermediate global feature of the first other feature extraction unit.

[0195] Optionally, the second intermediate local feature determination unit includes:

[0196] The second local key location acquisition subunit is used to extract the intermediate fusion features of the concatenated previous feature extraction unit by using the spatial awareness aggregation unit in the other feature extraction units besides the first other feature extraction unit, based on the spatial attention mechanism, to obtain the second local key detail features.

[0197] The third size adjustment subunit is used to adjust the size of the second global key position feature to obtain the intermediate local features of the other feature extraction units besides the first other feature extraction unit, so that the size of the intermediate local features of the other feature extraction units besides the first other feature extraction unit is the same as the size of the intermediate global features of the other feature extraction units besides the first other feature extraction unit.

[0198] Optionally, the text image is an image of Dongba pictographs, and a text image includes at least one Dongba pictograph.

[0199] The pictographic character recognition device provided in the embodiments of the present invention can execute the pictographic character recognition method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the method.

[0200] Example 7

[0201] Figure 7 A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0202] like Figure 7As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0203] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0204] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as pictographic character recognition methods.

[0205] In some embodiments, the pictographic recognition method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the pictographic recognition method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the pictographic recognition method by any other suitable means (e.g., by means of firmware).

[0206] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0207] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0208] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0209] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0210] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0211] A computing system can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system. It addresses the shortcomings of traditional physical hosts and VPS (Virtual Private Server) services, such as high management difficulty and weak business scalability.

[0212] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0213] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for recognizing pictographs, characterized in that, include: Input the text image into the recognition model; The text image is processed by at least two concatenated feature extraction units of the recognition model to obtain target features; wherein, the feature extraction unit includes a hybrid sensing mapping unit and / or a spatial sensing aggregation unit; the hybrid sensing mapping unit is used to obtain global effective features and global key location features; the spatial sensing aggregation unit is used to obtain local key detail features; different feature extraction units are used to extract features of different sizes; The target features are classified through the classification layer of the recognition model to determine the text recognition result corresponding to the text image; The first feature extraction unit among the at least two feature extraction units includes a hybrid sensing mapping unit, and the other feature extraction units among the at least two feature extraction units include a hybrid sensing mapping unit and a spatial sensing aggregation unit. The step of extracting features from a text image using at least two concatenated feature extraction units of the recognition model to obtain target features includes: The text image is subjected to feature extraction by the hybrid perceptual mapping unit in the first feature extraction unit to obtain the intermediate global features of the first feature extraction unit; wherein, the first feature extraction unit is the feature extraction unit in the recognition model that directly extracts features from the text image; The intermediate global features of the other feature extraction units are obtained by extracting features from the intermediate global features of the previous feature extraction unit through the hybrid sensing mapping unit in the other feature extraction units. The intermediate global features of the first feature extraction unit are extracted by the spatially aware aggregation unit in the first other feature extraction unit to obtain the intermediate local features of the first other feature extraction unit; The intermediate local features of the first other feature extraction unit are fused with the intermediate global features of the first other feature extraction unit to obtain the intermediate fused features of the first other feature extraction unit; By using the spatially-aware aggregation unit in the feature extraction units other than the first feature extraction unit, the intermediate fused features of the previous feature extraction unit are extracted to obtain the intermediate local features of the feature extraction units other than the first feature extraction unit. The intermediate local features of all feature extraction units other than the first feature extraction unit are fused with the intermediate global features of all feature extraction units other than the first feature extraction unit to obtain the intermediate fused features of all feature extraction units other than the first feature extraction unit. The intermediate fusion features of the last other feature extraction unit are determined as the target features.

2. The method according to claim 1, characterized in that, The step of extracting features from the text image through the hybrid perceptual mapping unit in the first feature extraction unit to obtain the intermediate global features of the first feature extraction unit includes: The text image is subjected to feature extraction through at least one feature extraction layer of the hybrid perceptual mapping unit in the first feature extraction unit to obtain a first feature map; Feature extraction is performed on the first feature map based on the channel attention mechanism to obtain the global effective features of the first feature map. The channel attention mechanism is used to focus on the foreground information in the text image and ignore the background information in the text image. Based on the spatial attention mechanism, feature extraction is performed on the first feature map to obtain the global key position features of the first feature map. The spatial attention mechanism is used to focus on the position of the text in the text image and ignore the position where the text is not in the text image. The first feature map, the global effective features of the first feature map, and the global key location features of the first feature map are fused to obtain the first fused feature of the first feature extraction unit; The intermediate global features of the first feature extraction unit are determined based on the first fusion feature.

3. The method according to claim 1, characterized in that, Also includes: The intermediate global features of the concatenated previous feature extraction unit are resized by the pooling layer in the other feature extraction units, and the intermediate global features are updated so that the updated intermediate global features have the same size as the intermediate local features of the other feature extraction units. The step of extracting intermediate global features from the concatenated previous feature extraction unit through the hybrid sensing mapping unit in the other feature extraction units to obtain the intermediate global features of the other feature extraction units includes: The updated intermediate global features are extracted through at least one feature extraction layer of the hybrid sensing mapping unit in the other feature extraction units to obtain a second feature map; Feature extraction is performed on the second feature map based on the channel attention mechanism to obtain the global effective features of the second feature map; Based on the spatial attention mechanism, feature extraction is performed on the second feature map to obtain the global key location features of the second feature map; The second feature map, the global effective features of the second feature map, and the global key location features of the second feature map are fused to obtain the second fused feature of the other feature extraction units; The intermediate global features of the other feature extraction units are determined based on the second fusion feature.

4. The method according to claim 1, characterized in that, The step of extracting intermediate global features of the first feature extraction unit through the spatially aware aggregation unit in the first other feature extraction unit to obtain intermediate local features of the first other feature extraction unit includes: The spatial awareness aggregation unit in the first other feature extraction unit extracts the intermediate global features of the first feature extraction unit based on the spatial attention mechanism to obtain the first local key detail features; The size of the first local key detail feature is adjusted to obtain the intermediate local feature of the first other feature extraction unit, so that the size of the intermediate local feature of the first other feature extraction unit is the same as that of the intermediate global feature of the first other feature extraction unit.

5. The method according to claim 1, characterized in that, The step of extracting intermediate fused features from the concatenated previous feature extraction units through spatially aware aggregation units (excluding the first feature extraction unit) to obtain intermediate local features from the other feature extraction units (excluding the first feature extraction unit) includes: By using the spatial awareness aggregation unit in the feature extraction units other than the first other feature extraction unit, the intermediate fusion features of the concatenated previous feature extraction unit are extracted based on the spatial attention mechanism to obtain the second local key detail features. The size of the second local key detail feature is adjusted to obtain the intermediate local features of the feature extraction units other than the first other feature extraction unit, so that the size of the intermediate local features of the feature extraction units other than the first other feature extraction unit is the same as the size of the intermediate global features of the feature extraction units other than the first other feature extraction unit.

6. The method according to claim 1, characterized in that, The text images are images of Dongba pictographs, and each text image includes at least one Dongba pictograph.

7. A pictographic character recognition device, characterized in that, include: The input module is used to input text images into the recognition model; The target feature acquisition module is used to extract features from text images through at least two concatenated feature extraction units of the recognition model to obtain target features; wherein, the feature extraction unit includes a hybrid perception mapping unit and / or a spatial perception aggregation unit; the hybrid perception mapping unit is used to acquire local effective features and local key detail features; the spatial perception aggregation unit is used to acquire local key detail features; different feature extraction units are used to extract features of different sizes; The result determination module is used to classify the target features through the classification layer of the recognition model and determine the text recognition result corresponding to the text image. The first feature extraction unit among the at least two feature extraction units includes a hybrid sensing mapping unit, and the other feature extraction units among the at least two feature extraction units include a hybrid sensing mapping unit and a spatial sensing aggregation unit. The target feature acquisition module includes: The first intermediate global feature determination unit is used to extract features from the text image through the hybrid perception mapping unit in the first feature extraction unit to obtain the intermediate global features of the first feature extraction unit; wherein, the first feature extraction unit is the feature extraction unit in the recognition model that directly extracts features from the text image; The second intermediate global feature determination unit is used to extract the intermediate global features of the previous feature extraction unit by using the hybrid sensing mapping unit in the other feature extraction units to obtain the intermediate global features of the other feature extraction units. The first intermediate local feature determination unit is used to extract the intermediate global features of the first feature extraction unit through the spatial awareness aggregation unit in the first other feature extraction unit to obtain the intermediate local features of the first other feature extraction unit; The first intermediate fusion feature determination unit is used to fuse the intermediate local features of the first other feature extraction unit with the intermediate global features of the first other feature extraction unit to obtain the intermediate fusion feature of the first other feature extraction unit; The second intermediate local feature determination unit is used to extract the intermediate fusion features of the previous feature extraction unit by using the spatial perception aggregation unit in the other feature extraction units besides the first other feature extraction unit, so as to obtain the intermediate local features of the other feature extraction units besides the first other feature extraction unit. The second intermediate fusion feature determination unit is used to fuse the intermediate local features of the feature extraction units other than the first other feature extraction unit with the intermediate global features of the feature extraction units other than the first other feature extraction unit to obtain the intermediate fusion features of the feature extraction units other than the first other feature extraction unit. The target feature acquisition unit determines the intermediate fusion features of the last other feature extraction unit as the target feature.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the pictographic character recognition method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the pictographic character recognition method according to any one of claims 1-6.

Images