Text recognition method and device, readable storage medium and electronic equipment

Abstract

The invention relates to a text recognition method and device, a readable storage medium and electronic equipment. The method comprises the steps that a text image of a to-be-recognized text is acquired; acquiring image feature information of the text image; the image feature information serves as input of a pre-generated text recognition model, and a recognition result of the to-be-recognized text is obtained; the text recognition model is used for encoding the image feature information to obtain global semantic features, determining a target attention weight of each feature component in the image feature information, and decoding the global semantic features according to the target attention weights to obtain a recognition result of the to-be-recognized text, the target attention weight is used for representing the influence degree of each feature component in the image feature information on the recognition result.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular, to a text recognition method, device, readable storage medium, and electronic device. Background Art

[0002] In the intelligent recording scenario, it is common for user-uploaded images to contain irregular text. In order to achieve the effect of intelligent recording, it is necessary to identify the irregular text, thereby improving the effect of test paper restoration.

[0003] Normally, irregular text is first corrected into regular text using a correction network before recognition. Therefore, the accuracy of text recognition depends to a large extent on the correction effect of the correction network. When encountering more complex irregular text, the correction effect of the correction network is not ideal, resulting in low text recognition accuracy and poor recognition effect. Summary of the Invention

[0004] To overcome the problems existing in the related art, the present disclosure provides a text recognition method, device, readable storage medium and electronic device.

[0005] According to a first aspect of an embodiment of the present disclosure, a text recognition method is provided, the method comprising: Obtain a text image of the text to be recognized; Acquiring image feature information of the text image; The image feature information is used as the input of a pre-generated text recognition model to obtain a recognition result of the text to be recognized; the text recognition model is used to encode the image feature information to obtain a global semantic feature, and determine the target attention weight of each feature component in the image feature information, and decode the global semantic feature according to the target attention weight to obtain a recognition result of the text to be recognized, and the target attention weight is used to characterize the degree of influence of each feature component in the image feature information on the recognition result.

[0006] Optionally, the text recognition model includes an encoding model and a decoding model, wherein the encoding model is used to encode the image feature information to obtain global semantic features; The decoding model is used to determine the target attention weight of each feature component in the image feature information; decode the global semantic feature according to the target attention weight to obtain the recognition result of the text to be recognized.

[0007] Optionally, the decoding model is used to determine the target attention weight of each feature component in the image feature information in the following manner: Each component in the image feature information is weighted by the attention layer to obtain the target attention weight corresponding to each component in the image feature information.

[0008] Optionally, the decoding model includes a plurality of decoding functions; decoding the global semantic feature according to the target attention weight to obtain a recognition result of the text to be recognized includes: Using the global semantic features and the output of the previous level decoding function as the input of the current decoding function, and obtaining decoding results output by multiple decoding functions; performing weighted processing on the decoding results output by the multiple decoding functions according to the target attention weight corresponding to each feature component in the image feature information to obtain target decoding results output by the multiple decoding functions; A recognition result of the text to be recognized is obtained according to the target decoding result.

[0009] Optionally, obtaining image feature information of the text image includes: The text image is used as the input of an image feature extraction model, image feature extraction is performed on the text image, and image feature information corresponding to the text image output by the image feature extraction model is obtained.

[0010] Optionally, the image feature extraction model includes a residual block, which includes a plurality of residual layers connected in series, and is used to flow the image feature information output by the previous residual layer into the next residual layer; The step of using the text image as an input of an image feature extraction model, performing image feature extraction on the text image, and obtaining image feature information corresponding to the text image output by the image feature extraction model includes: The text image is input into the residual block to obtain image feature information output by the last residual layer in the residual layers of the multiple residual connections.

[0011] Optionally, the text recognition model is trained in the following manner: Obtaining a set of training sample images containing sample texts, wherein the texts are marked in the training sample images; Acquire sample image feature information of the training sample images in the training sample image set; The pre-generated text recognition model is trained based on the training sample image set to obtain the text recognition model.

[0012] Optionally, the recognition result of the text to be recognized includes a plurality of predicted characters and a prediction probability corresponding to each of the predicted characters, and the method further includes: For each predicted probability of the plurality of predicted probabilities, determining whether the predicted probability is less than or equal to a preset probability threshold; When it is determined that the predicted probability is less than or equal to the preset probability threshold, the predicted character corresponding to the predicted probability is deleted from the recognition result to obtain the target recognition result.

[0013] According to a second aspect of an embodiment of the present disclosure, a text recognition device is provided, the device comprising: A first acquisition module is used to acquire a text image of a text to be recognized; A second acquisition module is used to acquire image feature information of the text image; A recognition module is used to use the image feature information as input to a pre-generated text recognition model to obtain a recognition result of the text to be recognized; the text recognition model is used to encode the image feature information to obtain a global semantic feature, and determine the target attention weight of each feature component in the image feature information, and decode the global semantic feature according to the target attention weight to obtain a recognition result of the text to be recognized, and the target attention weight is used to characterize the degree of influence of each feature component in the image feature information on the recognition result.

[0014] Optionally, the text recognition model includes an encoding model and a decoding model, wherein the encoding model is used to encode the image feature information to obtain global semantic features; The decoding model is used to determine the target attention weight of each feature component in the image feature information; decode the global semantic feature according to the target attention weight to obtain the recognition result of the text to be recognized.

[0015] Optionally, the decoding model is used to determine the target attention weight of each feature component in the image feature information in the following manner: Each component in the image feature information is weighted by the attention layer to obtain the target attention weight corresponding to each component in the image feature information.

[0016] Optionally, the decoding model includes a plurality of decoding functions; decoding the global semantic feature according to the target attention weight to obtain a recognition result of the text to be recognized includes: Using the global semantic features and the output of the previous level decoding function as the input of the current decoding function, and obtaining decoding results output by multiple decoding functions; performing weighted processing on the decoding results output by the multiple decoding functions according to the target attention weight corresponding to each feature component in the image feature information to obtain target decoding results output by the multiple decoding functions; A recognition result of the text to be recognized is obtained according to the target decoding result.

[0017] Optionally, the second acquisition module is used to use the text image as input of an image feature extraction model, perform image feature extraction on the text image, and obtain image feature information corresponding to the text image output by the image feature extraction model.

[0018] Optionally, the image feature extraction model includes a residual block, which includes a plurality of residual layers connected in series, and is used to flow the image feature information output by the previous residual layer into the next residual layer; The second acquisition module is used to input the text image into the residual block to obtain image feature information output by the last residual layer in the residual layers of the multiple residual connections.

[0019] Optionally, the text recognition model is trained in the following manner: obtaining a training sample image set containing sample text, wherein the text is marked in the training sample image; obtaining sample image feature information of the training sample images in the training sample image set; and training the pre-generated text recognition model based on the training sample image set to obtain the text recognition model.

[0020] Optionally, the recognition result of the text to be recognized includes a plurality of predicted characters and a prediction probability corresponding to each of the predicted characters, and the apparatus further includes: a determination module, configured to determine, for each of the plurality of predicted probabilities, whether the predicted probability is less than or equal to a preset probability threshold; The target recognition module is used to delete the predicted character corresponding to the predicted probability from the recognition result when it is determined that the predicted probability is less than or equal to the preset probability threshold, so as to obtain the target recognition result.

[0021] According to a third aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the text recognition method provided in the first aspect of the present disclosure are implemented.

[0022] According to a fourth aspect of an embodiment of the present disclosure, there is provided an electronic device, comprising: a memory having a computer program stored thereon; A processor; configured to execute the computer program in the memory to implement the steps of the text recognition method provided in the first aspect of the present disclosure.

[0023] The technical solutions provided by the embodiments of the present disclosure may have the following beneficial effects: The present disclosure obtains a text image of a text to be recognized; obtains image feature information of the text image; uses the image feature information as input of a pre-generated text recognition model to obtain a recognition result of the text to be recognized; the text recognition model is used to encode the image feature information to obtain a global semantic feature, and determine the target attention weight of each feature component in the image feature information, and decode the global semantic feature according to the target attention weight to obtain a recognition result of the text to be recognized, and the target attention weight is used to characterize the degree of influence of each feature component in the image feature information on the recognition result. In this way, in the process of text recognition, the encoding model in the pre-generated text recognition model can be used to directly recognize the text image to be recognized to avoid the low accuracy of text recognition due to the inaccuracy of the correction network, and the target attention weight of each feature component determined can be used to implicitly model the relationship between character features and characters, and can more accurately recognize character information in irregular text images, thereby improving the accuracy of text recognition.

[0024] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following detailed description, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings: Figure 1 The figure is a flowchart of a text recognition method according to an exemplary embodiment of the present disclosure.

[0026] Figure 2 FIG. 1 is a schematic diagram showing an image containing text to be recognized according to an exemplary embodiment of the present disclosure.

[0027] Figure 3 It is a structural diagram of a coding module according to an exemplary embodiment of the present disclosure.

[0028] Figure 4 The figure is a block diagram of a text recognition device according to an exemplary embodiment of the present disclosure.

[0029] Figure 5 is based on Figure 4 The illustrated embodiment shows a block diagram of a text recognition device.

[0030] Figure 6 is a block diagram of an electronic device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0031] The following describes the specific embodiments of the present disclosure in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present disclosure and are not intended to limit the present disclosure.

[0032] It should be noted that all actions of acquiring signals, information or data in the present disclosure are carried out in compliance with the corresponding data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0033] First, the application scenario of the present disclosure is explained. The present disclosure can be applied to the intelligent question recording scenario. In this scenario, it is common that irregular text is contained in the image uploaded by the user. In order to achieve the effect of intelligent recording, it is necessary to identify editable text information from the irregular text image. For example, the present disclosure can be directly applied to terminal devices, such as smart phones, tablet computers, personal digital assistants (PDAs), etc.; in related technologies, in some cases, a correction network can be first used to correct the irregular text to obtain a corrected text image, and then text recognition is performed on the corrected text image. Usually, STN (English: Spatial Transformer Network, Chinese: Spatial Transformer Network) and STN based on TPS (English: Thin PlateSpines, Chinese: Thin Plate Spline Interpolation) are used to correct the text on the irregular text image. Therefore, the accuracy of the text recognition effect depends to a large extent on the correction effect of the correction network. When the correction effect of the correction network is inaccurate, the accuracy of the text recognition will be affected.

[0034] It should be noted that when the present disclosure is directly applied to a terminal device, the terminal device is required to store a text recognition model that has completed the training process. If the text recognition model stored on the terminal device has not completed the training process, the training process needs to be completed on the terminal device first.

[0035] In order to solve the above technical problems, the present disclosure provides a text recognition method, device, readable storage medium and electronic device, the text recognition method obtains a text image of a text to be recognized; obtains image feature information of the text image; uses the image feature information as input to a pre-generated text recognition model to obtain a recognition result of the text to be recognized; the text recognition model is used to encode the image feature information to obtain a global semantic feature, and determine the target attention weight of each feature component in the image feature information, and decode the global semantic feature according to the target attention weight to obtain a recognition result of the text to be recognized, wherein the target attention weight is used to characterize the degree of influence of each feature component in the image feature information on the recognition result. In this way, in the process of text recognition, the encoding model in the pre-generated text recognition model can be used to directly recognize the text image to be recognized, avoiding the low accuracy of text recognition due to the inaccuracy of the correction network, and using the determined target attention weight of each feature component to implicitly model the relationship between character features and characters, it can more accurately recognize character information in irregular text images, and improve the accuracy of text recognition.

[0036] The technical solution of the present disclosure is described in detail below with reference to specific embodiments.

[0037] Figure 1 is a flowchart of a text recognition method according to an exemplary embodiment of the present disclosure. Figure 1 As shown, the text recognition method is applied to terminals such as smart phones, tablet computers, and personal digital assistants (PDAs). The method includes the following steps: In step S101 , a text image of a text to be recognized is obtained.

[0038] In some embodiments, after acquiring an image, the image contains text to be recognized, and the text in the text area of the image needs to be recognized. When the size of the image is large, in order to reduce the amount of calculation, the image can be cropped to obtain a text image including the text to be recognized; and when the size of the image is small, in order to improve the recognition accuracy, the image can be enlarged.

[0039] For example, Figure 2 As shown, Figure 2 This is a schematic diagram of an image containing text to be recognized according to an exemplary embodiment of the present disclosure. The image includes area a and area b. Area a includes the text to be recognized. Therefore, when the size of the image is large, the area other than area a in the image can be cropped to obtain area a including the text to be recognized. The image of area a can be used as a text image including the text to be recognized.

[0040] In step S102, image feature information of the text image is obtained.

[0041] The image feature information of the text image may include a feature map obtained after performing feature extraction processing on the text image.

[0042] In some embodiments, the text image can be used as the input of the image feature extraction model, and image feature extraction is performed on the text image to obtain image feature information corresponding to the text image output by the image feature extraction model. The image feature extraction model includes a residual block, which includes multiple residual layers connected in series, and is used to flow the image feature information output by the previous residual layer into the next residual layer; the text image can be input into the residual block to obtain the image feature information output by the last residual layer in the multiple residual-connected residual layers. Specifically, for each residual block, if the number of channels of the input and output feature maps is different, a 1*1 convolution can be used to expand or reduce the number of channels.

[0043] Optionally, the image feature extraction model can use a ResNet (English: Residual Neural Network, Chinese: residual network) backbone network to extract image feature information to obtain image feature information corresponding to the text image. When the obtained image feature information is a feature map of x×y×z, y can be the height of the feature map, x can be the width of the feature map, and z can be the number of image channels of the feature map. The feature map is subsequently used to extract the overall features of the entire image.

[0044] For example, the image feature extraction model can also include multiple convolution layers and pooling layers, wherein the convolution layer and the pooling layer can be connected in series, the convolution layer can be used to extract features of the input data, and then the extracted features are passed through the pooling layer and filtered by the pooling layer, so that the most representative features in the receptive field can be extracted, the output scale can be effectively reduced, and the amount of parameters required by the model can be reduced; and considering that more information can be retained in the width direction of the text image, which is conducive to the recognition of narrow characters, such as the distinction between "i" and "l", for the convolution layer, all convolution sizes can be 3*3; for the maximum pooling layer, in addition to 2*2 maximum pooling, 1*2 maximum pooling can also be used.

[0045] In step S103, the image feature information is used as input to a pre-generated text recognition model to obtain a recognition result of the text to be recognized.

[0046] Among them, the text recognition model is used to encode the image feature information to obtain a global semantic feature. The global semantic feature can be a global context semantic feature, which can be used to describe the spatial semantic information (or spatial detail information) of the text image. The spatial semantic information can be used to represent the relationship between the current pixel and other pixels in the world, that is, each pixel can perceive the information of all other pixels in the image; and determine the target attention weight of each feature component in the image feature information, and based on the target attention weight, the target attention weight is used to characterize the degree of influence of each feature component in the image feature information on the recognition result.

[0047] In some embodiments, after the image feature information of the text image is obtained, multiple feature components corresponding to the image feature information can be determined based on the feature information included in the image feature information.

[0048] For example, the image feature information may be arranged in the width direction of the text image to obtain a plurality of feature components, wherein the feature component may refer to a column of the image feature information in the width direction.

[0049] In some embodiments, after obtaining a plurality of feature components, the plurality of feature components may be used as input to the pre-generated text recognition model to obtain a recognition result of the text to be recognized.

[0050] In this step, the encoding model in the text recognition model can be used to encode the multiple feature components to obtain global semantic features, and then the target attention weight corresponding to each feature component in the multiple feature components can be determined through the attention layer in the decoding model. The global semantic features are then decoded according to the target attention weight corresponding to each feature component to obtain the recognition result of the text to be recognized. The pre-generated text recognition model can be trained in the following manner: first, a training sample image set containing sample text can be obtained, wherein the text is marked in the training sample image; then, sample image feature information of the training sample image in the training sample image set is obtained; and then, based on the training sample image set, the pre-generated text recognition model is trained to obtain the text recognition model.

[0051] Optionally, the text recognition model includes an encoding model and a decoding model. The encoding model can be used to encode the image feature information to obtain global semantic features; the decoding model can be used to first determine the target attention weight of each feature component in the image feature information; and then decode the global semantic features according to the target attention weight to obtain the recognition result of the text to be recognized.

[0052] In some embodiments, the encoding model can adopt an encoding model based on a BERT model to encode multiple feature components in the image feature information, input the multiple feature components into multiple time steps included in the encoding model respectively, and output global semantic features based on preset time steps in the multiple time steps, wherein the multiple feature components are input into multiple time steps in sequence, each feature component corresponds to a time step, and the preset time step can be the last time step of the encoding model.

[0053] like Figure 3 As shown, Figure 3 This is a schematic diagram of a coding model according to an exemplary embodiment of the present disclosure. For example, when the size of the image feature information is H×W×C, the vector dimension of each feature component may be W, and each feature component may be represented as v1, v2, v3, ..., v w , then each feature component can be input into an input token, and then the maximum pooling operation can be performed along the height direction of the feature component to obtain the token embedding of the input token, and added to the position embedding to obtain the final input. Since the relationship between the two images is not involved in the technical solution disclosed in this disclosure, segment embedding is not required to define it. After that, each input can be input into a time step in the BERT model, and the final global semantic feature h is output based on the last time step of the encoding model.

[0054] In some embodiments, the decoding model can first perform weighted processing on each feature component in the image feature information through the attention layer to obtain the target attention weight corresponding to each component in the image feature information, and then use the global semantic feature and the output of the previous level decoding function as the input of the current decoding function to obtain the decoding results output by multiple decoding functions; then, according to the target attention weight corresponding to each feature component in the image feature information, the decoding results output by multiple decoding functions are weighted to obtain the target decoding results output by multiple decoding functions; finally, according to the target decoding result, the recognition result of the text to be recognized is obtained.

[0055] For example, at the current time step, the attention layer can use the global semantic features and the feature components input at the current time step to perform attention calculations. Since the feature components include the location information of the current focused area, the target attention weight of the current feature component can be obtained based on the global semantic features and the current feature components. The target attention weight of the current feature component can generally be determined using the Self Attention model in the prior art, and this disclosure is not limited to this.

[0056] For example, the decoding model can adopt a decoding model based on the Transformer model. During the decoding process of the decoding model, the decoding model only decodes one character at each time step, and the output of the decoding model in the time step is the input of the decoding model in the next time step. The operation is repeated until the decoding is completed to obtain the recognition result. The decoding process is similar to the encoding process, and the decoding process uses a masked attention mechanism. It should be noted that, in general, there are as many decoders as there are encoders. The decoding model can also include a feedforward neural network. The structure and setting of the feedforward neural network can be based on the network structure in the transformer model commonly used in this field, and the present disclosure does not limit this.

[0057] For example, the final global semantic vector h can first be input into the decoding model to perform weighted processing on each component in the image feature information through the attention layer to obtain the target attention weight corresponding to each component in the image feature information. The target attention weight can be expressed as A1, A2, A3, A4..., respectively. According to the input feature components v1, v2, v3, v4..., and the corresponding target attention weights A1, A2, A3, A4..., the global semantic feature is decoded according to the decoding model to obtain the recognition result of the text to be recognized.

[0058] In some embodiments, the recognition result of the text to be recognized includes multiple predicted characters and the predicted probability corresponding to each of the predicted characters. After obtaining the recognition result of the text to be recognized, it is also possible to determine for each of the multiple predicted probabilities whether the predicted probability is less than or equal to a preset probability threshold; if it is determined that the predicted probability is less than or equal to the preset probability threshold, the predicted character corresponding to the predicted probability is deleted from the recognition result to obtain the target recognition result.

[0059] The above method is adopted to obtain a text image of the text to be recognized; obtain image feature information of the text image; use the image feature information as the input of a pre-generated text recognition model to obtain the recognition result of the text to be recognized; the text recognition model is used to encode the image feature information to obtain a global semantic feature, and determine the target attention weight of each feature component in the image feature information, and decode the global semantic feature according to the target attention weight to obtain the recognition result of the text to be recognized, and the target attention weight is used to characterize the degree of influence of each feature component in the image feature information on the recognition result. In this way, in the process of text recognition, the encoding model in the pre-generated text recognition model can be used to directly recognize the text image to be recognized to avoid the low accuracy of text recognition due to the inaccuracy of the correction network, and the target attention weight of each feature component determined can be used to implicitly model the relationship between character features and characters, so that character information in irregular text images can be more accurately recognized, thereby improving the accuracy of text recognition.

[0060] Figure 4 FIG. 4 is a block diagram of a text recognition device 400 according to an exemplary embodiment of the present disclosure. Figure 4 , the device comprises: A first acquisition module 401 is used to acquire a text image of a text to be recognized; The second acquisition module 402 is used to obtain image feature information of the text image; The recognition module 403 is used to use the image feature information as the input of a pre-generated text recognition model to obtain the recognition result of the text to be recognized; the text recognition model is used to encode the image feature information to obtain global semantic features, and determine the target attention weight of each feature component in the image feature information, and decode the global semantic features according to the target attention weight to obtain the recognition result of the text to be recognized, and the target attention weight is used to characterize the degree of influence of each feature component in the image feature information on the recognition result.

[0061] Optionally, the text recognition model includes an encoding model and a decoding model, wherein the encoding model is used to encode the image feature information to obtain a global semantic feature; The decoding model is used to determine the target attention weight of each feature component in the image feature information; the global semantic feature is decoded according to the target attention weight to obtain the recognition result of the text to be recognized.

[0062] Optionally, the decoding model is used to determine the target attention weight of each feature component in the image feature information in the following manner: Each component in the image feature information is weighted by the attention layer to obtain the target attention weight corresponding to each component in the image feature information.

[0063] Optionally, the decoding model includes a plurality of decoding functions; decoding the global semantic feature according to the target attention weight to obtain a recognition result of the text to be recognized includes: The global semantic feature and the output of the previous level decoding function are used as the input of the current decoding function to obtain the decoding results output by multiple decoding functions; Performing weighted processing on the decoding results output by the multiple decoding functions according to the target attention weight corresponding to each feature component in the image feature information to obtain target decoding results output by the multiple decoding functions; According to the target decoding result, the recognition result of the text to be recognized is obtained.

[0064] Optionally, the second acquisition module 402 is configured to use the text image as input of an image feature extraction model, perform image feature extraction on the text image, and obtain image feature information corresponding to the text image output by the image feature extraction model.

[0065] Optionally, the image feature extraction model includes a residual block, which includes a plurality of residual layers connected in series, and is used to flow the image feature information output by the previous residual layer into the next residual layer; The second acquisition module 402 is configured to input the text image into the residual block to obtain image feature information output by the last residual layer among the multiple residual-connected residual layers.

[0066] Optionally, the text recognition model is trained in the following manner: obtaining a training sample image set containing sample text, wherein the text is marked in the training sample image; obtaining sample image feature information of the training sample images in the training sample image set; and training the pre-generated text recognition model based on the training sample image set to obtain the text recognition model.

[0067] Figure 5 is based on Figure 4 The embodiment shown is a block diagram of a text recognition device. Figure 5 The recognition result of the text to be recognized includes a plurality of predicted characters and a prediction probability corresponding to each of the predicted characters. The device further includes: A determination module 404 is configured to determine, for each of the plurality of prediction probabilities, whether the prediction probability is less than or equal to a preset probability threshold; The target recognition module 405 is used to delete the predicted character corresponding to the predicted probability from the recognition result when it is determined that the predicted probability is less than or equal to the preset probability threshold, so as to obtain a target recognition result.

[0068] The above scheme is adopted, by obtaining a text image of the text to be recognized; obtaining image feature information of the text image; using the image feature information as the input of a pre-generated text recognition model to obtain the recognition result of the text to be recognized; the text recognition model is used to encode the image feature information to obtain a global semantic feature, and determine the target attention weight of each feature component in the image feature information, and decode the global semantic feature according to the target attention weight to obtain the recognition result of the text to be recognized, and the target attention weight is used to characterize the degree of influence of each feature component in the image feature information on the recognition result. In this way, in the process of text recognition, the encoding model in the pre-generated text recognition model can be used to directly recognize the text image to be recognized to avoid the low accuracy of text recognition due to the inaccuracy of the correction network, and the target attention weight of each feature component determined can be used to implicitly model the relationship between character features and characters, so that character information in irregular text images can be more accurately recognized, thereby improving the accuracy of text recognition.

[0069] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0070] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon, which implement the steps of the target detection method provided by the present disclosure when the program instructions are executed by a processor.

[0071] Figure 6 FIG. 6 is a block diagram of an electronic device 600 according to an exemplary embodiment of the present disclosure. Figure 6 As shown, the electronic device 600 may include: a processor 601 , a memory 602 , and may further include one or more of a multimedia component 603 , an input / output (I / O) interface 604 , and a communication component 605 .

[0072] The processor 601 is used to control the overall operation of the electronic device 600 to complete all or part of the steps in the above-mentioned text recognition method. The memory 602 is used to store various types of data to support the operation of the electronic device 600. This data may include, for example, instructions for any application or method operating on the electronic device 600, as well as application-related data such as contact information, sent and received messages, images, audio, video, etc. The memory 602 can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The multimedia component 603 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in the memory 602 or transmitted via the communication component 605. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 604 provides an interface between the processor 601 and other interface modules, which may be a keyboard, a mouse, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 605 is used for wired or wireless communication between the electronic device 600 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, etc., or a combination of one or more thereof, is not limited here. Therefore, the corresponding communication component 605 may include: a Wi-Fi module, a Bluetooth module, an NFC module, etc.

[0073] In an exemplary embodiment, the electronic device 600 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-mentioned text recognition method.

[0074] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided. When the program instructions are executed by a processor, the steps of the above-described text recognition method are implemented. For example, the computer-readable storage medium may be the aforementioned memory 602 including the program instructions. The program instructions may be executed by the processor 601 of the electronic device 600 to perform the above-described text recognition method.

[0075] In another exemplary embodiment, a computer program product is further provided. The computer program product includes a computer program that can be executed by a programmable device, and the computer program has a code portion for performing the above-mentioned text recognition method when executed by the programmable device.

[0076] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings. However, the present disclosure is not limited to the specific details of the above embodiments. Within the technical concept of the present disclosure, various simple modifications can be made to the technical solutions of the present disclosure, and these simple modifications all fall within the scope of protection of the present disclosure.

[0077] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.

[0078] In addition, the various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.

Claims

1. A text recognition method, characterized in that: The method comprises: Obtain a text image of the text to be recognized; Acquiring image feature information of the text image; The image feature information is used as the input of a pre-generated text recognition model to obtain a recognition result of the text to be recognized; the text recognition model is used to encode the image feature information to obtain a global semantic feature, and determine the target attention weight of each feature component in the image feature information, and decode the global semantic feature according to the target attention weight to obtain a recognition result of the text to be recognized, and the target attention weight is used to characterize the degree of influence of each feature component in the image feature information on the recognition result.

2. The method according to claim 1, characterized in that The text recognition model includes an encoding model and a decoding model, wherein the encoding model is used to encode the image feature information to obtain global semantic features; The decoding model is used to determine the target attention weight of each feature component in the image feature information; decode the global semantic feature according to the target attention weight to obtain the recognition result of the text to be recognized.

3. The method according to claim 2, characterized in that The decoding model is used to determine the target attention weight of each feature component in the image feature information in the following manner: Each component in the image feature information is weighted through the attention layer to obtain the target attention weight corresponding to each component in the image feature information.

4. The method according to claim 2, characterized in that The decoding model includes a plurality of decoding functions; decoding the global semantic features according to the target attention weight to obtain the recognition result of the text to be recognized includes: Using the global semantic features and the output of the previous level decoding function as the input of the current decoding function, and obtaining decoding results output by multiple decoding functions; performing weighted processing on the decoding results output by the multiple decoding functions according to the target attention weight corresponding to each feature component in the image feature information to obtain target decoding results output by the multiple decoding functions; A recognition result of the text to be recognized is obtained according to the target decoding result.

5. The method according to claim 1, wherein The acquiring of image feature information of the text image includes: The text image is used as the input of an image feature extraction model, image feature extraction is performed on the text image, and image feature information corresponding to the text image output by the image feature extraction model is obtained.

6. The method according to claim 5, characterized in that The image feature extraction model includes a residual block, which includes multiple residual layers connected in series, and is used to flow the image feature information output by the previous residual layer into the next residual layer; The step of using the text image as an input of an image feature extraction model, performing image feature extraction on the text image, and obtaining image feature information corresponding to the text image output by the image feature extraction model includes: The text image is input into the residual block to obtain image feature information output by the last residual layer in the residual layers of the multiple residual connections.

7. The method according to claim 1, characterized in that The text recognition model is trained in the following way: Obtaining a set of training sample images containing sample texts, wherein the texts are marked in the training sample images; Acquire sample image feature information of the training sample images in the training sample image set; The pre-generated text recognition model is trained based on the training sample image set to obtain the text recognition model.

8. The method according to any one of claims 1 to 6, characterized in that The recognition result of the text to be recognized includes a plurality of predicted characters and a prediction probability corresponding to each of the predicted characters. The method further includes: For each predicted probability of the plurality of predicted probabilities, determining whether the predicted probability is less than or equal to a preset probability threshold; When it is determined that the predicted probability is less than or equal to the preset probability threshold, the predicted character corresponding to the predicted probability is deleted from the recognition result to obtain the target recognition result.

9. A text recognition device, characterized in that: The device comprises: A first acquisition module is used to acquire a text image of a text to be recognized; A second acquisition module is used to acquire image feature information of the text image; A recognition module is used to use the image feature information as input to a pre-generated text recognition model to obtain a recognition result of the text to be recognized; the text recognition model is used to encode the image feature information to obtain a global semantic feature, and determine the target attention weight of each feature component in the image feature information, and decode the global semantic feature according to the target attention weight to obtain a recognition result of the text to be recognized, and the target attention weight is used to characterize the degree of influence of each feature component in the image feature information on the recognition result.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

11. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 8.

Images