Text structured information determination method, device, equipment and storage medium

Abstract

The present disclosure provides a method, apparatus, device, and storage medium for determining text structured information, relating to the fields of artificial intelligence technology, specifically deep learning, image processing, and computer vision technology, and can be applied to scenarios such as OCR. A specific implementation scheme comprises: determining the visual features of a field image and the initial text recognition result of the field image; correcting the initial text recognition result based on the visual features and the initial text recognition result to obtain a corrected text recognition result; and determining the text structured information of the field image based on the field category corresponding to the field image and the corrected text recognition result. The above-described technical solution can improve the accuracy of determining text structured information.

Description

Technical Field

[0001] The present disclosure relates to the fields of artificial intelligence technology, specifically deep learning, image processing, and computer vision technology, and can be applied to scenarios such as OCR. Background Art

[0002] Documents are an important means of storing information, containing a wealth of structured information. Capturing this structured information facilitates the construction of massive databases for data storage and management. However, currently available documents are typically in the form of images, which inevitably lead to errors when recognizing text in these images. Therefore, leveraging semantic information to correct recognized content and achieve more accurate parsing results is a current challenge. Summary of the Invention

[0003] The present disclosure provides a method, apparatus, device and storage medium for determining text structured information.

[0004] According to one aspect of the present disclosure, a method for determining text structured information is provided, the method comprising:

[0005] determining visual features of a field image and an initial text recognition result of the field image;

[0006] Correcting the initial text recognition result according to the visual feature and the initial text recognition result to obtain a corrected text recognition result;

[0007] The text structure information of the field image is determined according to the field category corresponding to the field image and the corrected text recognition result.

[0008] According to another aspect of the present disclosure, a device for determining text structured information is provided, the device comprising:

[0009] an initial text result determination module, configured to determine visual features of a field image and an initial text recognition result of the field image;

[0010] a corrected text result determination module, configured to correct the initial text recognition result based on the visual features and the initial text recognition result to obtain a corrected text recognition result;

[0011] The structured information determination module is used to determine the text structured information of the field image according to the field category corresponding to the field image and the corrected text recognition result.

[0012] According to another aspect of the present disclosure, an electronic device is provided, the electronic device comprising:

[0013] at least one processor; and

[0014] a memory communicatively connected to the at least one processor; wherein,

[0015] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the text structured information determination method described in any embodiment of the present disclosure.

[0016] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the method for determining text structured information according to any embodiment of the present disclosure.

[0017] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, which, when executed by a processor, implements the method for determining text structured information according to any embodiment of the present disclosure.

[0018] According to the technology disclosed in the present invention, the accuracy of determining text structured information can be improved.

[0019] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0021] Figure 1 is a flowchart of a method for determining text structured information provided according to an embodiment of the present disclosure;

[0022] Figure 2 is a flowchart of another method for determining text structured information provided according to an embodiment of the present disclosure;

[0023] Figure 3 is a flowchart of another method for determining text structured information provided according to an embodiment of the present disclosure;

[0024] Figure 4 is a structural diagram of a device for determining text structured information provided according to an embodiment of the present disclosure;

[0025] Figure 5 It is a block diagram of an electronic device used to implement the method for determining text structured information according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0026] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0027] It should be noted that the terms "initial", "corrected", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable where appropriate so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] In addition, it should be noted that the collection, storage, use, processing, transmission, provision and disclosure of text images and field images involved in the technical solution of the present invention comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0029] Figure 1 This is a flowchart of a method for determining text structured information according to an embodiment of the present disclosure. This embodiment is applicable to the situation of how to more accurately determine the structured information of a document in an image-based document scenario. The method can be executed by a text structured information determination device, which can be implemented in software and / or hardware and can be integrated into an electronic device that carries the text structured information determination function, such as a server. Figure 1 As shown, the text structured information determination method of this embodiment may include:

[0030] S101 : Determine visual features of a field image and an initial text recognition result of the field image.

[0031] In this embodiment, a field image refers to an image containing a field; wherein the field may be text content at the character level, text line level, or paragraph level; for example, it may be a single field, text line, or paragraph.

[0032] The so-called visual features are used to characterize the image-level features of field images. They can be shallow features such as stroke structure, or deep features such as the features of deep network outputs in convolutional neural networks. They can be represented in the form of vectors or matrices.

[0033] The so-called initial text recognition result refers to the result of initial recognition of text in the field image.

[0034] Alternatively, a preset image feature extraction method may be used to extract features from the field image to obtain visual features of the field image. The preset image feature extraction method may be an image feature extraction algorithm well known to those skilled in the art, such as an image feature extraction model based on a machine learning algorithm or a deep learning algorithm.

[0035] Another optional approach is to use a text recognition model to recognize the field image and obtain an initial text recognition result of the field image. The text recognition model can be a model for text recognition known in the art.

[0036] Furthermore, the features output by the hidden layer in the text recognition model can be used as visual features of the field image.

[0037] S102: Correcting the initial text recognition result according to the visual features and the initial text recognition result to obtain a corrected text recognition result.

[0038] In this embodiment, the corrected text recognition result refers to a text result obtained by correcting the inaccurately recognized characters in the initial text recognition result.

[0039] Specifically, the initial text recognition result can be vectorized to obtain a processed initial text recognition result, and then the visual features and the processed initial text recognition result can be spliced ​​together to obtain spliced ​​features. The spliced ​​features can then be processed based on a correction model to obtain a corrected text recognition result. The correction model can be a pre-trained neural network model.

[0040] S103: Determine text structure information of the field image according to the field category corresponding to the field image and the corrected text recognition result.

[0041] In this embodiment, the text structured information may refer to document information in the form of key-value pairs. For example, the key is hospital, and the corresponding key value is XXX hospital.

[0042] Specifically, the corrected text recognition results of the field categories corresponding to the field images may be matched to serve as text structured information of the field images.

[0043] The technical solution provided by the embodiments of the present disclosure determines the visual features of a field image and the initial text recognition result for the field image. It then corrects the initial text recognition result based on the visual features and the initial text recognition result to obtain a corrected text recognition result. Furthermore, the text structured information of the field image is determined based on the field category corresponding to the field image and the corrected text recognition result. This technical solution, which combines the visual features of the field image with the text recognition result of the field itself to correct the text recognition result of the field image, enables more accurate end-to-end extraction of the document's structured information.

[0044] Based on the above embodiment, as an optional method of the present disclosure, before determining the visual features of the field image and the initial text recognition result of the field image, it is also possible to perform field detection on the text image to obtain the field category and position information of the field; and determine the field image of the field from the text image based on the position information.

[0045] The text image refers to an image containing text. The field category refers to the type information of the field. The position information refers to the position information of the field in the text image, which can be the vertex coordinate information of the bounding box containing the field.

[0046] Specifically, a text detection model can be used to perform field detection on a text image to obtain the field category and location information of the field. Furthermore, based on the field location information, the region containing each field can be segmented from the text image to obtain a field image of the field. The text detection model can be a model obtained by training a convolutional neural network.

[0047] It can be understood that compared with the solution of extracting text structured information from the entire text image, the present disclosure first cuts out the field image from the text image, and then determines the text structured information of the field image, that is, determines the text structured information from a finer-grained field image, avoiding the mutual influence between fields, thereby making the determination of the text structured information more accurate.

[0048] Figure 2 This is a flowchart of another method for determining text structured information provided by an embodiment of the present disclosure. Based on the above embodiment, this embodiment further optimizes "correcting the initial text recognition result based on visual features and the initial text recognition result to obtain a corrected text recognition result" and provides an optional implementation scheme. Figure 2 As shown, the text structured information determination method of this embodiment may include:

[0049] S201 : Determine visual features of a field image and an initial text recognition result of the field image.

[0050] S202: Perform feature extraction on the initial text recognition result to obtain embedded features of the initial text recognition result.

[0051] In this embodiment, the embedded features refer to semantic features obtained after processing the initial text recognition results, and can be represented in the form of a matrix or a vector.

[0052] Alternatively, the initial text recognition results can be fed into a text feature extraction model, which can then be trained to obtain embedded features of the initial text recognition results. The text feature extraction model can be pre-trained using a convolutional neural network.

[0053] S203: Correcting the initial text recognition result based on the visual features and the embedded features to obtain a corrected text recognition result.

[0054] An optional approach is to fuse the visual features and embedded features and use the fused features to correct the initial text recognition results to obtain corrected text recognition results. For example, the visual features and embedded features can be concatenated and then fed back into the text recognition model to obtain corrected text recognition results.

[0055] S204: Determine text structure information of the field image according to the field category corresponding to the field image and the corrected text recognition result.

[0056] The technical solution provided by the embodiments of the present disclosure determines the visual features of a field image and the initial text recognition result of the field image, then performs feature extraction on the initial text recognition result to obtain embedded features of the initial text recognition result. Based on the visual features and the embedded features, the initial text recognition result is corrected to obtain a corrected text recognition result. Furthermore, based on the field category corresponding to the field image and the corrected text recognition result, the text structured information of the field image is determined. This technical solution introduces embedded features, namely semantic features, and combines visual and semantic features to correct the initial text recognition result, thereby obtaining more accurate text structured information.

[0057] On the basis of the above embodiments, as an optional method of the present disclosure, feature extraction is performed on the initial text recognition result to obtain embedded features of the initial text recognition result. The initial text recognition result can be encoded to obtain encoded features of the initial text recognition result; and the encoded features are mapped to obtain embedded features of the initial text recognition result.

[0058] Specifically, the initial text recognition result can be encoded based on a preset encoding rule to obtain the encoding features of the initial text recognition result. The encoding features are then mapped using a fully connected layer, and the features output by the fully connected layer are used as embedded features of the initial text recognition result.

[0059] It can be understood that encoding and mapping the initial text recognition results can fully learn the semantic information of the initial text recognition results and the correlation between the characters therein, so that the obtained embedded features can more accurately reflect the initial text recognition results, thereby laying the foundation for correcting the initial text recognition results.

[0060] Figure 3 This is a flowchart of another method for determining text structured information provided by an embodiment of the present disclosure. Based on the above embodiment, this embodiment further optimizes "correcting the initial text recognition result according to visual features and embedded features to obtain a corrected text recognition result" and provides an optional method. Figure 3 As shown, the text structured information determination method of this embodiment may include:

[0061] S301 : Determine visual features of a field image and an initial text recognition result of the field image.

[0062] S302: extract features from the initial text recognition result to obtain embedded features of the initial text recognition result.

[0063] S303: Determine multimodal features based on the visual features and the embedded features.

[0064] S304: Correct the initial text recognition result based on the multimodal features to obtain a corrected text recognition result.

[0065] In this embodiment, multimodal features refer to features that fuse different modalities and can be represented in the form of vectors or matrices.

[0066] An optional approach is to scale-normalize the visual features and embedded features, then superimpose the normalized visual features and embedded features, using the superimposed features as multimodal features. The multimodal features are then fed back into the text recognition model to obtain a corrected text recognition result based on the initial text recognition result.

[0067] S305 : Determine text structure information of the field image according to the field category corresponding to the field image and the corrected text recognition result.

[0068] The technical solution provided by the embodiments of the present disclosure determines the visual features of a field image and the initial text recognition result of the field image, then performs feature extraction on the initial text recognition result to obtain embedded features of the initial text recognition result. Based on the visual features and the embedded features, multimodal features are determined. Based on the multimodal features, the initial text recognition result is corrected to obtain a corrected text recognition result. Furthermore, based on the field category corresponding to the field image and the corrected text recognition result, the text structured information of the field image is determined. The above technical solution introduces multimodal features and fully integrates visual features such as stroke structure with text semantic features to correct the initial text recognition result, thereby making the correction result more accurate.

[0069] On the basis of the above embodiments, as an optional method of the present disclosure, the initial text recognition result is corrected according to the multimodal features to obtain a corrected text recognition result, including: determining the correction features of the initial text recognition result according to the multimodal features; determining the corrected text recognition result of the initial text recognition result according to the correction features.

[0070] The correction feature may be a feature corresponding to each character in the initial text recognition result, and may be represented in a matrix or vector form; for example, it may be a correction probability vector corresponding to each character.

[0071] Specifically, the multimodal features can be input into the correction model to obtain the correction features of the initial text recognition results, that is, the correction probability vector corresponding to each character in the initial text recognition results. Then, the correction probability vector is converted into text to obtain the corrected text recognition results. The correction model can be a deep learning model, such as a transformer model.

[0072] It can be understood that text correction based on multimodal features not only fully utilizes the characteristics of the characters themselves, such as stroke structure, but also fully utilizes the semantic information of the text and its contextual relevance.

[0073] Figure 4 This is a schematic diagram of a text structured information determination device provided according to an embodiment of the present disclosure. This embodiment is applicable to the situation of how to more accurately determine the structured information of a document in an image-based document scenario. The device can be implemented in software and / or hardware and can be integrated into an electronic device that carries the text structured information determination function, such as a server. Figure 4 As shown, the text structured information determination device 400 includes:

[0074] An initial text result determination module 401 is used to determine visual features of a field image and an initial text recognition result of the field image;

[0075] The corrected text result determination module 402 is configured to correct the initial text recognition result based on the visual features and the initial text recognition result to obtain a corrected text recognition result;

[0076] The structured information determination module 403 is configured to determine the text structured information of the field image according to the field category corresponding to the field image and the corrected text recognition result.

[0077] The technical solution provided by the embodiments of the present disclosure determines the visual features of a field image and the initial text recognition result for the field image. It then corrects the initial text recognition result based on the visual features and the initial text recognition result to obtain a corrected text recognition result. Furthermore, the text structured information of the field image is determined based on the field category corresponding to the field image and the corrected text recognition result. This technical solution, which combines the visual features of the field image with the text recognition result of the field itself to correct the text recognition result of the field image, enables more accurate end-to-end extraction of the document's structured information.

[0078] Furthermore, the device also includes:

[0079] The field detection module is used to detect fields in text images and obtain the field category and position information of the fields;

[0080] The field image determination module is used to determine the field image of the field from the text image according to the position information.

[0081] Furthermore, the text correction result determination module 402 includes:

[0082] An embedding feature determination unit, configured to extract features from the initial text recognition result to obtain an embedding feature of the initial text recognition result;

[0083] The corrected text result determination unit is used to correct the initial text recognition result based on the visual features and the embedded features to obtain a corrected text recognition result.

[0084] Furthermore, the embedding feature determination unit is specifically configured to:

[0085] Encoding the initial text recognition result to obtain encoding features of the initial text recognition result;

[0086] The encoded features are mapped to obtain the embedded features of the initial text recognition results.

[0087] Furthermore, the text correction result determination unit includes:

[0088] A multimodal feature determination subunit, configured to determine multimodal features based on visual features and embedded features;

[0089] The corrected text result determination subunit is used to correct the initial text recognition result according to the multimodal features to obtain the corrected text recognition result.

[0090] Furthermore, the text correction result determination subunit is specifically used for:

[0091] Determine the correction features of the initial text recognition results based on the multimodal features;

[0092] A corrected text recognition result of the initial text recognition result is determined based on the corrected features.

[0093] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0094] Figure 5 is a block diagram of an electronic device for implementing the method for determining text structured information according to an embodiment of the present disclosure; Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0095] like Figure 5 As shown, the electronic device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. Various programs and data required for the operation of the electronic device 500 can also be stored in the RAM 503. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0096] Multiple components in the electronic device 500 are connected to the I / O interface 505, including: an input unit 506, such as a keyboard, a mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a magnetic disk, an optical disk, etc.; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows the electronic device 500 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0097] The computing unit 501 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as the text structured information determination method. For example, in some embodiments, the text structured information determination method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 508. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the text structured information determination method described above can be performed. Alternatively, in other embodiments, the computing unit 501 can be configured to perform the text structured information determination method in any other suitable manner (e.g., via firmware).

[0098] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0099] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0100] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0101] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0102] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0103] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0104] Artificial intelligence (AI) is the study of how computers can simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It encompasses both hardware and software technologies. AI hardware technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, and big data processing. AI software technologies primarily encompass computer vision, speech recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graphs.

[0105] Cloud computing refers to a technology system that provides network access to elastically scalable shared pools of physical or virtual resources. These resources can include servers, operating systems, networks, software, applications, and storage devices, and can be deployed and managed on-demand in a self-service manner. Cloud computing technology provides efficient and powerful data processing capabilities for the application of technologies such as artificial intelligence and blockchain, as well as for model training.

[0106] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0107] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A method for determining text structured information, comprising: determining visual features of a field image and an initial text recognition result of the field image; Encoding the initial text recognition result to obtain an encoding feature of the initial text recognition result; Mapping the encoded features to obtain embedded features of the initial text recognition result; determining a multimodal feature based on the visual feature and the embedded feature; Correcting the initial text recognition result according to the multimodal feature to obtain a corrected text recognition result; The text structure information of the field image is determined according to the field category corresponding to the field image and the corrected text recognition result.

2. The method according to claim 1, wherein Before determining the visual features of the field image and the initial text recognition result of the field image, the method further includes: Perform field detection on text images to obtain field category and position information of the fields; A field image of the field is determined from the text image according to the position information.

3. The method according to claim 1, wherein Correcting the initial text recognition result according to the multimodal feature to obtain a corrected text recognition result includes: determining, based on the multimodal features, correction features of the initial text recognition result; A corrected text recognition result of the initial text recognition result is determined according to the correction feature.

4. A device for determining text structured information, comprising: an initial text result determination module, configured to determine visual features of a field image and an initial text recognition result of the field image; A correction text result determination module, the correction text result determination module comprising: an embedded feature determination unit, configured to encode the initial text recognition result to obtain an encoding feature of the initial text recognition result; Mapping the encoded features to obtain embedded features of the initial text recognition result; Correction text result determination unit, used for determining a multimodal feature based on the visual feature and the embedded feature; Correcting the initial text recognition result according to the multimodal feature to obtain a corrected text recognition result; The structured information determination module is used to determine the text structured information of the field image according to the field category corresponding to the field image and the corrected text recognition result.

5. The device according to claim 4, wherein The device further comprises: The field detection module is used to detect fields in text images and obtain the field category and position information of the fields; A field image determining module is configured to determine the field image of the field from the text image according to the position information.

6. The device according to claim 4, wherein The corrected text result determination subunit is specifically used for: determining, based on the multimodal features, correction features of the initial text recognition result; A corrected text recognition result of the initial text recognition result is determined according to the correction feature.

7. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the method for determining text structured information according to any one of claims 1 to 3.

8. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to enable a computer to execute the method for determining text structured information according to any one of claims 1 to 3.

9. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the method for determining text structured information according to any one of claims 1 to 3.

Images