Target field extraction method, system, terminal and medium

Abstract

The present invention provides a target field extraction method, system, terminal, and medium. The method comprises: obtaining a medical record image of a medical record document, performing text recognition on the medical record image, and obtaining a text recognition result; inputting the text information, orientation information, and visual information in the text recognition result into a multimodal pre-trained model for entity prediction to obtain an entity list; performing field alignment based on the row information in the text recognition result and the entity list to obtain a key-value entity pair, and querying a pre-set target field; and performing field extraction on the medical record document based on the target field and the key-value entity pair to obtain the target field value of the medical record document. The present invention achieves the effect of multimodal entity prediction by inputting the text information, orientation information, and visual information in the text recognition result into a multimodal pre-trained model for entity prediction, thereby improving the accuracy of entity prediction, reducing information granularity, eliminating the need for a large amount of training data, and improving the efficiency of target field extraction.

Description

Technical Field

[0001] The present invention relates to the field of medical technology, and in particular to a target field extraction method, system, terminal and medium. Background Art

[0002] Optical Character Recognition (OCR) refers to the process by which an electronic device (such as a scanner or digital camera) examines characters printed on paper, determines their shape by detecting dark and light patterns, and then uses character recognition methods to translate these shapes into computer text. That is, for printed characters, OCR uses an optical method to convert the text in a paper document into a black and white dot matrix image file, and then uses recognition software to convert the text in the image into text format for further editing and processing by word processing software.

[0003] The traditional medical industry involves a large amount of document information, such as registration forms, test forms, prescription forms, outpatient treatment manuals, inpatient medical records, etc. Most of this information exists in various hospital information systems and is accessed by users in paper form or on screen. For medical workers, there is a need to standardize and preserve it. Therefore, the problem of target field extraction is receiving more and more attention.

[0004] In the existing target field extraction process, unstructured full-text information extraction is generally used to extract target fields to support information extraction tasks in different layout formats. However, due to the unstructured extraction of full documents, the information granularity is too coarse and a large amount of training data is required to adapt to different scenarios, resulting in low target field extraction efficiency. Summary of the Invention

[0005] The purpose of the embodiments of the present invention is to provide a target field extraction method, system, terminal and medium, aiming to solve the problem of low target field extraction efficiency in the existing target field extraction process.

[0006] The embodiment of the present invention is implemented as follows: a target field extraction method, the method comprising:

[0007] Acquire a medical record image of a medical record document, and perform text recognition on the medical record image to obtain a text recognition result;

[0008] Inputting the text information, orientation information, and visual information in the text recognition result into a multimodal pre-training model for entity prediction to obtain an entity list;

[0009] Performing field alignment based on the row information in the text recognition result and the entity list to obtain key-value entity pairs, and querying a preset target field;

[0010] Field extraction is performed on the medical record document according to the target field and the key-value entity pair to obtain the target field value of the medical record document.

[0011] Preferably, extracting fields from the medical record document based on the target field and the key-value entity pair includes:

[0012] Get the Prompt template and adjust the parameters according to the target field;

[0013] Matching the key-value entity pairs according to the target field and the Prompt template after parameter adjustment, and marking the matched key-value entity pairs;

[0014] The key value in the marked key-value entity pair is obtained to obtain the target field value.

[0015] Preferably, performing text recognition on the medical record image to obtain a text recognition result includes:

[0016] Performing image preprocessing on the medical record image, and binarizing the preprocessed medical record image;

[0017] performing denoising on the binarized medical record image, and performing tilt correction on the denoised medical record image;

[0018] Performing character segmentation on the tilt-corrected medical record image, and performing character recognition on the character segmented medical record image to obtain text information;

[0019] The position information of each character in the text information is obtained to obtain the orientation information, and convolution processing is performed on the image corresponding to the text information to obtain the visual information.

[0020] Preferably, the performing field alignment based on the row information in the text recognition result and the entity list includes:

[0021] According to the row information, respectively obtain the row number value of each entity in the entity list, and respectively obtain the label information of each entity, the label information including the type label, the position label and the association label;

[0022] Field alignment is performed on the entity list according to the tag information and the row number value to obtain the key-value entity pair.

[0023] Preferably, after obtaining the key value in the marked key-value entity pair and obtaining the target field value, the step further includes:

[0024] Query the character filling position of each target field value in the Prompt template respectively, and query the keyword of each target field value in the corresponding key-value entity pair respectively;

[0025] Fill in the characters of the queried keyword in the Prompt template according to the character filling position;

[0026] If the keyword is not found in the target field value corresponding to the key-value entity pair, the preset characters are filled in the Prompt template according to the character filling position.

[0027] Preferably, before inputting the text information, orientation information and visual information in the text recognition result into the multimodal pre-training model for entity prediction, the method further includes:

[0028] Performing text recognition on the text image of each sample text respectively to obtain sample text, sample orientation, and sample vision, and inputting the sample text, the sample orientation, and the sample vision into the multimodal pre-training model for entity prediction to obtain an entity prediction list;

[0029] Querying the entity truth list of each sample text, and determining the model loss based on the entity truth list and the entity prediction list;

[0030] Parameters of the multimodal pre-trained model are updated according to the model loss until the multimodal pre-trained model converges, and the converged multimodal pre-trained model is output.

[0031] Another object of an embodiment of the present invention is to provide a target field extraction system, the system comprising:

[0032] A text recognition module is used to obtain a medical record image of a medical record document and perform text recognition on the medical record image to obtain a text recognition result;

[0033] An entity prediction module, configured to input text information, orientation information, and visual information from the text recognition result into a multimodal pre-trained model for entity prediction to obtain an entity list;

[0034] A field alignment module, configured to perform field alignment based on the row information in the text recognition result and the entity list, obtain key-value entity pairs, and query a preset target field;

[0035] The field extraction module is used to extract fields from the medical record document according to the target field and the key-value entity pair to obtain the target field value of the medical record document.

[0036] Preferably, the field extraction module is further used to:

[0037] Get the Prompt template and adjust the parameters according to the target field;

[0038] Matching the key-value entity pairs according to the target field and the Prompt template after parameter adjustment, and marking the matched key-value entity pairs;

[0039] The key value in the marked key-value entity pair is obtained to obtain the target field value.

[0040] Another object of an embodiment of the present invention is to provide a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0041] Another object of an embodiment of the present invention is to provide a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0042] In an embodiment of the present invention, text information, orientation information and visual information in the text recognition results are input into a multimodal pre-training model for entity prediction, thereby achieving the effect of multimodal prediction of entities, improving the accuracy of entity prediction, reducing information granularity, eliminating the need for a large amount of training data, improving the efficiency of target field extraction, and performing field alignment based on the row information and entity list in the text recognition results, thereby improving the accuracy of field extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 is a flow chart of a target field extraction method provided by the first embodiment of the present invention;

[0044] Figure 2 is a flow chart of a target field extraction method provided by a second embodiment of the present invention;

[0045] Figure 3 is a schematic structural diagram of a target field extraction system provided by a third embodiment of the present invention;

[0046] Figure 4 It is a schematic structural diagram of a terminal device provided in the fourth embodiment of the present invention. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0048] In order to illustrate the technical solution of the present invention, specific embodiments are provided below.

[0049] Example 1

[0050] See also Figure 1 , is a flow chart of a target field extraction method provided by the first embodiment of the present invention. The target field extraction method can be applied to any terminal device or system. The target field extraction method includes the following steps:

[0051] Step S10, obtaining a medical record image of a medical record document, and performing text recognition on the medical record image to obtain a text recognition result;

[0052] The medical record image can be acquired based on any device having an image acquisition module, and text recognition is performed on the medical record image to obtain text information, orientation information, and visual information in the medical record image. The text information includes character information, the orientation information includes the one-dimensional and two-dimensional coordinates of each character in the medical record image, and the visual information is used to represent the convolution features of the image.

[0053] Optionally, in this step, performing text recognition on the medical record image to obtain a text recognition result includes:

[0054] Performing image preprocessing on the medical record image, and binarizing the preprocessed medical record image; wherein, by preprocessing the medical record image, the image quality of the medical record image can be effectively improved;

[0055] performing denoising on the binarized medical record image and performing tilt correction on the denoised medical record image; wherein the denoising is performed on the binarized medical record image to reduce noise data in the medical record image, and the tilt correction is performed on the denoised medical record image to achieve an effect of correcting the position of each character in the medical record image;

[0056] Performing character segmentation on the tilt-corrected medical record image, and performing character recognition on the character segmented medical record image to obtain text information; wherein, performing character segmentation on the tilt-corrected medical record image facilitates the recognition operation of each character in the medical record image and improves the accuracy of character recognition;

[0057] The position information of each character in the text information is obtained to obtain the orientation information, and convolution processing is performed on the image corresponding to the text information to obtain the visual information.

[0058] Step S20: inputting the text information, orientation information, and visual information in the text recognition result into a multimodal pre-training model for entity prediction to obtain an entity list;

[0059] Among them, trimodal information is composed of text information, orientation information and visual information, and the trimodal information is used as input information of the multimodal pre-training model for entity prediction to obtain the entity list. In this step, by fusing the three modal information of text information, orientation information and visual information, the medical record image can be effectively visually structured.

[0060] Step S30, performing field alignment based on the row information in the text recognition result and the entity list to obtain key-value entity pairs, and querying a preset target field;

[0061] Among them, by aligning the fields with the row information and entity list in the recognition results, a structured result of key-value entity pairs (KV pairs) can be effectively formed, which reduces the complexity for subsequent field extraction. In this step, the target field can be set according to needs. The target field is used to represent the predefined extraction field name.

[0062] Optionally, in this step, aligning fields based on the row information in the text recognition result and the entity list includes:

[0063] According to the row information, respectively obtain the row number value of each entity in the entity list, and respectively obtain the label information of each entity; wherein the label information includes a type label, a position label, and an association label;

[0064] Performing field alignment on the entity list according to the tag information and the row number value to obtain the key-value entity pair;

[0065] This embodiment also optimizes the KV tag system. The traditional Key and Value tag systems are expanded into a tag system with alignment information. The tags used in this embodiment are:

[0066] HEADER-document header, KEY-RIGHT-key whose value is on the right, KEY-DOWN-key whose value is on the bottom;

[0067] KEY-KEY-RIGHT - the parent KEY of the right KEY, KEY-KEY-DOWN - the parent KEY of the bottom KEY, VALUE - value;

[0068] The type tag is used to represent the document header, the position tags include KEY-RIGHT and KEY-DOWN, and the association tags include KEY-KEY-RIGHT and KEY-KEY-DOWN. In this step, the tabular fields or non-tabular fields in the medical record image can be effectively and automatically aligned through the tags with alignment information to obtain the key-value entity pair.

[0069] Step S40, extracting fields from the medical record document according to the target field and the key-value entity pair to obtain the target field value of the medical record document;

[0070] Among them, based on the target field and key-value entity pair, the field of the medical record document can be effectively extracted to obtain the target field value of the medical record document;

[0071] Optionally, in this step, extracting fields from the medical record document based on the target field and the key-value entity pair includes:

[0072] Get the Prompt template and adjust the parameters according to the target field;

[0073] Matching the key-value entity pairs according to the target field and the Prompt template after parameter adjustment, and marking the matched key-value entity pairs;

[0074] Obtain the key value in the marked key-value entity pair to obtain the target field value;

[0075] Among them, the target field is extracted based on the Prompt method. After obtaining the aligned KV entity pairs, the value of the target field is filtered out through the Prompt method according to the predefined extraction field name. In this step, a unified Prompt template is used: "[CLS]KEY[SEP]VALUE is TARGET".

[0076] Furthermore, in this step, after obtaining the key value in the marked key-value entity pair and obtaining the target field value, the step further includes:

[0077] Query the character filling position of each target field value in the Prompt template respectively, and query the keyword of each target field value in the corresponding key-value entity pair respectively;

[0078] Fill in the characters of the queried keyword in the Prompt template according to the character filling position;

[0079] If the keyword is not found in the corresponding key-value entity pair for the target field value, the preset characters are filled in the Prompt template according to the character filling position;

[0080] Among them, the field identifier of the target field corresponding to each target field value is matched with the pre-stored position query table to obtain the character filling position. The position query table stores the correspondence between different field identifiers and the corresponding character filling positions. Based on the character filling position, the queried keyword is filled in the Prompt template to complete the character filling task in the Prompt template.

[0081] In this embodiment, field extraction also includes:

[0082] a) Fine-tuning stage: Only fields with strong domain characteristics are fine-tuned on the Chinese pre-trained language model (such as BERT or GPT), such as "diagnosis name," "medical order," and "charge item name," and fine-tuned according to the Prompt template.

[0083] b) Prediction phase: Based on the Prompt template, query all aligned KV results for the target field TARGET in the configuration using the template above, and return all VALUEs corresponding to the Prompt model prediction of YES. If the VALUE does not have a corresponding KEY, the KEY information in the template is represented by an empty string.

[0084] Among them, the use of the KV pair-based Prompt solution to extract target fields effectively utilizes the knowledge of the Prompt native model of general domain corpus, and then fine-tunes it with the corpus knowledge of special fields to effectively support the needs of adding changes to the subsequent target fields of the project.

[0085] In this embodiment, entity prediction is performed by inputting text information, orientation information and visual information in the text recognition results into a multimodal pre-training model to achieve the effect of multimodal entity prediction, thereby improving the accuracy of entity prediction, reducing information granularity, eliminating the need for a large amount of training data, and improving the efficiency of target field extraction. Field alignment is performed based on the row information and entity list in the text recognition results, thereby improving the accuracy of field extraction.

[0086] Example 2

[0087] See also Figure 2 , is a flow chart of a secret embedding and sharing method provided by a second embodiment of the present invention. This embodiment is used to further refine the steps before step S20 in the first embodiment, including the steps of:

[0088] Step S50, performing text recognition on the text image of each sample text respectively to obtain sample text, sample orientation and sample vision, and inputting the sample text, the sample orientation and the sample vision into the multimodal pre-trained model for entity prediction to obtain an entity prediction list;

[0089] The sample text can be set according to the needs, and the entity prediction effect of each sample text can be achieved by inputting the sample text, sample orientation and sample vision into the multimodal pre-training model;

[0090] Step S60, querying the entity true list of each sample text, and determining the model loss based on the entity true list and the entity predicted list;

[0091] Among them, the model loss is determined based on the real entity list and the predicted entity list, which can effectively calculate the entity prediction error of the multimodal pre-trained model;

[0092] Step S70, updating the parameters of the multimodal pre-trained model according to the model loss until the multimodal pre-trained model converges, and outputting the converged multimodal pre-trained model;

[0093] Among them, the parameters of the multimodal pre-training model are updated based on the model loss to improve the accuracy of the multimodal pre-training model. When the number of iterations of the multimodal pre-training model is greater than or equal to the number threshold, or the calculated model loss is less than the loss threshold, the multimodal pre-training model is judged to have converged.

[0094] In this embodiment, text recognition is performed on the text image of each sample text respectively to obtain sample text, sample orientation and sample vision. Based on the sample text, sample orientation and sample vision, the training data of the multimodal pre-training model can be effectively guaranteed. The model loss is determined based on the entity real list and the entity prediction list, and the entity prediction error of the multimodal pre-training model can be effectively calculated. The parameters of the multimodal pre-training model are updated based on the model loss until the multimodal pre-training model converges. The converged multimodal pre-training model can effectively integrate information of the three modes of text information, orientation information and visual information to obtain the entity list of the medical record document.

[0095] Example 3

[0096] See also Figure 3 , is a schematic diagram of the structure of a target field extraction system 100 provided in a third embodiment of the present invention, comprising: a text recognition module 10, an entity prediction module 11, a field alignment module 12, and a field extraction module 13, wherein:

[0097] The text recognition module 10 is used to obtain a medical record image of a medical record document, and perform text recognition on the medical record image to obtain a text recognition result.

[0098] Optionally, the text recognition module 10 is further configured to: perform image preprocessing on the medical record image, and binarize the preprocessed medical record image;

[0099] performing denoising on the binarized medical record image, and performing tilt correction on the denoised medical record image;

[0100] Performing character segmentation on the tilt-corrected medical record image, and performing character recognition on the character segmented medical record image to obtain text information;

[0101] The position information of each character in the text information is obtained to obtain the orientation information, and convolution processing is performed on the image corresponding to the text information to obtain the visual information.

[0102] The entity prediction module 11 is used to input the text information, orientation information and visual information in the text recognition result into a multimodal pre-training model to perform entity prediction and obtain an entity list.

[0103] Optionally, the entity prediction module 11 is further configured to: perform text recognition on the text image of each sample text respectively to obtain sample text, sample orientation, and sample vision, and input the sample text, the sample orientation, and the sample vision into the multimodal pre-trained model for entity prediction to obtain an entity prediction list;

[0104] Querying the entity truth list of each sample text, and determining the model loss based on the entity truth list and the entity prediction list;

[0105] Parameters of the multimodal pre-trained model are updated according to the model loss until the multimodal pre-trained model converges, and the converged multimodal pre-trained model is output.

[0106] The field alignment module 12 is configured to perform field alignment based on the row information in the text recognition result and the entity list, obtain key-value entity pairs, and query a preset target field.

[0107] Optionally, the field alignment module 12 is further configured to: obtain, based on the row information, a row number value of each entity in the entity list, and obtain label information of each entity, the label information including a type label, a position label, and an association label;

[0108] Field alignment is performed on the entity list according to the tag information and the row number value to obtain the key-value entity pair.

[0109] Furthermore, the field alignment module 12 is further configured to: query the character filling position of each target field value in the Prompt template, and query the keyword of each target field value in the corresponding key-value entity pair;

[0110] Fill in the characters of the queried keyword in the Prompt template according to the character filling position;

[0111] If the keyword is not found in the target field value corresponding to the key-value entity pair, the preset characters are filled in the Prompt template according to the character filling position.

[0112] The field extraction module 13 is configured to extract fields from the medical record document according to the target field and the key-value entity pair to obtain the target field value of the medical record document.

[0113] Optionally, the field extraction module 13 is further configured to: obtain a Prompt template and adjust parameters according to the target field;

[0114] Matching the key-value entity pairs according to the target field and the Prompt template after parameter adjustment, and marking the matched key-value entity pairs;

[0115] The key value in the marked key-value entity pair is obtained to obtain the target field value.

[0116] In this embodiment, the text information, orientation information and visual information in the text recognition results are input into a multimodal pre-training model for entity prediction, so as to achieve the effect of multimodal prediction of entities, improve the accuracy of entity prediction, reduce the granularity of information, eliminate the need for a large amount of training data, improve the efficiency of target field extraction, and perform field alignment based on the row information and entity list in the text recognition results, thereby improving the accuracy of field extraction.

[0117] Example 4

[0118] Figure 4 This is a structural block diagram of a terminal device 2 provided in the fourth embodiment of the present application. Figure 4 As shown, the terminal device 2 of this embodiment includes: a processor 20, a memory 21, and a computer program 22 stored in the memory 21 and executable on the processor 20, such as a program for the target field extraction method. When the processor 20 executes the computer program 22, the steps of each embodiment of the target field extraction method described above are implemented.

[0119] Exemplarily, the computer program 22 may be divided into one or more modules, which are stored in the memory 21 and executed by the processor 20 to implement the present application. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, which are used to describe the execution process of the computer program 22 in the terminal device 2. The terminal device may include, but is not limited to, a processor 20 and a memory 21.

[0120] The processor 20 may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0121] The memory 21 may be an internal storage unit of the terminal device 2, such as a hard disk or memory of the terminal device 2. The memory 21 may also be an external storage device of the terminal device 2, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 2. Furthermore, the memory 21 may include both an internal storage unit of the terminal device 2 and an external storage device. The memory 21 is used to store the computer program and other programs and data required by the terminal device. The memory 21 may also be used to temporarily store data that has been output or is about to be output.

[0122] In addition, the functional modules in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0123] If the integrated module is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Among them, the computer-readable storage medium can be non-volatile or volatile. Based on this understanding, the present application implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable storage medium may include: any entity or device that can carry computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in computer-readable storage media can be appropriately increased or decreased according to the requirements of legislation and patent practices in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practices, computer-readable storage media do not include electrical carrier signals and telecommunications signals.

[0124] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A target field extraction method, characterized in that: The method comprises: Acquire a medical record image of a medical record document, and perform text recognition on the medical record image to obtain a text recognition result; Inputting text information, orientation information, and visual information in the text recognition result into a multimodal pre-trained model for entity prediction to obtain an entity list, wherein the orientation information includes the coordinates of the characters in the medical record image, and the visual information is used to represent the convolution features of the medical record image; Performing field alignment based on the row information in the text recognition result and the entity list to obtain key-value entity pairs, and querying a preset target field; Extracting fields from the medical record document according to the target field and the key-value entity pair to obtain the target field value of the medical record document; The performing field alignment according to the row information in the text recognition result and the entity list includes: According to the row information, respectively obtain the row number value of each entity in the entity list, and respectively obtain the tag information of each entity; wherein the tag information includes a type tag, a position tag, and an association tag, the type tag is used to characterize the document header, the position tags include KEY-RIGHT and KEY-DOWN, and the association tags include KEY-KEY-RIGHT and KEY-KEY-DOWN, KEY-KEY-RIGHT is the parent KEY of the right KEY, KEY-KEY-DOWN is the parent KEY of the lower KEY, HEADER represents the document header, KEY-RIGHT represents the KEY whose value is on the right, and KEY-DOWN represents the KEY whose value is below; Field alignment is performed on the entity list according to the tag information and the row number value to obtain the key-value entity pair.

2. The target field extraction method according to claim 1, wherein: Extracting fields from the medical record document according to the target field and the key-value entity pair includes: Get the Prompt template and adjust the parameters according to the target field; Matching the key-value entity pairs according to the target field and the Prompt template after parameter adjustment, and marking the matched key-value entity pairs; The key value in the marked key-value entity pair is obtained to obtain the target field value.

3. The target field extraction method according to claim 1, wherein: The performing text recognition on the medical record image to obtain a text recognition result includes: Performing image preprocessing on the medical record image, and binarizing the preprocessed medical record image; performing denoising on the binarized medical record image, and performing tilt correction on the denoised medical record image; Performing character segmentation on the tilt-corrected medical record image, and performing character recognition on the character segmented medical record image to obtain text information; The position information of each character in the text information is obtained to obtain the orientation information, and convolution processing is performed on the image corresponding to the text information to obtain the visual information.

4. The target field extraction method according to claim 2, wherein: After acquiring the key value in the marked key-value entity pair and obtaining the target field value, the method further includes: Query the character filling position of each target field value in the Prompt template respectively, and query the keyword of each target field value in the corresponding key-value entity pair respectively; Fill in the characters of the queried keyword in the Prompt template according to the character filling position; If the keyword is not found in the target field value corresponding to the key-value entity pair, the preset characters are filled in the Prompt template according to the character filling position.

5. The target field extraction method according to any one of claims 1 to 4, characterized in that: Before inputting the text information, orientation information and visual information in the text recognition result into the multimodal pre-training model for entity prediction, the method further includes: Performing text recognition on the text image of each sample text respectively to obtain sample text, sample orientation, and sample vision, and inputting the sample text, the sample orientation, and the sample vision into the multimodal pre-training model for entity prediction to obtain an entity prediction list; Querying the entity truth list of each sample text, and determining the model loss based on the entity truth list and the entity prediction list; Parameters of the multimodal pre-trained model are updated according to the model loss until the multimodal pre-trained model converges, and the converged multimodal pre-trained model is output.

6. A target field extraction system, characterized in that: The system comprises: A text recognition module is used to obtain a medical record image of a medical record document and perform text recognition on the medical record image to obtain a text recognition result; An entity prediction module, configured to input text information, orientation information, and visual information from the text recognition results into a multimodal pre-trained model for entity prediction to obtain an entity list, wherein the orientation information includes the coordinates of characters in the medical record image, and the visual information is used to represent the convolutional features of the medical record image; A field alignment module, configured to perform field alignment based on the row information in the text recognition result and the entity list, obtain key-value entity pairs, and query a preset target field; A field extraction module, configured to extract fields from the medical record document according to the target field and the key-value entity pair, and obtain the target field value of the medical record document; The field alignment module is further configured to: obtain, based on the row information, a row number value of each entity in the entity list, and obtain label information of each entity; wherein the label information includes a type label, a position label, and an association label, the type label being used to characterize a document header, the position label including KEY-RIGHT and KEY-DOWN, and the association label including KEY-KEY-RIGHT and KEY-KEY-DOWN, wherein KEY-KEY-RIGHT is the parent KEY of the right KEY, KEY-KEY-DOWN is the parent KEY of the lower KEY, HEADER represents the document header, KEY-RIGHT represents the KEY whose value is on the right, and KEY-DOWN represents the KEY whose value is on the lower KEY; Field alignment is performed on the entity list according to the tag information and the row number value to obtain the key-value entity pair.

7. The target field extraction system according to claim 6, wherein: The field extraction module is also used to: Get the Prompt template and adjust the parameters according to the target field; Matching the key-value entity pairs according to the target field and the Prompt template after parameter adjustment, and marking the matched key-value entity pairs; The key value in the marked key-value entity pair is obtained to obtain the target field value.

8. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images