A transaction data extraction method, device terminal and storage medium

Abstract

The embodiment of the present application proposes a transaction data extraction method, a device terminal and a storage medium, wherein in the above-mentioned transaction data extraction method, after obtaining the transaction data image, the transaction data image is segmented based on the image features and position features of the content in the transaction data image to obtain corresponding segmented images, and text recognition is performed on each of the segmented images to obtain the text information corresponding to each segmented image, and the transaction key fields in the corresponding segmented image and the specific text information corresponding to the transaction key fields are extracted from the text information, so as to achieve direct extraction of the key fields contained in the corresponding text information in each segmented image after image segmentation for bills of any layout, thereby improving the completeness of information extraction and improving user experience.

Description

[Technical field]

[0001] The embodiments of the present application relate to the field of intelligent terminal technology, and in particular to a transaction data extraction method, a device terminal and a storage medium. [Background technology]

[0002] In the field of insurance claims, insurance companies receive a large amount of claims materials, and after entering and reviewing a wide variety of materials, they make claims for users. It is very important to enter and review the user information and item details in the medical bills, but insurance companies usually use manual entry, which requires a lot of manpower, and the labor cost is extremely high. In addition, the entry is time-consuming and inefficient, and it is easy to miss records and incomplete information, which affects the final claim results.

[0003] In the prior art, in order to solve the problem of low efficiency of manual data entry, after performing OCR (Optical Character Recognition) on the bill, the field recognition results of each field are intelligently matched, transcoded and verified in combination with the medical knowledge base and business logic rules, and finally the transcoded results are output as required. However, this technology needs to identify specific bills according to the type of bill, and cannot process bills of any layout. At the same time, it can only match the fields to be extracted according to the medical knowledge base, and cannot extract fields that do not exist in the medical knowledge base. [Summary of the invention]

[0004] The embodiments of the present application provide a transaction data extraction method, a device terminal and a storage medium, so that during the process of extracting bill information, bills of any layout can be processed, and key fields can be extracted without the support of an additional knowledge base, thereby improving the completeness of information extraction and improving user experience.

[0005] In a first aspect, an embodiment of the present application provides a transaction data extraction method, which is applied to an electronic device terminal, and the method includes: obtaining a transaction data image, segmenting the transaction data image based on image features and position features of the content in the transaction data image, and obtaining corresponding segmented images; performing text recognition on each of the segmented images respectively, and obtaining text information corresponding to each segmented image; extracting transaction key fields in the corresponding segmented image and specific text information corresponding to the transaction key fields from the text information.

[0006] In the above-mentioned transaction data extraction method, after obtaining the transaction data image, the transaction data image is segmented based on the image features and position features of the content in the transaction data image to obtain corresponding segmented images, and text recognition is performed on each of the segmented images to obtain the text information corresponding to each segmented image, and the transaction key fields in the corresponding segmented image and the specific text information corresponding to the transaction key fields are extracted from the text information. Compared with the prior art, the technical solution of the embodiment of the present application does not require the use of an additional medical knowledge base after text recognition, and can directly extract the key fields contained in the corresponding text information in each segmented image after image segmentation for bills of any layout, thereby improving the completeness of information extraction and improving user experience.

[0007] In one embodiment, the segmenting of the transaction data image based on image features and position features of the content in the transaction data image includes: identifying a transaction data title area of ​​the transaction data image based on regional features of a pre-stored transaction data title area; identifying a row and column arrangement area in the transaction data image, identifying the row and column arrangement area as a transaction data detail area, and determining the remaining image areas except the title area based on a relative positional relationship with the transaction data detail area.

[0008] In one embodiment, the determining of the remaining image areas except the title area based on the relative position relationship with the transaction data detail area includes: determining the image area between the transaction data title area and the transaction data detail area as a transaction data identification area, wherein the transaction data identification area is an area of ​​unique identification information representing the transaction data; and determining the image area between the transaction data detail area and the underlying blank area of ​​the transaction data image as a transaction data summary area.

[0009] In one embodiment, the electronic device terminal is pre-set with a data set storing the transaction key fields, the data set includes a first key field, the first key field includes a key field representing the transaction data detail area, and the extraction of the transaction key fields in the corresponding segmented image and the specific text information corresponding to the transaction key fields from the text information includes: obtaining the first text information within a preset range where the vertex coordinates of the transaction data detail area are located, when the first text information includes the first key field, determining that the first text information is the title row of the transaction data detail area; extracting the second text information excluding the title row from the text information of the transaction data detail area, establishing a first corresponding relationship between the title row and the second text information, and establishing a corresponding relationship between the first corresponding relationship and the transaction data detail area.

[0010] In one embodiment, the step of extracting the transaction key field in the corresponding segmented image and the specific text information corresponding to the transaction key field from the text information also includes: when the first key field is not included in the first text information, taking the secondary vertex coordinates adjacent to the vertex coordinates as the current vertex coordinates, and executing the step of obtaining the first text information within the preset range area where the vertex coordinates of the transaction data details area are located.

[0011] In one embodiment, the data set contains key fields that respectively represent the transaction data title area, the transaction data identification area and the transaction data summary area, and the extracting of the transaction key fields in the corresponding segmented image and the specific text information corresponding to the transaction key fields from the text information also includes: when the text information of the transaction data title area, the transaction data identification area and the transaction data summary area contains the corresponding key fields, establishing corresponding relationships between the text information of the transaction data title area, the transaction data identification area and the transaction data summary area and the corresponding areas.

[0012] In a second aspect, an embodiment of the present application provides an electronic device terminal, comprising: an image segmentation module, used to obtain a transaction data image, and segment the transaction data image based on image features and position features of the content in the transaction data image to obtain a corresponding segmented image; a text information acquisition module, used to perform text recognition on each of the segmented images respectively, and obtain text information corresponding to each segmented image; a keyword extraction module, used to extract transaction key fields in the corresponding segmented image and specific text information corresponding to the transaction key fields from the text information.

[0013] In one embodiment, the image segmentation module includes: a title area identification submodule, used to identify the transaction data title area of ​​the transaction data image based on the regional characteristics of the pre-stored transaction data title area; a detail area identification submodule, used to identify the row and column arrangement area in the transaction data image and identify the row and column arrangement area as the transaction data detail area; other area identification submodule, used to determine the remaining image areas except the title area based on the relative position relationship with the transaction data detail area.

[0014] In one embodiment, the other area identification submodule includes: an identification area identification subunit, used to determine that the image area between the transaction data title area and the transaction data detail area is a transaction data identification area, and the transaction data identification area is an area of ​​unique identification information representing the transaction data; a total area identification subunit, used to determine that the image area between the transaction data detail area and the underlying blank area of ​​the transaction data image is a transaction data total area.

[0015] In one embodiment, the electronic device terminal is pre-set with a data set for storing the transaction key fields, the data set includes a first key field, the first key field includes a key field representing the transaction data detail area, and the keyword extraction module includes: a detail area title row extraction submodule, used to obtain the first text information within a preset range area where the vertex coordinates of the transaction data detail area are located, when the first text information includes the first key field, determine that the first text information is the title row of the transaction data detail area; a detail area correspondence establishment submodule, used to extract the second text information except the title row from the text information of the transaction data detail area, establish a first correspondence between the title row and the second text information, and establish a correspondence between the first correspondence and the transaction data detail area.

[0016] In one embodiment, the detail area title row extraction submodule is also used to, when the first key field is not included in the first text information, use the secondary vertex coordinates adjacent to the vertex coordinates as the current vertex coordinates, and execute the step of obtaining the first text information within the preset range area where the vertex coordinates of the transaction data detail area are located.

[0017] In one embodiment, the data set contains key fields that respectively represent the transaction data title area, the transaction data identification area and the transaction data summary area, and the keyword extraction module also includes: a corresponding relationship establishment submodule, which is used to establish a corresponding relationship between the text information of the transaction data title area, the transaction data identification area and the transaction data summary area and the corresponding area when the text information of the transaction data title area, the transaction data identification area and the transaction data summary area contains the corresponding key fields.

[0018] In a third aspect, an embodiment of the present application provides a computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement the steps of the transaction data extraction method.

[0019] It should be understood that the second to third aspects of the embodiments of the present application are consistent with the technical solutions of the first aspect of the embodiments of the present application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar and will not be repeated here.

Brief Description of the Drawings

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 A flowchart of a transaction data extraction method provided by an embodiment of the present application;

[0022] Figure 2 An image diagram of a medical bill provided for one embodiment of the present application;

[0023] Figure 3 A schematic diagram of an image after marking each area provided in an embodiment of the present application;

[0024] Figure 4 A schematic diagram of a text mark area provided for an embodiment of the present application;

[0025] Figure 5 A schematic diagram of an image of a detail area before row and column alignment provided by an embodiment of the present application;

[0026] Figure 6 A schematic diagram of an image of a detail area after row and column alignment provided by an embodiment of the present application;

[0027] Figure 7 A schematic diagram of the structure of an electronic device terminal provided by one embodiment of the present application. [Specific implementation method]

[0028] In order to better understand the technical solution of this specification, the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0029] It should be clear that the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this specification.

[0030] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit this specification. The singular forms of "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0031] First, some of the terms used in the embodiments of the present application are explained to facilitate understanding by those skilled in the art:

[0032] Transaction data: The transaction identifier formed by the two parties during the transaction process, including the identity information of the two parties, the transaction content and transaction code, etc. Its main carriers include receipts, invoices, settlement statements, etc.

[0033] The transaction data extraction method provided in the embodiment of the present application can be executed by an electronic device terminal, which can be a terminal device such as a smart phone, a tablet computer, a PC, a laptop, etc. In an optional embodiment, a service program for executing the transaction data extraction method can be installed on the electronic device.

[0034] Taking the extraction of text information from medical bills as an example, the technical problem that the technical solution provided in the embodiment of the present application actually solves is described as follows:

[0035] When extracting text information from medical bills, you need to first obtain the bill type of the bill, and then convert it to the corresponding medical knowledge base. If the bill type of the medical bill does not exist in the knowledge base, text extraction cannot be performed. The OCR result of the medical bill is matched with the key fields stored in the medical knowledge base, and the recognition results of each field are output as required. Therefore, the type of medical bills in the prior art must conform to the bill type in the medical knowledge base, and medical bills of any layout cannot be processed. When matching the fields of medical bills, it is also necessary to match according to the standard fields of the medical knowledge base itself, and fields outside the medical knowledge base cannot be extracted.

[0036] Figure 1 A flow chart of a transaction data extraction method is provided for the present application. The transaction data extraction method can be applied to an electronic device terminal. The transaction data extraction method comprises the following steps:

[0037] Step S101, acquiring a transaction data image, and segmenting the transaction data image based on image features and position features of content in the transaction data image to obtain a corresponding segmented image.

[0038] Optionally, the transaction data image may be divided into a transaction data title area, a transaction data identification area, a transaction data detail area and a transaction data summary area.

[0039] Optionally, there are two specific segmentation methods:

[0040] Segmentation method 1: The segmentation of transaction data images can be identified and classified according to the image features and position features of each segmented image area. When classifying, the transaction data images can be classified and trained by pre-establishing a deep learning model. For example, they can be divided into several categories such as title, detail, logo and total. After the deep learning model outputs the identification areas of the title, detail, logo and total parts, the segmented images of the title, detail, logo and total parts are obtained.

[0041] Optionally, before training the deep learning model, the transaction data sample images may be preprocessed, for example, a certain number of medical bill images of different types may be screened out as sample medical bills, and the multiple bill images may be proportionally scaled and normalized into a predetermined format, such as a matrix size of 512*512, wherein the transaction data sample images contain identification information that marks the corresponding image area information, such as a target matrix.

[0042] Segmentation method 2: Analyze the specific features of the transaction data title area, transaction data identification area, transaction data detail area, and transaction data total area:

[0043] Optionally, the segmentation process for the transaction data header area may include:

[0044] Step S1011, identifying the transaction data title area of ​​the transaction data image according to the pre-stored area features of the transaction data title area.

[0045] Optionally, the regional feature of the transaction data title area can be the largest font size compared to other areas, or it can be the top-level and centered position area in the transaction data image. After identifying the transaction data title area based on the regional features of the transaction data title area, the transaction data title area is marked, such as a target matrix. It should be noted that the font size is determined in the field of image recognition after judging the number and dispersion of pixels in the corresponding area. The specific font size judgment method is set by those skilled in the art according to actual conditions, and the embodiments of the present application do not limit it.

[0046] Optionally, the segmentation process for the transaction data detail area may include:

[0047] Step S1012, identifying the row and column arrangement area in the transaction data image, marking the row and column arrangement area as the transaction data detail area, and determining the remaining image area except the title area based on the relative position relationship with the transaction data detail area.

[0048] Optionally, the row and column arrangement area can be characterized by the equal vertical line spacing between any two fields in the area, or by the presence of an external frame line in the area, or by the consistency of the font type of each field in the area. However, in actual transaction data, using one of the above features as a judgment feature can easily be confused with other non-transaction data detail areas. Therefore, in the actual segmentation process, the above features can be used as the basis to form a combined feature for comprehensive judgment.

[0049] Optionally, the segmentation process of the transaction data identification area includes:

[0050] Step S1013: determine that the image area between the transaction data title area and the transaction data detail area is a transaction data identification area, and the transaction data identification area is an area representing unique identification information of the transaction data.

[0051] Optionally, the transaction data identification area is generally located between the transaction data title area and the transaction data detail area, so after determining the area between the transaction data title area and the transaction data detail area, the transaction data identification area can be further determined.

[0052] Optionally, the process of segmenting the transaction data aggregate area includes:

[0053] Step S1014: determine that the image area between the transaction data detail area and the bottom blank area of ​​the transaction data image is the transaction data summary area.

[0054] Optionally, after determining the transaction data detail area, the image area between the transaction data detail area and the bottom blank area of ​​the transaction data image may be further determined as the transaction data summary area.

[0055] It should be noted that the above-mentioned step S1011 and step S1012 can be performed simultaneously or in sequence, and the order between step S1011 and step S1012 can be interchanged, and the same applies to the above-mentioned step S1013 and step S1014. The actual sequence setting is set by technical personnel in this field according to actual conditions, and the embodiments of the present application are not limited thereto.

[0056] Step S102: Perform text recognition on each of the segmented images to obtain text information corresponding to each of the segmented images.

[0057] Optionally, the text recognition process can adopt OCR technology, and after text recognition, the text boxes in each segmented image and the position information and text information of each text box are determined respectively, wherein the text box is one or more quadrilateral boxes that frame the text string in the segmented image, and each text box surrounds part of the text string in the segmented image, wherein the position information of the text box includes the horizontal and vertical coordinates of the upper left point of the text box in the segmented image, the frame width and frame height of the text box, and the text information of the text box is the content of the text string framed by the text box.

[0058] Step S103, extracting the transaction key field in the corresponding segmented image and the specific text information corresponding to the transaction key field from the text information.

[0059] Optionally, the electronic device terminal is pre-set with a data set storing transaction key fields, and the data set includes transaction key fields representing a transaction data title area, a transaction data identification area, a transaction data detail area and a transaction data total area.

[0060] Optionally, the data set includes a first key field representing a transaction data header area, and the process of matching the first key field from the transaction data header area includes:

[0061] Step S1031, obtaining first text information within a preset range where the vertex coordinates of the transaction data detail area are located, and when the first text information contains the first key field, determining that the first text information is the title row of the transaction data detail area.

[0062] Optionally, before extracting text information for key field matching, the row and column arrangement structure of the transaction data detail area may also be aligned, that is, after determining the position coordinates of the first row of text boxes in the transaction data detail area, the text boxes in each of the remaining rows are aligned based on the position coordinates of each first row of text boxes, using alignment methods such as centering, left alignment, and right alignment, so as to form a regular row and column arrangement structure, and determine the number of rows and columns of the row and column arrangement structure.

[0063] Optionally, a coordinate system is established with the blank area in the upper left corner of the transaction data detail area as the origin, and the origin is used as the vertex to determine a preset range area with the vertex at the location. The preset range is, for example, a range of a preset horizontal length x or a preset vertical length y, that is, a range between coordinates (0, 0) and (x, 0) or a range between coordinates (0, 0) and (0, y). If the text information within the range contains the first key field representing the transaction data detail area, such as project name, specification model, unit, unit price, quantity, amount, etc., then the text information is used as the title row of the transaction data detail area to indicate the category information described in each row or column in the transaction data detail area.

[0064] In one embodiment, when the first text information does not include the first key field, the sub-vertex coordinates adjacent to the vertex coordinates are used as the current vertex coordinates, and the step of obtaining the first text information within a preset range where the vertex coordinates of the transaction data detail area are located is performed.

[0065] Optionally, if the first key field is not matched within the preset range area of ​​the origin, the secondary vertex coordinates of the adjacent vertex coordinates are used as the current vertex coordinates, the secondary vertex coordinates are, for example, the coordinates (1, 0) or the coordinates (0, 1) of the adjacent vertex coordinates (0, 0). If the text information within the preset range area of ​​the secondary vertex coordinates contains the first key field representing the transaction data detail area, such as project name, specification model, unit, unit price, quantity, amount, etc., then the text information is used as the title row of the transaction data detail area.

[0066] It should be noted that the preset range area can be longer than the interval between the vertex coordinates and the sub-vertex coordinates, or shorter than the interval between the vertex coordinates and the sub-vertex coordinates, or the two can be equal. The specific value is set by technical personnel in this field according to actual conditions and is not limited in this application.

[0067] Step S1032, extracting the second text information except the title row from the text information of the transaction data detail area, establishing a first correspondence between the title row and the second text information, and establishing a correspondence between the first correspondence and the transaction data detail area.

[0068] Optionally, after determining the title row, the correspondence between each title row text box and each of the remaining rows of text boxes is determined according to the row and column arrangement structure. For example, the title row text box includes the text information of "project name", "amount", and "unit" in sequence, and the text information of a certain row of text boxes is "hospitalization fee", "50", and "yuan" in sequence, then the correspondence established between the title row and the row is "project name"─"hospitalization fee", "amount"─"50", "unit"─"yuan". After the correspondence between the title row and each of the remaining rows is established, the correspondence between these correspondences and the transaction data detail area is established. For example, the transaction data detail area corresponds to the "hospitalization fee (project name)", "50 (amount)" and "yuan (unit)" information.

[0069] Optionally, the above data set also includes key fields that respectively represent the transaction data title area, the transaction data identification area and the transaction data total area, such as the second key field representing the transaction data title area, the third key field representing the transaction data identification area, and the fourth key field representing the transaction data total area.

[0070] The data set includes a second key field representing the transaction data title area, and the process of matching the second key field from the transaction data title area includes:

[0071] When the text information in the transaction data title area includes the second key field, a corresponding relationship between the text information and the transaction data title area is established.

[0072] Optionally, the matching process of the key fields in the transaction data title area can be performed using Named Entity Recognition (NER) technology to identify and extract place names and receipt categories in the transaction data title area. Receipt categories include outpatient and inpatient, etc., and the text information that establishes a corresponding relationship with the transaction data title area is the place names and receipt categories in the transaction data title area.

[0073] The data set includes a third key field representing the transaction data identification area, and the process of matching the third key field from the transaction data identification area includes:

[0074] When the text information in the transaction data identification area includes the third key field, a corresponding relationship between the text information and the transaction data identification area is established.

[0075] Optionally, the matching process of the key fields of the transaction data identification area can be performed using Named Entity Recognition (NER) technology to identify and extract identification information in the transaction data identification area, such as invoice code, invoice number, discharge time, admission time, number of days of hospitalization, etc. The text information that establishes a corresponding relationship with the transaction data identification area is the text information in the text boxes belonging to the invoice code, invoice number, discharge time, admission time, number of days of hospitalization, etc. in the transaction data identification area.

[0076] The data set includes a fourth key field representing a transaction data aggregation area. The process of matching the fourth key field from the transaction data aggregation area includes:

[0077] When the text information in the transaction data aggregation area includes the fourth key field, a corresponding relationship between the text information and the transaction data aggregation area is established.

[0078] Optionally, the matching process of the key fields of the transaction data summary area can be performed using Named Entity Recognition (NER) technology to identify and extract the summary information in the transaction data summary area, such as personal payment, total amount, medical insurance pooling payment, etc. The text information that establishes a corresponding relationship with the transaction data summary area is the text information in the text boxes belonging to personal payment, total amount, medical insurance pooling payment, etc. in the transaction data summary area.

[0079] In the above-mentioned transaction data extraction method, after obtaining the transaction data image, the transaction data image is segmented based on the image features and position features of the content in the transaction data image to obtain corresponding segmented images, and text recognition is performed on each of the segmented images to obtain the text information corresponding to each segmented image, and the transaction key fields in the corresponding segmented image and the specific text information corresponding to the transaction key fields are extracted from the text information, so as to achieve direct extraction of the key fields contained in the corresponding text information in each segmented image after image segmentation for bills of any layout, thereby improving the completeness of information extraction and improving user experience.

[0080] Figure 2 A schematic diagram of an image of a medical bill provided in this application, the process of extracting the image includes:

[0081] Step S201, the medical bill is image classified using a convolutional neural network, wherein the convolutional neural network can classify the medical bill image with the target rectangular mark after pre-training, mainly into the title area, mark area, detail area and total area. The image diagram after marking is as follows Figure 3 shown.

[0082] Step S202, respectively, performs text recognition on the title area, the logo area, the detail area and the summary area. After the text recognition is completed, the area containing text information is marked. The schematic diagram of the text marking of the marked area is shown in FIG. Figure 4 shown.

[0083] It should be noted that in this embodiment, the title area, the logo area, the detail area and the total area are separately recognized for text, and each area may also be recognized based on the complete medical bill image.

[0084] Step S203, use NER technology to perform keyword matching on the title area to extract the place name and receipt type, for example Figure 2The text of "XX city" and "XX charges" is extracted, that is, a corresponding relationship is established between the title area of ​​the medical bill and "XX city" and "XX charges", that is, the place name information of the medical bill is "XX city", and the receipt type information is "XX charges".

[0085] Step S204, identifying the row and column arrangement structure of the detail area according to the coordinate relationship of the detail area, and removing the image information that does not conform to the row and column arrangement structure, for example Figure 4 The stamp of the original medical bill is located in the middle of the detail area, that is, the lower semicircular stamp frame and the words "XX Department Supervision". Then the text of each row and column in the detail area is centered. The image diagram before alignment is as follows Figure 5 As shown, the schematic diagram of the aligned image is as follows Figure 6 As shown, after alignment, the title row in the detail area is extracted, for example Figure 2 The "item name", "quantity / unit", "amount (yuan)" and "remarks" in the medical bill are extracted and used as the title row. According to the row and column arrangement structure, it is identified that "item name" corresponds to "medical examination fee", "(medical examination fee)" and "(medical examination fee)" respectively, "quantity / unit" corresponds to "1.00", "1.00" and "1.00" respectively, "amount (yuan)" corresponds to "25.00", "6.00" and "19.00" respectively, and "remarks" corresponds to "", "self-paid" and "medical insurance" respectively. That is, the detailed area of ​​the medical bill corresponds to "medical examination fee (item name) 25.00 (amount (yuan)) 1.00 (quantity / unit)", "medical examination fee (item name) 6.00 (amount (yuan)) 1.00 (quantity / unit) self-paid (remarks)" and "medical examination fee (item name) 19.00 (amount (yuan)) 1.00 (quantity / unit) medical insurance (remarks)" and other information.

[0086] Step S205, use NER technology to perform keyword matching on the marked area to extract the invoice code, invoice number and other information therein, for example Figure 2 The "Bill Code" and "Bill Number" fields in the "Bill Code: 11111111111" and "Bill Number: 1111111111" are subjected to text extraction, that is, the "Bill Code" information of the identification area corresponding to "1111111111" and the "Bill Number" information of the identification area corresponding to "1111111111" are extracted.

[0087] Step S206, use NER technology to perform keyword matching on the summary area to extract information such as personal payment, total amount, and medical insurance pooling payment. For example Figure 2The fields "Medical Insurance Pooling Fund Payment: 0.00", "Personal Payment: 19.00", and "Total Amount (in capital letters) 25 Yuan" are extracted, that is, "0.00" corresponds to the "Medical Insurance Pooling Fund Payment" information in the total area, "19.00" corresponds to the "Personal Payment" information in the total area, and "(in capital letters) 25 Yuan" corresponds to the "Total Amount" information in the total area.

[0088] Step S207, storing the information obtained in the above steps S203-S206 into the storage space corresponding to the medical bill.

[0089] Figure 7 The present application provides a structural schematic diagram of an electronic device terminal, the electronic device terminal comprising:

[0090] The image segmentation module 301 is used to obtain a transaction data image, and segment the transaction data image based on image features and position features of the content in the transaction data image to obtain a corresponding segmented image.

[0091] The text information acquisition module 302 is used to perform text recognition on each of the segmented images to acquire text information corresponding to each of the segmented images.

[0092] The keyword extraction module 303 is used to extract the transaction key field in the corresponding segmented image and the specific text information corresponding to the transaction key field from the text information.

[0093] In one implementation manner, the image segmentation module 301 includes:

[0094] The title area identification submodule is used to identify the transaction data title area of ​​the transaction data image according to the area characteristics of the transaction data title area stored in advance.

[0095] The detail area identification submodule is used to identify the row and column arrangement area in the transaction data image, and mark the row and column arrangement area as the transaction data detail area.

[0096] The other area identification submodule is used to determine the remaining image areas except the title area based on the relative position relationship with the transaction data detail area.

[0097] In one embodiment, the other region identification submodule includes:

[0098] The identification area recognition subunit is used to determine that the image area between the transaction data title area and the transaction data detail area is a transaction data identification area, and the transaction data identification area is an area of ​​unique identification information representing the transaction data.

[0099] The summary area identification subunit is used to determine that the image area between the transaction data detail area and the bottom blank area of ​​the transaction data image is the transaction data summary area.

[0100] In one embodiment, the electronic device terminal is pre-set with a data set storing the transaction key fields, the data set includes a first key field, the first key field includes a key field representing the transaction data detail area, and the keyword extraction module 303 includes:

[0101] The detail area title row extraction submodule is used to obtain the first text information within a preset range where the vertex coordinates of the transaction data detail area are located, and when the first text information contains the first key field, determine that the first text information is the title row of the transaction data detail area.

[0102] The detail area correspondence establishment submodule is used to extract the second text information except the title line from the text information of the transaction data detail area, establish a first correspondence between the title line and the second text information, and establish a correspondence between the first correspondence and the transaction data detail area.

[0103] In one embodiment, the detail area title row extraction submodule is also used to, when the first key field is not included in the first text information, use the secondary vertex coordinates adjacent to the vertex coordinates as the current vertex coordinates, and execute the step of obtaining the first text information within the preset range area where the vertex coordinates of the transaction data detail area are located.

[0104] In one implementation manner, the data set includes key fields representing the transaction data title area, the transaction data identification area, and the transaction data total area, respectively, and the keyword extraction module 303 includes:

[0105] A correspondence establishing submodule is used to establish a correspondence between the text information of the transaction data title area, the transaction data identification area and the transaction data summary area and the corresponding area when the text information of the transaction data title area, the transaction data identification area and the transaction data summary area contains the corresponding key field.

[0106] The embodiment of the present application also provides a computer-readable storage medium on which computer instructions are stored, and when the instructions are executed by a processor, the steps of the above transaction data extraction method are implemented. The readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device.

[0107] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0108] In the description of the embodiments of the present invention, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of this specification. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0109] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of this specification includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of this specification belong.

[0110] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0111] It should be noted that the terminals involved in the embodiments of the present application may include but are not limited to personal computers (PCs), personal digital assistants (PDAs), wireless handheld devices, tablet computers, mobile phones, MP3 players, MP4 players, etc.

[0112] In the several embodiments provided in this specification, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0113] In addition, each functional unit in each embodiment of this specification may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0114] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform some steps of the method described in each embodiment of this specification. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, and other media that can store program codes.

[0115] The above description is only a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this specification should be included in the scope of protection of this specification.

Claims

1. A transaction data extraction method, applied to an electronic device terminal, characterized in that: The method comprises: Acquire a transaction data image, and segment the transaction data image based on image features and position features of content in the transaction data image to obtain a corresponding segmented image; Performing text recognition on each of the segmented images respectively to obtain text information corresponding to each of the segmented images; Extracting the transaction key field in the corresponding segmented image and the specific text information corresponding to the transaction key field from the text information; The segmenting of the transaction data image based on the image features and position features of the content in the transaction data image includes: identifying the transaction data title area of ​​the transaction data image according to the area features of the transaction data title area stored in advance; Identifying a row-column arrangement region in the transaction data image, marking the row-column arrangement region as a transaction data detail region, and determining the remaining image regions except the title region based on a relative positional relationship with the transaction data detail region; The electronic device terminal is pre-set with a data set storing the transaction key field, the data set includes a first key field, the first key field includes a key field representing the transaction data detail area, and the extracting of the transaction key field in the corresponding segmented image and the specific text information corresponding to the transaction key field from the text information includes: Obtaining first text information within a preset range where the vertex coordinates of the transaction data detail area are located, and when the first text information contains the first key field, determining that the first text information is a title row of the transaction data detail area; Extracting second text information other than the title row from the text information in the transaction data detail area, establishing a first correspondence between the title row and the second text information, and establishing a correspondence between the first correspondence and the transaction data detail area; The step of extracting the transaction key field in the corresponding segmented image and the specific text information corresponding to the transaction key field from the text information further includes: When the first text information does not include the first key field, the sub-vertex coordinates adjacent to the vertex coordinates are used as current vertex coordinates, and the step of obtaining the first text information within the preset range where the vertex coordinates of the transaction data detail area are located is performed.

2. The method according to claim 1, characterized in that The determining of the remaining image areas except the title area based on the relative position relationship with the transaction data detail area includes: Determine an image area between the transaction data title area and the transaction data detail area as a transaction data identification area, wherein the transaction data identification area is an area representing unique identification information of the transaction data; An image area between the transaction data detail area and a bottom blank area of ​​the transaction data image is determined as a transaction data summary area.

3. The method according to claim 1, characterized in that The data set includes key fields representing the transaction data title area, the transaction data identification area, and the transaction data summary area, respectively. The extracting of the transaction key fields in the corresponding segmented image and the specific text information corresponding to the transaction key fields from the text information also includes: When the text information of the transaction data title area, the transaction data identification area and the transaction data summary area contains the corresponding key fields, a corresponding relationship is established between the text information of the transaction data title area, the transaction data identification area and the transaction data summary area and the corresponding area.

4. An electronic equipment terminal, characterized in that: The terminal comprises: An image segmentation module, used to obtain a transaction data image, and segment the transaction data image based on image features and position features of content in the transaction data image to obtain a corresponding segmented image; A text information acquisition module, used to perform text recognition on each of the segmented images to obtain text information corresponding to each of the segmented images; A keyword extraction module, used to extract the transaction key field in the corresponding segmented image and the specific text information corresponding to the transaction key field from the text information; The image segmentation module comprises: a title area recognition submodule, for recognizing the transaction data title area of ​​the transaction data image according to the area characteristics of the transaction data title area stored in advance; A detail area identification submodule, used to identify a row and column arrangement area in the transaction data image, and mark the row and column arrangement area as a transaction data detail area; Other area recognition submodule, used for determining the remaining image areas except the title area based on the relative position relationship with the transaction data detail area; The electronic device terminal is pre-set with a data set storing the transaction key fields, the data set includes a first key field, the first key field includes a key field representing the transaction data detail area, and the keyword extraction module includes: A detail area title row extraction submodule, used to obtain first text information within a preset range where the vertex coordinates of the transaction data detail area are located, and when the first text information contains the first key field, determine that the first text information is the title row of the transaction data detail area; A detail area correspondence establishment submodule, used for extracting second text information other than the title line from the text information of the transaction data detail area, establishing a first correspondence between the title line and the second text information, and establishing a correspondence between the first correspondence and the transaction data detail area; The detail area title row extraction submodule is also used to, when the first text information does not contain the first key field, use the secondary vertex coordinates adjacent to the vertex coordinates as the current vertex coordinates, and execute the step of obtaining the first text information within the preset range area where the vertex coordinates of the transaction data detail area are located.

5. The terminal according to claim 4, characterized in that The other area identification submodules include: An identification area recognition subunit, used to determine that the image area between the transaction data title area and the transaction data detail area is a transaction data identification area, and the transaction data identification area is an area representing unique identification information of the transaction data; The summary area identification subunit is used to determine that the image area between the transaction data detail area and the bottom blank area of ​​the transaction data image is the transaction data summary area.

6. The terminal according to claim 4, characterized in that The data set includes key fields that respectively represent the transaction data title area, the transaction data identification area, and the transaction data total area, and the keyword extraction module further includes: A correspondence establishing submodule is used to establish a correspondence between the text information of the transaction data title area, the transaction data identification area and the transaction data summary area and the corresponding area when the text information of the transaction data title area, the transaction data identification area and the transaction data summary area contains the corresponding key field.

7. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the instruction is executed by a processor, the steps of the method according to any one of claims 1 to 3 are implemented.

Images