A table parsing method, apparatus, device, and storage medium thereof

Abstract

This application belongs to the field of digital healthcare technology and is applied to the spreadsheet processing scenario of a medical digital platform. It relates to a table parsing method, apparatus, device, and storage medium, including: performing cell feature analysis on all table areas to obtain the coordinate and structural features of each cell within each table area; identifying the text content within all table areas and the cell where each text content is located based on the coordinate features; and generating descriptive codes for all table areas as the table parsing result based on the cell where the text content is located and the structural features. Through this table parsing method, the coordinate and structural features of cells in all table areas of a data form in a digital healthcare platform, as well as the text content within the cells, are parsed, facilitating the location, identification, and data reconstruction of text content within the data form of the digital healthcare platform by auxiliary recognition tools.

Description

Technical Field

[0001] This application relates to the field of digital medical technology and is applied to the spreadsheet processing scenario of medical digital platform. In particular, it relates to a spreadsheet parsing method, apparatus, device and its storage medium. Background Technology

[0002] Table processing is widely used in the field of data processing, especially in the field of digital healthcare technology. It often involves medical document images that contain table areas, such as billing documents listing medical charges or purchase invoices listing medical supply vouchers. These images all contain table areas and belong to the data forms in the digital healthcare platform.

[0003] Current table processing methods mainly include three types. The first type obtains the table structure by whole row or column area, generating the corresponding parsing results based on text content, row information, and column information. However, this method cannot handle tables that span multiple rows or columns. The second type uses graph neural networks to obtain the relationships between different cells, including row relationships, column relationships, cell relationships, key-value relationships, hierarchical relationships, and no relationships. This method is mostly based on local cell relationships, which makes it difficult to restore the overall structure, resulting in complex post-processing and poor robustness. The third type uses HTML hypertext representation. For example, Ping An Property & Casualty Insurance's TABLEMASTER and Baidu's SLANet use twenty or thirty HTML tags to encode the table structure. This method results in too many tag types and an excessive amount of data required for annotation. None of the above existing methods are conducive to assisting recognition tools in locating, recognizing, and restoring the text content within data forms in digital healthcare platforms. Summary of the Invention

[0004] The purpose of this application is to provide a table parsing method, apparatus, device and storage medium to solve the problem that the table parsing methods in the prior art are not conducive to the location, recognition and data restoration of text content in data forms in digital medical platforms by auxiliary recognition tools.

[0005] To address the aforementioned technical problems, this application provides a table parsing method, employing the following technical solution:

[0006] A table parsing method includes the following steps:

[0007] Acquire a target image, wherein the target image includes a table area;

[0008] The target image is segmented according to a preset image segmentation model to obtain all table regions within the target image, and the table regions are then distinguished by marking to obtain the distinguishing code values ​​for all table regions.

[0009] A preset table description model is used to perform cell feature analysis on all table areas to obtain the coordinate features and structural features of each cell in all table areas;

[0010] Based on the preset text recognition model, the distinguishing code value, and the coordinate features, the text content in all table areas is identified, the table area where the text content is located is identified, and the cell where the text content is located is identified.

[0011] Based on the cell where the text content is located and the structural features, a description code for all table regions is generated as the table parsing result corresponding to each table region in the target image.

[0012] Furthermore, the step of performing image segmentation processing on the target image according to a preset image segmentation model to obtain all table regions within the target image specifically includes:

[0013] Using tables as target segmentation instances, an image segmentation model based on Mask R-CNN is used to perform instance contour segmentation on the target image to obtain the instance contours of all tables in the target image.

[0014] Based on the instance outlines of all tables within the target image, obtain the regions of all tables within the target image;

[0015] The step of performing differentiation marking processing on all table regions to obtain the differentiation code values ​​of all table regions specifically includes:

[0016] Based on the instance outlines of all tables within the target image, filter out all instance outlines that are not connected together;

[0017] The outlines of all unconnected instances are distinguished and marked, which serve as the distinguishing code values ​​for all table regions.

[0018] Furthermore, the preset table description model is a dual-branch description model, including a cell coordinate feature description branch and a cell structure feature description branch. The step of using the preset table description model to perform cell feature analysis on all table areas to obtain the coordinate features and structural features of each cell in all table areas specifically includes:

[0019] Based on the cell coordinate feature description branch in the table description model, obtain the coordinate features of each cell in all table areas;

[0020] Based on the cell structure feature description branch in the table description model, obtain the structural features of each cell in all table areas.

[0021] Furthermore, the step of obtaining the coordinate features of each cell within all table areas based on the cell coordinate feature description branch in the table description model specifically includes:

[0022] Select any point from the target image as the origin of the coordinate system and set a unit vector;

[0023] Based on the cell coordinate feature description branch, the coordinate origin, and the unit vector, calculate the vertex coordinate value of each cell in all table areas;

[0024] The vertex coordinates are used as the coordinate features.

[0025] Furthermore, the steps of identifying the text content within all table areas, identifying the table areas where the text content is located, and identifying the cells where the text content is located, based on the preset text recognition model, the distinguishing code value, and the coordinate features, specifically include:

[0026] Based on the text recognition model, identify the text content within all the table areas;

[0027] The location region of the text content is determined by the origin of the coordinate system and the unit vector.

[0028] Based on the distinguishing code value and the location region, identify the table region where the text content is located;

[0029] Based on the vertex coordinates of each cell within all the table areas and the location area, the cell where the text content is located is identified.

[0030] Furthermore, the step of obtaining the structural features of each cell within all table areas based on the cell structure feature description branch in the table description model specifically includes:

[0031] Obtain the structural feature description code value that has been pre-input into the table description model, wherein the structural feature description code value includes a first category description code value, a second category description code value and a third category description code value. The first category description code value describes whether the current cell is a blank cell. The second category description code value describes whether the current cell is a merged cell and whether it is merged horizontally or vertically. The third category description code value describes whether the current cell is the end cell of the row.

[0032] Based on the cell structure feature description branch and the structure feature description code value, identify the structure feature description code value contained in each cell of all table areas;

[0033] The structural feature descriptor value contained in each cell is used as the structural feature of the corresponding cell.

[0034] Furthermore, the step of generating descriptive codes for all table regions based on the cell where the text content is located and the structural features, as the table parsing results corresponding to each table region in the target image, specifically includes:

[0035] Based on the structural features of each cell within all the table areas, obtain the structural feature descriptor value contained in each cell within all the table areas;

[0036] Based on the cell where the text content is located, identify the text content contained in each cell within all table areas;

[0037] Based on the structural feature description code value contained in each cell and the text content contained in each cell, generate a quadruple description text corresponding to each cell;

[0038] Obtain the quadruple description text corresponding to each cell in all table regions, organize and generate the description code of all table regions, and use it as the table parsing result corresponding to all table regions in the target image.

[0039] After performing the step of generating descriptive codes for all table regions based on the cell where the text content is located and the structural features, as the table parsing results corresponding to each table region in the target image, the method further includes:

[0040] Based on the description encoding and the coordinate features of each cell within all the table regions, all table regions in the target image are reconstructed and generated.

[0041] To address the aforementioned technical problems, this application also provides a table parsing device, which employs the following technical solution:

[0042] A table parsing apparatus, comprising:

[0043] A target image acquisition module is used to acquire a target image, wherein the target image includes a table area;

[0044] The table region acquisition and marking module is used to perform image segmentation processing on the target image according to a preset image segmentation model, obtain all table regions in the target image, and perform distinguishing marking processing on all table regions to obtain the distinguishing code value of all table regions;

[0045] The cell feature analysis module is used to perform cell feature analysis on all table areas using a preset table description model, and to obtain the coordinate features and structural features of each cell in all table areas.

[0046] The text content information recognition module is used to recognize the text content in all table areas, the table areas where the text content is located, and the cells where the text content is located, based on a preset text recognition model, the distinguishing code value, and the coordinate features.

[0047] The table parsing result generation module is used to generate description codes for all table regions based on the cell where the text content is located and the structural features, as the table parsing results corresponding to all table regions in the target image.

[0048] To address the aforementioned technical problems, this application also provides a computer device that employs the following technical solution:

[0049] A computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the table parsing method described above.

[0050] To address the aforementioned technical problems, this application also provides a computer-readable storage medium, employing the technical solution described below:

[0051] A computer-readable storage medium storing computer-readable instructions that, when executed by a processor, implement the steps of the table parsing method described above.

[0052] Compared with the prior art, the embodiments of this application have the following main advantages:

[0053] The table parsing method described in this application involves: acquiring a target image; performing image segmentation processing on the target image according to a preset image segmentation model to obtain all table regions within the target image; performing distinguishing label processing on all table regions to obtain distinguishing code values ​​for all table regions; using a preset table description model to perform cell feature analysis on each table region to obtain the coordinate features and structural features of each cell within each table region; based on a preset text recognition model, distinguishing code values, and the coordinate features of each cell within each table region, identifying the text content within each table region, identifying the table region where the text content is located, and identifying the cell where the text content is located; and generating descriptive codes for all table regions as the table parsing results corresponding to each table region in the target image based on the cell where the text content is located and the structural features of each cell within each table region. This table parsing method parses the coordinate features and structural features of cells in all table regions of a data form in a digital healthcare platform, as well as the text content within the cells, facilitating the location, recognition, and data reconstruction of text content within data forms in a digital healthcare platform using auxiliary recognition tools. Attached Figure Description

[0054] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 This is an exemplary system architecture diagram to which this application can be applied;

[0056] Figure 2 This is a flowchart of an embodiment of the table parsing method according to this application;

[0057] Figure 3 yes Figure 2 A flowchart of a specific embodiment of step 202 shown;

[0058] Figure 4 yes Figure 2 A flowchart of a specific embodiment of step 203 shown;

[0059] Figure 5 yes Figure 4 A flowchart of a specific embodiment of step 401 shown;

[0060] Figure 6 yes Figure 4 A flowchart of a specific embodiment of step 402 shown;

[0061] Figure 7 yes Figure 2 A flowchart of a specific embodiment of step 204 shown;

[0062] Figure 8 yes Figure 2 A flowchart of a specific embodiment of step 205 shown;

[0063] Figure 9 This is a schematic diagram of the structure of one embodiment of the table parsing apparatus according to this application;

[0064] Figure 10 yes Figure 9 A schematic diagram of a specific embodiment of the table region acquisition and marking module 902 described above;

[0065] Figure 11 yes Figure 9 A schematic diagram of a specific embodiment of the cell feature analysis module 903 described above;

[0066] Figure 12 This is a schematic diagram of the structure of one embodiment of the computer device according to this application. Detailed Implementation

[0067] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.

[0068] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0069] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0070] like Figure 1As shown, system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0071] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social media platform software, etc.

[0072] Terminal devices 101, 102, and 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 players (Moving Picture Experts Group Audio Layer IV), laptops, and desktop computers, etc.

[0073] Server 105 can be a server that provides various services, such as a backend server that supports the pages displayed on terminal devices 101, 102, and 103.

[0074] It should be noted that the table parsing method provided in this application embodiment is generally executed by a server / terminal device, and correspondingly, the table parsing device is generally set in the server / terminal device.

[0075] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0076] Continue to refer to Figure 2 The diagram illustrates a flowchart of an embodiment of the table parsing method according to this application. The table parsing method includes the following steps:

[0077] Step 201: Obtain the target image, wherein the target image includes a table area.

[0078] In this embodiment, the target image refers to a medical document image containing a table area, such as a billing document image listing medical charges or a purchase invoice image listing medical supply vouchers. The common feature of these target images is that they all contain a table area and belong to data forms within a digital healthcare platform.

[0079] Step 202: Perform image segmentation processing on the target image according to the preset image segmentation model to obtain all table regions in the target image, and perform distinguishing label processing on all table regions to obtain the distinguishing code value of all table regions.

[0080] Continue to refer to Figure 3 , Figure 3 yes Figure 2 A flowchart of a specific embodiment of step 202 shown includes:

[0081] Step 301: Using tables as target segmentation instances, an image segmentation model based on Mask R-CNN is used to perform instance contour segmentation processing on the target image to obtain the instance contours of all tables in the target image;

[0082] In this embodiment, the image segmentation model based on Mask R-CNN is an instance segmentation model. Since this application processes table regions, the table is treated as an instance, and instance contour segmentation is performed. Obtaining the instance contours of all tables within the target image completes the image segmentation process. It is not necessary to label each pixel; only the edge contours of the object of interest need to be found. Here, the "object of interest" in the Mask R-CNN-based image segmentation model is the table instance. Mask R-CNN is based on the Faster R-CNN framework, adding a fully connected segmentation network after the basic feature network, transforming the original classification and regression tasks into classification, regression, and segmentation tasks.

[0083] Step 302: Obtain all table regions in the target image based on the instance outlines of all tables in the target image;

[0084] Step 303: Based on the instance outlines of all tables in the target image, filter out all instance outlines that are not connected together;

[0085] In this embodiment, since there is an adjacency relationship between the cells in the table, the instance outlines of all tables in the target image will be adjacent. Therefore, all the instance outlines that are connected together should be treated as a whole table area, and all the instance outlines that are not connected together should be filtered out to obtain the table area.

[0086] Step 304: Distinguish and mark all unconnected instance outlines as distinguishing code values ​​for all table regions.

[0087] The purpose of setting a distinguishing code value for all table areas is to identify the specific table area where the table data content is located based on the distinguishing code value.

[0088] Step 203: Perform cell feature analysis on all table areas using a preset table description model to obtain the coordinate features and structural features of each cell in all table areas.

[0089] In this embodiment, the preset table description model is a dual-branch description model, including a cell coordinate feature description branch and a cell structure feature description branch.

[0090] Continue to refer to Figure 4 , Figure 4 yes Figure 2 A flowchart of a specific embodiment of step 203 shown includes:

[0091] Step 401: Obtain the coordinate features of each cell within all table areas according to the cell coordinate feature description branch in the table description model;

[0092] Continue to refer to Figure 5 , Figure 5 yes Figure 4 A flowchart of a specific embodiment of step 401 shown includes:

[0093] Step 501: Select any point from the target image as the origin of the coordinate system and set a unit vector;

[0094] Step 502: Calculate the vertex coordinates of each cell within all table areas based on the cell coordinate feature description branch, the coordinate origin, and the unit vector.

[0095] Step 503: Use the vertex coordinate values ​​as the coordinate features.

[0096] In this embodiment, the cell coordinate feature description branch uses four feature points and eight feature values ​​to describe the cell coordinate features. Its main purpose is to obtain the coordinates of the four vertices of the cell, use the coordinate points corresponding to the four vertex coordinates as feature points, and use the coordinate value corresponding to each feature point as the feature value.

[0097] Step 402: Obtain the structural features of each cell within all table areas according to the cell structure feature description branch in the table description model.

[0098] Continue to refer to Figure 6, Figure 6 yes Figure 4 A flowchart of a specific embodiment of step 402 shown includes:

[0099] Step 601: Obtain the structural feature description code value that has been pre-input into the table description model;

[0100] The structural feature description code value includes a first category description code value, a second category description code value, and a third category description code value. The first category description code value describes whether the current cell is a blank cell. The second category description code value describes whether the current cell is a merged cell and whether it is merged horizontally or vertically. The third category description code value describes whether the current cell is the end cell of the row.

[0101] In this embodiment, a first category description code value, a second category description code value, and a third category description code value are preset.

[0102] Specifically, the first category description code value describes whether the current cell is a blank cell. For example, there are two cells in the target image, cell A and cell B. Cell A contains text content, while cell B does not contain text content. That is, cell A is not a blank cell, and cell B is a blank cell. Assuming that the first category description code value includes a first description code value "0" and a second description code value "1", where the first description code value "0" represents a non-blank cell and the second description code value "1" represents a blank cell, then cell A corresponds to the first description code value "0", and cell B corresponds to the second description code value "1".

[0103] Specifically, the second category description code value describes whether the current cell is a merged cell and whether it is merged horizontally or vertically. For example, there are three cells in the target image, namely cell A, cell B, and cell C. Cell A is not a merged cell, while cells B and C are merged cells. The difference is that cell B is a horizontally merged cell, and cell C is a vertically merged cell. Assuming that the second category description code value includes the third description code value "2", the fourth description code value "3", and the fifth description code value "4", the third description code value "2" represents a non-merged cell, the fourth description code value "3" represents a horizontally merged cell, and the fifth description code value "4" represents a vertically merged cell, then cell A corresponds to the third description code value "2", cell B corresponds to the fourth description code value "3", and cell C corresponds to the fifth description code value "4".

[0104] Similarly, the third category description code value describes whether the current cell is a row-ending cell. For example, if there are two cells in the target image, cell A and cell B, and cell A is not a row-ending cell while cell B is, assuming that the third category description code value includes the sixth description code value "5" and the seventh description code value "6", where the sixth description code value "5" represents a non-row-ending cell and the seventh description code value "6" represents a row-ending cell, then cell A corresponds to the sixth description code value "5" and cell B corresponds to the seventh description code value "6".

[0105] Step 602: Based on the cell structure feature description branch and the structure feature description code value, identify the structure feature description code value contained in each cell of all table areas;

[0106] Taking cells A and B as examples, cell A contains structural feature description code values ​​of “0”, “2”, and “5”, while cell B contains structural feature description code values ​​of “1”, “3”, and “6”.

[0107] Step 603: Use the structural feature description code value contained in each cell as the structural feature of the corresponding cell.

[0108] By describing the cell structure features, the structural features of each cell in all table areas are obtained, which makes it easier to identify whether each cell is a blank cell, whether it is a merged cell and whether it is merged horizontally or vertically, whether it is a cell at the end of a row, and to restore the table area based on the identification results.

[0109] Step 204: Based on the preset text recognition model, the distinguishing code value, and the coordinate features, identify the text content in all table areas, identify the table areas where the text content is located, and identify the cells where the text content is located.

[0110] Continue to refer to Figure 7 , Figure 7 yes Figure 2 A flowchart of a specific embodiment of step 204 shown includes:

[0111] Step 701: Based on the text recognition model, identify the text content in all table areas, wherein the text recognition model includes an OCR text recognition model;

[0112] Step 702: Determine the location region of the text content using the origin of the coordinate system and the unit vector;

[0113] Step 703: Identify the table regions where the text content is located based on the distinguishing code value and the location region;

[0114] Step 704: Identify the cell where the text content is located based on the vertex coordinates of each cell in all the table areas and the location area.

[0115] By identifying the vertex coordinates of the cell and the location of the text content, the cell in which the text content is located can be identified, which facilitates the subsequent restoration of the text content and restores the text content to the corresponding cell.

[0116] Step 205: Based on the cell where the text content is located and the structural features, generate description codes for all table regions as table parsing results corresponding to all table regions in the target image.

[0117] Continue to refer to Figure 8 , Figure 8 yes Figure 2 A flowchart of a specific embodiment of step 205 shown includes:

[0118] Step 801: Based on the structural features of each cell in all the table areas, obtain the structural feature description code value contained in each cell in all the table areas;

[0119] Step 802: Based on the cell where the text content is located, identify the text content contained in each cell within all table areas;

[0120] Step 803: Based on the structural feature description code value contained in each cell and the text content contained in each cell, generate the quadruple description text corresponding to each cell;

[0121] In this embodiment, referring to cells A and B as described above, it is known that the structural feature description code values ​​contained in cell A are "0", "2" and "5", that is, cell A is a non-blank cell, a non-merged cell, and a non-end-of-line cell. Since cell A is a non-blank cell, the text content contained in cell A is obtained. Assuming the text content is "quantity of HPV test reagents purchased", the quadruple description text corresponding to cell A is ["0", "2", "5", "quantity of HPV test reagents purchased"]. Similarly, the structural feature description code values ​​contained in cell B are "1", "3" and "6", that is, cell B is a blank cell, a merged cell, and a end-of-line cell. Since cell B is a blank cell, that is, cell B does not contain text content, the quadruple description text corresponding to cell B is ["1", "3", "6", "NULL"].

[0122] Step 804: Obtain the quadruple description text corresponding to each cell in all table regions, organize and generate the description code of all table regions, and use it as the table parsing result corresponding to all table regions in the target image.

[0123] By using the structural feature description code value contained in each cell and the text content contained in each cell, a quadruple description text corresponding to each cell is generated. Based on the quadruple description text corresponding to each cell, the description codes of all table regions are generated, so as to restore and generate all table regions in the target image based on the quadruple description text. The description codes of all table regions are serialized data content composed of quadruple description text.

[0124] In this embodiment, after performing the step of generating description codes for all table regions based on the cell where the text content is located and the structural features as the table parsing results corresponding to all table regions in the target image, the method further includes: reconstructing all table regions in the target image based on the description codes and the coordinate features of each cell in all table regions.

[0125] The table parsing method described above parses the coordinate and structural features of cells in all table areas of a data form in a digital healthcare platform, as well as the text content within those cells. Based on the structural features and text content, a four-tuple description text is generated. This facilitates the reconstruction of the data form in the digital healthcare platform by combining the four-tuple description text and the coordinate features, and also enables auxiliary recognition tools to locate and identify the text content within the data form in the digital healthcare platform.

[0126] This application acquires a target image; performs image segmentation processing on the target image according to a preset image segmentation model to obtain all table regions within the target image, and performs distinguishing label processing on all table regions to obtain distinguishing code values ​​for all table regions; uses a preset table description model to perform cell feature analysis on each table region to obtain the coordinate features and structural features of each cell in each table region; based on a preset text recognition model, distinguishing code values, and coordinate features of each cell in each table region, it identifies the text content in each table region, identifies the table region where the text content is located, and identifies the cell where the text content is located; based on the cell where the text content is located and the structural features of each cell in each table region, it generates descriptive codes for all table regions as the table parsing results corresponding to each table region in the target image. Through this table parsing method, the coordinate features and structural features of cells in all table regions of a data form in a digital healthcare platform, as well as the text content in the cells, are parsed, facilitating the location, recognition, and data reconstruction of text content within data forms in a digital healthcare platform by auxiliary recognition tools.

[0127] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0128] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0129] In this embodiment, the target image is segmented according to a preset image segmentation model to obtain all table regions within the target image. Distinguishing features are then applied to each table region to obtain its unique code value. A preset table description model is used to analyze the cell features of each table region, obtaining the coordinate features and structural features of each cell within each table region. Based on a preset text recognition model, the unique code value, and the coordinate features of each cell within each table region, the text content, the table region where the text content is located, and the cell where the text content is located are identified. Based on the cell where the text content is located and the structural features of each cell within each table region, a descriptive code for each table region is generated as the table parsing result corresponding to each table region in the target image. This table parsing method parses the coordinate and structural features of cells in all table regions of a data form in a digital healthcare platform, as well as the text content within the cells, facilitating the location, recognition, and data reconstruction of text content within the data forms of the digital healthcare platform by auxiliary recognition tools.

[0130] Further reference Figure 9 As a response to the above Figure 2 To implement the method shown, this application provides an embodiment of a table parsing apparatus, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0131] like Figure 9 As shown, the table parsing device 900 described in this embodiment includes: a target image acquisition module 901, a table region acquisition and marking module 902, a cell feature analysis module 903, a text content information recognition module 904, and a table parsing result generation module 905. Wherein:

[0132] The target image acquisition module 901 is used to acquire a target image, wherein the target image includes a table area;

[0133] The table region acquisition and marking module 902 is used to perform image segmentation processing on the target image according to a preset image segmentation model, obtain all table regions in the target image, and perform distinguishing marking processing on all table regions to obtain the distinguishing code value of all table regions.

[0134] The cell feature analysis module 903 is used to perform cell feature analysis on all table areas using a preset table description model, and to obtain the coordinate features and structural features of each cell in all table areas.

[0135] The text content information recognition module 904 is used to recognize the text content in all table areas, the table areas where the text content is located, and the cells where the text content is located, based on a preset text recognition model, the distinguishing code value, and the coordinate features.

[0136] The table parsing result generation module 905 is used to generate description codes for all table regions based on the cell where the text content is located and the structural features, as the table parsing results corresponding to all table regions in the target image.

[0137] Continue to refer to Figure 10 , Figure 10 yes Figure 9 A schematic diagram of a specific embodiment of the table region acquisition and marking module 902 described herein. The table region acquisition and marking module 902 includes a table region acquisition submodule 10a and a table region marking submodule 10b. Wherein:

[0138] The table region acquisition submodule 10a is used to segment instances with tables as the target, and to perform instance contour segmentation processing on the target image using an image segmentation model based on Mask R-CNN to obtain the instance contours of all tables in the target image. It is also used to obtain all table regions in the target image based on the instance contours of all tables in the target image.

[0139] The table region marking submodule 10b is used to filter out all unconnected instance contours based on the instance contours of all tables in the target image, and is also used to perform distinguishing marking processing on all unconnected instance contours as distinguishing code values ​​for all table regions.

[0140] Continue to refer to Figure 11 , Figure 11 yes Figure 9 The diagram below shows a specific embodiment of the cell feature analysis module 903 described above. The cell feature analysis module 903 includes a cell coordinate feature analysis submodule 11a and a cell structure feature analysis submodule 11b. Wherein:

[0141] The cell coordinate feature analysis submodule 11a is used to obtain the coordinate features of each cell in all table areas according to the cell coordinate feature description branch in the table description model. Specifically, it selects a point in the target image as the coordinate origin and sets a unit vector; calculates the vertex coordinate value of each cell in all table areas according to the cell coordinate feature description branch, the coordinate origin and the unit vector; and uses the vertex coordinate value as the coordinate feature.

[0142] The cell structure feature analysis submodule 11b is used to obtain the structural features of each cell in all table areas according to the cell structure feature description branch in the table description model. Specifically, it obtains the structural feature description code value that has been pre-input into the table description model; identifies the structural feature description code value contained in each cell in all table areas according to the cell structure feature description branch and the structural feature description code value; and uses the structural feature description code value contained in each cell as the structural feature of the corresponding cell.

[0143] This application acquires a target image; performs image segmentation processing on the target image according to a preset image segmentation model to obtain all table regions within the target image, and performs distinguishing label processing on all table regions to obtain distinguishing code values ​​for all table regions; uses a preset table description model to perform cell feature analysis on each table region to obtain the coordinate features and structural features of each cell in each table region; based on a preset text recognition model, distinguishing code values, and coordinate features of each cell in each table region, it identifies the text content in each table region, identifies the table region where the text content is located, and identifies the cell where the text content is located; based on the cell where the text content is located and the structural features of each cell in each table region, it generates descriptive codes for all table regions as the table parsing results corresponding to each table region in the target image. Through this table parsing method, the coordinate features and structural features of cells in all table regions of a data form in a digital healthcare platform, as well as the text content in the cells, are parsed, facilitating the location, recognition, and data reconstruction of text content within data forms in a digital healthcare platform by auxiliary recognition tools.

[0144] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware through computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0145] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0146] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 12 , Figure 12 This is a basic structural block diagram of the computer device in this embodiment.

[0147] The computer device 12 includes a memory 12a, a processor 12b, and a network interface 12c that are interconnected via a system bus. It should be noted that only the computer device 12 with memory 12a, processor 12b, and network interface 12c is shown in the figure; however, it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0148] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.

[0149] The memory 12a includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 12a may be an internal storage unit of the computer device 12, such as the hard disk or memory of the computer device 12. In other embodiments, the memory 12a may also be an external storage device of the computer device 12, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. Of course, the memory 12a may include both internal storage units and external storage devices of the computer device 12. In this embodiment, the memory 12a is typically used to store the operating system and various application software installed on the computer device 12, such as computer-readable instructions for a table parsing method. In addition, the memory 12a can also be used to temporarily store various types of data that have been output or will be output.

[0150] In some embodiments, the processor 12b may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. The processor 12b is typically used to control the overall operation of the computer device 12. In this embodiment, the processor 12b is used to execute computer-readable instructions stored in the memory 12a or to process data, for example, to execute computer-readable instructions for the table parsing method.

[0151] The network interface 12c may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the computer device 12 and other electronic devices.

[0152] The computer device proposed in this embodiment belongs to the field of digital medical technology and is applied in the spreadsheet processing scenario of a medical digital platform. This application acquires a target image; performs image segmentation processing on the target image according to a preset image segmentation model to obtain all table regions within the target image, and performs distinguishing label processing on all table regions to obtain distinguishing code values ​​for all table regions; uses a preset table description model to perform cell feature analysis on each table region to obtain the coordinate features and structural features of each cell within each table region; based on a preset text recognition model, distinguishing code values, and the coordinate features of each cell within each table region, it identifies the text content within each table region, identifies the table region where the text content is located, and identifies the cell where the text content is located; based on the cell where the text content is located and the structural features of each cell within each table region, it generates a description code for all table regions as the table parsing result corresponding to each table region in the target image. Through this table parsing method, the coordinate features and structural features of cells in all table regions of the data form in the digital medical platform, as well as the text content in the cells, are parsed, facilitating the auxiliary recognition tool to locate, recognize, and reconstruct the text content within the data form of the digital medical platform.

[0153] This application also provides another embodiment, namely, providing a computer-readable storage medium storing computer-readable instructions that can be executed by a processor to cause the processor to perform the steps of the table parsing method described above.

[0154] The computer-readable storage medium proposed in this embodiment belongs to the field of digital medical technology and is applied to spreadsheet processing scenarios in medical digital platforms. This application acquires a target image; performs image segmentation processing on the target image according to a preset image segmentation model to obtain all table regions within the target image, and performs distinguishing label processing on all table regions to obtain distinguishing code values ​​for all table regions; uses a preset table description model to perform cell feature analysis on each table region to obtain the coordinate features and structural features of each cell within each table region; based on a preset text recognition model, distinguishing code values, and the coordinate features of each cell within each table region, it identifies the text content within each table region, identifies the table region where the text content is located, and identifies the cell where the text content is located; based on the cell where the text content is located and the structural features of each cell within each table region, it generates descriptive codes for all table regions as the table parsing results corresponding to each table region in the target image. Through this table parsing method, the coordinate features and structural features of cells in all table regions of a data form in a digital medical platform, as well as the text content within the cells, are parsed, facilitating the location, recognition, and data reconstruction of text content within data forms in a digital medical platform by auxiliary recognition tools.

[0155] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0156] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, the purpose of providing these embodiments is to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.

Claims

1. A table parsing method, characterized in that, Includes the following steps: Acquire a target image, wherein the target image includes a table area; The target image is segmented according to a preset image segmentation model to obtain all table regions within the target image, and the table regions are then distinguished by marking to obtain the distinguishing code values ​​for all table regions. A preset table description model is used to perform cell feature analysis on all table areas to obtain the coordinate features and structural features of each cell in all table areas; Based on the preset text recognition model, the distinguishing code value, and the coordinate features, the text content in all table areas is identified, the table area where the text content is located is identified, and the cell where the text content is located is identified. Based on the cell where the text content is located and the structural features, a descriptive code for all table regions is generated as the table parsing result corresponding to each table region in the target image. Specifically, this includes: Based on the structural features of each cell within all the table areas, obtain the structural feature descriptor value contained in each cell within all the table areas; Based on the cell where the text content is located, identify the text content contained in each cell within all table areas; Based on the structural feature descriptor value contained in each cell and the text content contained in each cell, a four-tuple description text corresponding to each cell is generated. Specifically, the format of the four-tuple description text corresponding to each cell is: ["first category description code value", "second category description code value", "third category description code value", "the text content"], wherein the structural feature descriptor value includes a first category description code value, a second category description code value, and a third category description code value. The first category description code value describes whether the current cell is a blank cell, the second category description code value describes whether the current cell is a merged cell and whether it is merged horizontally or vertically, and the third category description code value describes whether the current cell is a row-ending cell. Obtain the quadruple description text corresponding to each cell in all the table regions, organize them to generate the description code of all the table regions, and use it as the table parsing result corresponding to all the table regions in the target image.

2. The table parsing method according to claim 1, characterized in that, The step of performing image segmentation processing on the target image according to a preset image segmentation model to obtain all table regions within the target image specifically includes: Using tables as target segmentation instances, an image segmentation model based on Mask R-CNN is used to perform instance contour segmentation on the target image to obtain the instance contours of all tables in the target image. Based on the instance outlines of all tables within the target image, obtain the regions of all tables within the target image; The step of performing differentiation marking processing on all table regions to obtain the differentiation code values ​​of all table regions specifically includes: Based on the instance outlines of all tables within the target image, filter out all instance outlines that are not connected together; The outlines of all unconnected instances are distinguished and marked, which serve as the distinguishing code values ​​for all table regions.

3. The table parsing method according to claim 1, characterized in that, The preset table description model is a dual-branch description model, including a cell coordinate feature description branch and a cell structure feature description branch. The step of using the preset table description model to perform cell feature analysis on all table areas to obtain the coordinate features and structural features of each cell in all table areas specifically includes: Based on the cell coordinate feature description branch in the table description model, obtain the coordinate features of each cell in all table areas; Based on the cell structure feature description branch in the table description model, obtain the structural features of each cell in all table areas.

4. The table parsing method according to claim 3, characterized in that, The step of obtaining the coordinate features of each cell within all table areas based on the cell coordinate feature description branch in the table description model specifically includes: Select any point from the target image as the origin of the coordinate system and set a unit vector; Based on the cell coordinate feature description branch, the coordinate origin, and the unit vector, calculate the vertex coordinate value of each cell in all table areas; The vertex coordinates are used as the coordinate features.

5. The table parsing method according to claim 4, characterized in that, The steps of identifying text content within all table areas, identifying the table areas where the text content is located, and identifying the cells where the text content is located, based on a preset text recognition model, the distinguishing code value, and the coordinate features, specifically include: Based on the text recognition model, identify the text content within all the table areas; The location region of the text content is determined by the origin of the coordinate system and the unit vector. Based on the distinguishing code value and the location region, identify the table region where the text content is located; Based on the vertex coordinates of each cell within all the table areas and the location area, the cell where the text content is located is identified.

6. The table parsing method according to claim 3, characterized in that, The step of obtaining the structural features of each cell within all table areas based on the cell structure feature description branch in the table description model specifically includes: Obtain the structural feature description code value that has been pre-input into the table description model; Based on the cell structure feature description branch and the structure feature description code value, identify the structure feature description code value contained in each cell of all table areas; The structural feature descriptor value contained in each cell is used as the structural feature of the corresponding cell.

7. The table parsing method according to claim 6, characterized in that, After performing the step of generating descriptive codes for all table regions based on the cell where the text content is located and the structural features, as the table parsing results corresponding to each table region in the target image, the method further includes: Based on the description encoding and the coordinate features of each cell within all the table regions, all table regions in the target image are reconstructed and generated.

8. A table parsing device, characterized in that, include: A target image acquisition module is used to acquire a target image, wherein the target image includes a table area; The table region acquisition and marking module is used to perform image segmentation processing on the target image according to a preset image segmentation model, obtain all table regions in the target image, and perform distinguishing marking processing on all table regions to obtain the distinguishing code value of all table regions. The cell feature analysis module is used to perform cell feature analysis on all table areas using a preset table description model, and to obtain the coordinate features and structural features of each cell in all table areas. The text content information recognition module is used to recognize the text content in all table areas, the table areas where the text content is located, and the cells where the text content is located, based on a preset text recognition model, the distinguishing code value, and the coordinate features. The table parsing result generation module is used to generate descriptive codes for all table regions based on the cell where the text content is located and the structural features, serving as the table parsing results for each table region in the target image. Specifically, this includes: Based on the structural features of each cell within all the table areas, obtain the structural feature descriptor value contained in each cell within all the table areas; Based on the cell where the text content is located, identify the text content contained in each cell within all table areas; Based on the structural feature descriptor value contained in each cell and the text content contained in each cell, a four-tuple description text corresponding to each cell is generated. Specifically, the format of the four-tuple description text corresponding to each cell is: ["first category description code value", "second category description code value", "third category description code value", "the text content"], wherein the structural feature descriptor value includes a first category description code value, a second category description code value, and a third category description code value. The first category description code value describes whether the current cell is a blank cell, the second category description code value describes whether the current cell is a merged cell and whether it is merged horizontally or vertically, and the third category description code value describes whether the current cell is a row-ending cell. Obtain the quadruple description text corresponding to each cell in all the table regions, organize them to generate the description code of all the table regions, and use it as the table parsing result corresponding to all the table regions in the target image.

9. A computer device, characterized in that, The method includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the table parsing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the table parsing method as described in any one of claims 1 to 7.

Images