Data extraction method and device, electronic equipment and storage medium
By identifying and structuring the original table pictures, the material parameter information is automatically extracted, and the problem of time-consuming and error-prone manual extraction in the existing technology is solved, and efficient and accurate data extraction is achieved.
Patent Information
- Application Number
- CN202410887221.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-02
- Publication Date
- 2024-11-15
AI Technical Summary
In the prior art, the extraction of material parameter information relies on manual reading and manual entry, which takes a long time and is prone to errors.
By identifying the original table pictures, forming structured data, determining the table type, filling and extracting data, and automatically extracting target data using row index and column index.
Accurate data extraction of nested and non-necked header tables is realized, improving efficiency and accuracy, and simplifying the operation process.
Smart Images

Figure CN118968532A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of data processing technology, and in particular to a data extraction method, device, electronic device and storage medium. Background Art
[0002] In the production activities of enterprises, it is necessary to extract material parameter information from a large number of factory material lists provided by raw material suppliers for entry into the production system of the enterprise. The existing method of extracting material parameter information is mainly based on manually reading a large number of factory material lists, and then manually extracting and entering them. This method is time-consuming and prone to errors. Summary of the invention
[0003] The present disclosure provides a data extraction method, device, electronic device and storage medium to at least solve the above technical problems existing in the prior art.
[0004] According to a first aspect of the present disclosure, a data extraction method is provided, the method comprising: identifying data in an original table image to form structured data, the structured data comprising a row index and a column index; based on the structured data, determining the type of the original table, the type comprising a nested header table and a non-nested header table; performing data filling on the structured data corresponding to the nested header table so that each column of data in the obtained filled structured data has a corresponding header as a column index; performing data extraction on the row index and column index of the filled structured data corresponding to the nested header table or the structured data corresponding to the non-nested header table to obtain target data.
[0005] In one possible implementation, the identifying the data in the original table image to form structured data includes: identifying the data in the original table image to obtain table-type data; and formatting the table-type data to obtain structured data.
[0006] In one possible implementation, determining the type of the original table based on the structured data includes: determining whether a column index of the structured data includes specific characters; if included, determining that the type of the original table is a nested header table; if not included, determining that the type of the original table is a non-nested header table.
[0007] In one possible implementation, the structured data corresponding to the nested header table is filled with data so that each column of data in the filled structured data has a corresponding header as a column index, including: obtaining data in the left adjacent cell of the cell where the specific character is located as filling data; based on the filling data, the cell where the specific character is located is filled with data to obtain the filled structured data.
[0008] In one possible implementation, data extraction is performed on the row index and column index of the populated structured data corresponding to the nested header table to obtain target data, including: determining the first row of data in the structured data corresponding to the nested header table; and extracting data from the populated structured data based on the row index, column index and the first row of data of the structured data to obtain target data.
[0009] In one possible implementation, the data extraction for the row index and column index of the filled structured data corresponding to the nested header table or the structured data corresponding to the non-nested header table to obtain the target data also includes: renaming the column index in the structured data based on the name mapping table so that the name of the column index corresponding to the same parameter in different structured data is consistent; data extraction for the row index and renamed column index of the filled structured data corresponding to the nested header table or the structured data corresponding to the non-nested header table to obtain the target data.
[0010] According to a second aspect of the present disclosure, there is provided a data extraction device, the device comprising:
[0011] An image processing module, used to identify data in the original table image to form structured data, wherein the structured data includes a row index and a column index;
[0012] A judgment module, configured to determine the type of the original table based on the structured data, wherein the type includes a nested header table and a non-nested header table;
[0013] A data filling module is used to fill the structured data corresponding to the nested header table, so that each column of the filled structured data has a corresponding header as a column index;
[0014] The extraction module is used to extract data from the row index and column index of the filled structured data corresponding to the nested header table or the structured data corresponding to the non-nested header table to obtain target data.
[0015] In one possible implementation manner, the image processing module is further used to: identify data in the original table image to obtain table data; and convert the format of the table data to obtain structured data.
[0016] According to a third aspect of the present disclosure, there is provided an electronic device, including:
[0017] at least one processor; and
[0018] a memory communicatively connected to the at least one processor; wherein,
[0019] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the present disclosure.
[0020] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the method described in the present disclosure.
[0021] The data extraction method, device, electronic device and storage medium disclosed in the present invention first convert the original table image into structured data with row index and column index. Then, according to the content in the structured data, it is determined whether the original table is a nested header table. For the structured data of the non-nested header table, data extraction is performed directly according to the row index and column index to obtain the target data. For the structured data of the nested header table, it is necessary to first fill the columns with missing headers with data so that each column of data has a clear header as a column index. Then, data extraction is performed according to the row index and column index in the filled structured data to obtain the target data. The data extraction method disclosed in the present invention can accurately extract the target data of the nested header table and the non-nested header table, and has strong versatility. And the method is simple to operate, and the operator only needs to give the original table image and the address of the location where the target data needs to be stored, and the extraction of the target data can be automatically completed. Compared with the manual data extraction and entry in the prior art, the accuracy and efficiency of the present method are higher.
[0022] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, in which:
[0024] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts.
[0025] Figure 1 The following is a schematic diagram showing the implementation process of a data extraction method according to an embodiment of the present disclosure. Figure 1 ;
[0026] Figure 2 A schematic diagram of a nested header table according to an embodiment of the present disclosure is shown;
[0027] Figure 3A schematic diagram of a non-nested header table according to an embodiment of the present disclosure is shown;
[0028] Figure 4 A schematic diagram of an xlsx file of a nested header table according to an embodiment of the present disclosure is shown;
[0029] Figure 5 A schematic diagram of an xlsx file of a non-nested header table according to an embodiment of the present disclosure is shown;
[0030] Figure 6 A schematic diagram of structured data of a nested header table according to an embodiment of the present disclosure is shown;
[0031] Figure 7 A schematic diagram of structured data of a non-nested header table according to an embodiment of the present disclosure is shown;
[0032] Figure 8 A schematic diagram of target data according to an embodiment of the present disclosure is shown;
[0033] Fig. 9 The following is a schematic diagram showing the implementation process of a data extraction method according to an embodiment of the present disclosure. Figure 2 ;
[0034] Fig.10 A schematic diagram of the structure of a data extraction device according to an embodiment of the present disclosure is shown;
[0035] Fig.11 A schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION
[0036] In order to make the purpose, features, and advantages of the present disclosure more obvious and easy to understand, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.
[0037] Figure 1 The following is a schematic diagram showing the implementation process of a data extraction method according to an embodiment of the present disclosure. Figure 1 ,like Figure 1 As shown, according to a first aspect of an embodiment of the present disclosure, a data extraction method is provided, and the method includes:
[0038] Step 101, identifying the data in the original table image to form structured data, wherein the structured data includes a row index and a column index.
[0039] An original table image refers to an image file containing table data, which may be obtained by scanning a paper document, taking a photo, or converting from other digital sources (such as PDF files). For example, in the field of electronic product manufacturing, an original table image may be converted from a bill of materials file provided by a raw material supplier, which contains information such as material number, description, quantity, etc. First, the table structure in the original table image and the text data in the cells are recognized using table recognition technology. The recognition results are then converted into structured data with row and column indexes. Structured data is data stored in a table, database, or other form, in which each data item is in a specific row and column. Row and column indexes are used to accurately identify and locate each data item in the table.
[0040] In one embodiment of the present disclosure, after the data in the original table image is recognized, the recognition result is first converted into table data, and then the table data is formatted to obtain structured data.
[0041] First, use table recognition technologies such as paddleocr.PPStructure, Amazon Textract, Tesseract OCR, etc. to extract the text content in the original table image and obtain an editable table data. Tabular data usually consists of rows and columns, and can be in Excel file formats such as xlsx, xls, and xlsm. Then, you can use pandas (a data analysis package in python) in the Python library to read the tabular data and obtain structured data in DataFrame format.
[0042] For example, the PPStructure algorithm is used to Figure 2 and Figure 3 The original table image shown in the figure is recognized, and then the text content in the image is output as a table data in xlsx format, and the following is obtained: Figure 4 and Figure 5 The table data in xlsx format is shown. Figure 4 for Figure 2 The table data in xlsx format corresponding to the table shown, Figure 5 for Figure 3 The table shown corresponds to the tabular data in xlsx format. Then, the data in xlsx is read through pandas, and the data in the xlsx file is read into the DataFrame object of pandas to obtain structured data in DataFrame format. The conversion process from image to structured data is realized, which is convenient for subsequent processing of the tabular data in the image.
[0043] Step 102: Determine the type of the original table based on the structured data, where the type includes a nested header table and a non-nested header table.
[0044] A nested header table is a table that contains nested headers. A nested header means that in a table, the header is not a single header, but consists of multiple levels. A main header may include multiple sub-headers. For example Figure 2 In the nested header table shown, "Rated Current" is the main header, and the next level includes two sub-headers, "Typical Value" and "Maximum Value". A non-nested header is a table that does not contain a nested header. For example Figure 3 In the non-nested header table shown, each column of data corresponds to a unique header. In the structured data, the header of the first row is used as the column index, and the row name of the first column is used as the row index.
[0045] Since the main header of a nested table may span multiple columns, that is, it is in a merged cell. Therefore, when converting it into structured data, the algorithm cannot directly handle this complex header structure, which may result in some columns in the converted structured data not having a clear header. In this case, the algorithm will automatically assign a default column name to those columns without a clear header. For example, Figure 2 In the embedded header table shown in the figure, when it is converted into structured data, the column name of the "typical value" in "rated current" is "rated current", while the column name of the "maximum value" is the default column name automatically assigned by the algorithm. Therefore, it can be determined whether the original table includes a nested header based on whether the default column name automatically assigned by the algorithm exists in the header of the structured data.
[0046] Step 103, data filling is performed on the structured data corresponding to the nested header table, so that each column of data in the obtained filled structured data has a corresponding header as a column index.
[0047] If it is determined in step 102 that the original table corresponding to the structured data is a non-nested header table, it means that each data item in the structured data corresponds to a header with a clear meaning as a column index. For nested header tables, in the process of converting into structured data, due to the existence of nested headers, default column names are automatically assigned to certain columns, resulting in these columns not having clear headers as column indexes. Using default column names reduces the readability of the data because the default column names do not provide any information about the column content. Therefore, in this step, the position where the default column name exists in the structured data is filled with data using the original header corresponding to the column where the position is located, so that each column of data after filling has a clear header as a column index.
[0048] Step 104 , extracting data from the row index and column index of the filled structured data corresponding to the nested header table or the structured data corresponding to the non-nested header table to obtain target data.
[0049] Each data item in the supplemented structured data corresponding to the nested header table and the structured data corresponding to the non-nested header table has a clear row index and column index. Traverse the structured data and combine each data item with its corresponding row index and column index to form a key-value pair. The combination of the row index and the column index is used as the key, and the data item is used as the value, which is the target data to be extracted.
[0050] For example, the row index is the part number of the material "SICP0530S-R10L-LB1W", and the column index is the inductance value. In this case, the key is "SICP0530S-R10L-LB1W and inductance value". The key is used as the index to obtain the value corresponding to this key "0.1", and "0.1" is the target data.
[0051] In one embodiment, since the nested header table has multiple levels of headers, when extracting data, it is necessary not only to determine the target data based on the row index and column index, but also based on the first row of data. Figure 6 As shown, the data in the row where the data such as "Part Number", "Inductance Value", "Rated Current", and "Saturation Current" are located is the table header, that is, the column index. The row where the data such as "NaN", "Typical Value", and "Maximum Value" are located is the first row of data. For example, it is necessary to extract the typical value of the rated current value of SICP0530S-R10L-LB1W. At this time, the unique target data cannot be determined based on the row index "SICP0530S-R10L-LB1W" and the column index "Rated Current". Therefore, it is necessary to further determine the typical value under "Rated Current" based on the first row of data, so that the unique target data "32.0" can be extracted.
[0052] Taking the extraction of the typical values of the inductance and rated current of each component as an example, the typical values of the component number (PN), inductance value (L) and rated current (IRMS) of each component are used as the row index and column index respectively. Figure 7 And after filling in the values Figure 6 The structured data is extracted to obtain Figure 8 The target data can then be stored in the target database.
[0053] The data extraction method disclosed in the present invention can accurately extract the target data of nested header tables and non-nested header tables, and has strong versatility. The method is simple to operate, and the operator only needs to give the original table image and the address of the location where the target data needs to be stored (target database), and the target data can be automatically extracted. Compared with the manual data extraction and entry in the prior art, the accuracy and efficiency of the present method are higher.
[0054] In one embodiment of the present disclosure, determining the type of the original table based on the structured data includes: determining whether a column index of the structured data includes specific characters; if included, determining that the type of the original table is a nested header table; if not included, determining that the type of the original table is a non-nested header table.
[0055] In this embodiment, if the structured data is obtained by pandas, when pandas reads a table data with merged cells, it usually automatically assigns a column name in the form of "Unnamed: X" to the column without an obvious header, where X is a data representing the position of the column. "Unnamed: X" is a specific character. If the specific character "Unnamed: X" is detected in the structured data, it is determined that the original table corresponding to the structured data is a nested header table; if it does not exist, it is determined that the original table corresponding to the structured data is a non-nested header table.
[0056] For example, Figure 6 From the structured data shown, we can see that the data contains specific characters "Unnamed: 3" and "Unnamed: 5", which means that the original table corresponding to the structured data is a nested header table. Figure 7 The structured data shown in the figure does not contain the specific character "Unnamed: X", which means that the original table corresponding to the structured data is a non-nested header table. Among them, "NaN" indicates that the data is missing or the cell is empty.
[0057] In one embodiment of the present disclosure, the structured data corresponding to the nested header table is filled with data so that each column of the obtained filled structured data has a corresponding header as a column index, specifically: the data in the left adjacent cell of the cell where the specific character is located is obtained as the filling data; based on the filling data, the cell where the specific character is located is filled with data to obtain the filled structured data.
[0058] In tabular data, column names are usually sorted in a certain logical order. When a column name is assigned a specific character due to merging cells or other reasons, it is usually logically continuous with the column name on its left. The left adjacent cell of the cell where the specific character is located usually contains the real content of the merged cell. Therefore, in this embodiment, the data in the left cell is used as the filling data. Then the filling data is added to the cell where the specific character is located to obtain the filled structured data.
[0059] by Figure 6 For example, the specific characters are "Unnamed: 3" and "Unnamed: 5". Get the data "Rated current (Amp)" in the cell to the left of "Unnamed: 3". Fill "Rated current (Amp)" into the cell where "Unnamed: 3" is located as the filling data to obtain the filled structured data. Similarly, get the data "Saturated current (Amp)" in the cell to the left of "Unnamed: 5". Fill "Saturated current (Amp)" into the cell where "Unnamed: 5" is located as the filling data to obtain the filled structured data.
[0060] In this embodiment, using the data in the left cell as filler data is a reasonable and effective method when processing column names containing specific characters in structured data. The supplemented structured data improves the readability of the data.
[0061] In one embodiment of the present disclosure, the data extraction for the row index and column index of the filled structured data corresponding to the nested header table or the structured data corresponding to the non-nested header table to obtain the target data also includes: renaming the column index in the structured data based on the name mapping table so that the name of the column index corresponding to the same parameter in different structured data is consistent; data extraction for the row index and renamed column index of the filled structured data corresponding to the nested header table or the structured data corresponding to the non-nested header table to obtain the target data.
[0062] In the process of data processing, it is often encountered that the same parameter in different data sources (original table images) uses different column names. In order to improve the efficiency and accuracy of data processing, a name mapping table is used in this embodiment to rename the column index. The name mapping table contains the mapping relationship between different names of the same parameter. For example, for the parameter "part number", the name of the parameter in the target database is "PN" (abbreviation of PART NUMBER), and the name of the parameter in the original table image is "PART NUMBER", and the resulting structured data is also "PART NUMBER". The name mapping table records a variety of possible names that "PN" may correspond to. At this time, the "PARTNUMBER" in the structured data is renamed according to the name mapping table, and it is changed to the "PN" required in the target database.
[0063] After the column index is renamed in this embodiment, a unified column name can be used to write data processing and analysis codes, without having to write different codes for different data sources, which greatly improves the efficiency and maintainability of data processing.
[0064] In order to provide an overall understanding of the solution disclosed herein, the method is explained below with reference to a specific embodiment. Fig. 9 As shown, the method comprises the following steps:
[0065] Step 901, read the original table image, identify it through Paddleocr's PPStructure algorithm, and get the xlsx file. Use pandas to read the xlsx file and get the structured data. The first line of the structured data is the table header, which is used as the column index, and the first column is used as the row index;
[0066] Step 902, classifying the original table according to the structured data;
[0067] Step 903, determine whether the table header in the structured data includes "Unnamed"; if it does, determine that the original table is a nested header table, and execute step 904; if it does not, determine that the original table is a non-nested header table, and execute step 907;
[0068] Step 904, for the structured data corresponding to the nested header table, fill the cell where "Unnamed" is located with data according to the content of the cell to the left of the cell where "Unnamed" is located;
[0069] Step 905, renaming the column indexes for the structured data corresponding to the nested header table, so that the names for the unified parameters in different original tables are unified;
[0070] Step 906, extracting information based on the renamed row index, column index and the first row of data to obtain target data;
[0071] Step 907, for the structured data corresponding to the non-nested header table, rename the column index so that the names of the unified parameters in different original tables are unified;
[0072] Step 908, extract information according to the renamed row index and column index to obtain target data;
[0073] Step 909: write the target data into the target database.
[0074] According to another aspect of the present disclosure, a data extraction device is provided. Fig.10 A schematic diagram of the structure of a data extraction device according to an embodiment of the present disclosure is shown. Fig.10 As shown:
[0075] The device includes an image processing module 1001, which is used to identify data in an original table image to form structured data, wherein the structured data includes a row index and a column index; a judgment module 1002, which is used to determine the type of the original table based on the structured data, wherein the type includes a nested header table and a non-nested header table; a data filling module 1003, which is used to fill the structured data corresponding to the nested header table, so that each column of data in the filled structured data has a corresponding header as a column index; an extraction module 1004, which is used to extract data from the row index and column index of the filled structured data corresponding to the nested header table or the structured data corresponding to the non-nested header table to obtain target data.
[0076] In one embodiment of the present disclosure, the image processing module 1001 is further used to: identify data in the original table image to obtain table data; and convert the format of the table data to obtain structured data.
[0077] In one embodiment of the present disclosure, the judgment module 1002 is also used to include: determining whether the column index of the structured data includes specific characters; if included, determining that the type of the original table is a nested header table; if not included, determining that the type of the original table is a non-nested header table.
[0078] In one embodiment of the present disclosure, the data filling module 1003 is also used to obtain data in the left adjacent cell of the cell where the specific character is located as filling data; based on the filling data, the cell where the specific character is located is filled with data to obtain the filled structured data.
[0079] In one embodiment of the present disclosure, the extraction module 1004 is also used to determine the first row of data in the filled structured data corresponding to the nested header table; extract data from the filled structured data based on the row index, column index and the first row of data of the structured data to obtain target data.
[0080] In one embodiment of the present disclosure, the extraction module 1004 is also used to rename the column indexes in the structured data based on the name mapping table so that the names of the column indexes corresponding to the same parameter in different structured data are consistent; data extraction is performed on the row indexes and renamed column indexes of the filled structured data corresponding to the nested header table or the structured data corresponding to the non-nested header table to obtain the target data.
[0081] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.
[0082] Fig.11 A schematic block diagram of an example electronic device 1100 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.
[0083] like Fig.11 As shown, the device 1100 includes a computing unit 1101, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1102 or a computer program loaded from a storage unit 1108 into a random access memory (RAM) 1103. In the RAM 1103, various programs and data required for the operation of the device 1100 can also be stored. The computing unit 1101, the ROM 1102, and the RAM 1103 are connected to each other via a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.
[0084] A number of components in the device 1100 are connected to the I / O interface 1105, including: an input unit 1106, such as a keyboard, a mouse, etc.; an output unit 1107, such as various types of displays, speakers, etc.; a storage unit 1108, such as a disk, an optical disk, etc.; and a communication unit 1109, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1109 allows the device 1100 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0085] The computing unit 1101 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1101 performs the various methods and processes described above, such as a data extraction method. For example, in some embodiments, a data extraction method may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 1108. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 1100 via the ROM 1102 and / or the communication unit 1109. When the computer program is loaded into the RAM 1103 and executed by the computing unit 1101, one or more steps of a data extraction method described above may be performed. Alternatively, in other embodiments, the computing unit 1101 may be configured to execute a data extraction method in any other appropriate manner (eg, by means of firmware).
[0086] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0087] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0088] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0089] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0090] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.
[0091] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.
[0092] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.
[0093] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present disclosure, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.
[0094] The above is only a specific embodiment of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present disclosure, which should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.
Claims
1. A data extraction method, characterized in that: The method comprises: Recognize the data in the original table image to form structured data, where the structured data includes a row index and a column index; Based on the structured data, determining the type of the original table, the type including a nested header table and a non-nested header table; Filling the structured data corresponding to the nested header table so that each column of the filled structured data has a corresponding header as a column index; Data is extracted for the filled structured data corresponding to the nested header table or the row index and column index of the structured data corresponding to the non-nested header table to obtain target data.
2. The method according to claim 1, characterized in that The data in the original table image is recognized to form structured data, including: Recognize the data in the original table image to obtain table data; The table data is formatted to obtain structured data.
3. The method according to claim 1, characterized in that The determining the type of the original table based on the structured data includes: Determine whether a column index of structured data includes a specific character; If included, determining that the type of the original table is a nested header table; If not included, it is determined that the type of the original table is a non-nested header table.
4. The method according to claim 3, characterized in that The step of filling the structured data corresponding to the nested header table so that each column of the filled structured data has a corresponding header as a column index includes: Obtain data in the cell to the left of the cell where the specific character is located as filling data; Based on the filling data, the cell where the specific character is located is filled with data to obtain filled structured data.
5. The method according to claim 1, characterized in that Data is extracted based on the row index and column index of the filled structured data corresponding to the nested header table to obtain the target data, including: For the filled structured data corresponding to the nested header table, determine the first row of data in the structured data; Data extraction is performed on the filled structured data based on the row index, column index and the first row of data of the structured data to obtain target data.
6. The method according to claim 1, characterized in that The step of extracting data from the row index and column index of the filled structured data corresponding to the nested header table or the structured data corresponding to the non-nested header table to obtain target data further includes: Based on the name mapping table, the column indexes in the structured data are renamed so that the names of the column indexes corresponding to the same parameter in different structured data are consistent; Data extraction is performed on the row index and the renamed column index of the filled structured data corresponding to the nested header table or the structured data corresponding to the non-nested header table to obtain target data.
7. A data extraction device, characterized in that: The device comprises: An image processing module, used to identify data in the original table image to form structured data, wherein the structured data includes a row index and a column index; A judgment module, configured to determine the type of the original table based on the structured data, wherein the type includes a nested header table and a non-nested header table; A data filling module is used to fill the structured data corresponding to the nested header table, so that each column of the filled structured data has a corresponding header as a column index; The extraction module is used to extract data from the row index and column index of the filled structured data corresponding to the nested header table or the structured data corresponding to the non-nested header table to obtain target data.
8. The device according to claim 7, characterized in that The image processing module is further used for: Recognize the data in the original table image to obtain table data; The table data is formatted to obtain structured data.
9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.
10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to make a computer execute the method according to any one of claims 1-6.
