Document matching method and electronic device

Abstract

The application provides a document matching method and an electronic device, wherein the method comprises: identifying a to-be-queried document to extract a plurality of candidate character cell blocks contained in the to-be-queried document; matching a query keyword with each candidate character cell block to determine valid character cell blocks; and determining target logical lines where the valid character cell blocks are located according to position information of each valid character cell block.

Description

Technical Field

[0001] This application relates to the field of document processing technology, and more specifically, to a document matching method and an electronic device. Background Technology

[0002] In the field of document retrieval, the common approach is to identify the text contained in unstructured documents and then match keywords from the identified text. However, this text matching process requires error tolerance in order to improve recall, and it suffers from a lack of interpretability and significant performance variations under different data distributions. Summary of the Invention

[0003] The purpose of this application is to provide a document matching method and electronic device to improve the shortcomings of document recognition.

[0004] In a first aspect, the present invention provides a document matching method, comprising: identifying a document to be queried to extract multiple candidate character cells contained in the document to be queried; matching query keywords with each candidate character cell to determine valid character cells; and determining the target logical line where the valid character cells are located based on the position information of each valid character cell.

[0005] In the above implementation, the character cells contained in the document to be queried can be determined first according to the form of character cells. Compared with individual characters, the characters forming cells can better accommodate local errors and improve the accuracy and effectiveness of filtering effective character cells. Furthermore, after determining the effective character cells, the target logical row can be determined according to the position information of the effective character cells. Based on the presentation of the target logical row, the keywords to be filtered can be presented more accurately and effectively.

[0006] In an optional implementation, matching the query keywords with each candidate character cell to determine valid character cells includes: dividing the multiple candidate character cells into multiple logical columns, wherein each logical column includes one or more character cells; for each logical column, determining the column semantic label corresponding to the logical column based on the character cells contained in the logical column; filtering each column semantic label based on the query label in the query keywords to determine a semantic matching group that matches the query label; and filtering the candidate character cells in the semantic matching group based on the query content in the query keywords to determine valid character cells that match the query content.

[0007] In the above implementation, by parsing the document to be queried, multiple candidate character cells containing location information and character content are obtained. Based on the location information of each candidate character cell, all candidate character cells are divided into multiple logical columns. By extracting the column semantic labels of the logical columns, the logical columns with the same column semantic labels as the query labels are first filtered out. Then, the valid character cells that match the query content are retrieved from the filtered logical columns. Through semantic understanding, the false recall rate can be reduced while the accuracy rate can be improved.

[0008] In an optional implementation, the candidate character cell includes character content and position information. The position information includes the coordinates of the upper left corner and the lower right corner of the bounding rectangle of the character content. Dividing the multiple candidate character cells into multiple logical columns includes: for any two first candidate character cells and second candidate character cells, if the horizontal coordinate of the upper left corner of the bounding rectangle of the second candidate character cell is greater than the horizontal coordinate of the lower right corner of the bounding rectangle of the first candidate character cell, then the column information of the second candidate character cell is determined to be greater than the column information of the first candidate character cell; thereby determining the column information of all candidate character cells; and determining candidate character cells with the same column information as the same logical column to divide the multiple candidate character cells into multiple logical columns.

[0009] In an optional implementation, the step of filtering candidate character cells in the semantic matching group based on the query content in the query keyword to determine the valid character cells that match the query content includes: for each semantic matching group, calculating the matching value between the query content in the query keyword and the candidate character cells in the semantic matching group; and taking candidate character cells with matching values ​​greater than a first threshold as valid character cells.

[0010] In the above implementation, valid character cells can be determined by matching the query content, which can more accurately filter out the required character cells while having a certain degree of fault tolerance.

[0011] In an optional implementation, calculating the matching value between the query content in the query keyword and the candidate character cells in the semantic matching group includes: transforming the candidate character cells in the semantic matching group to obtain transformed character cells; and determining the matching value between the query content in the query keyword and the candidate character cells based on the transformed character cells and the query content in the query keyword.

[0012] In the above implementation, by modifying the candidate character cells, the error in character recognition can be mitigated, the accuracy of the calculated matching value can be improved, and thus the accuracy of the selected effective character cells can be further improved.

[0013] In an optional implementation, the transformation process of the candidate character cells to obtain transformed character cells includes: calculating the confidence level of each character in the candidate character cells; and masking characters with confidence levels less than a second threshold to obtain transformed character cells.

[0014] In the above implementation, some characters with low confidence can be masked out, so that the transformed character cell can more accurately represent the actual content presented in the document to be queried, thereby further improving the accuracy of the matching value calculation.

[0015] In an optional implementation, determining the matching value between the query content in the query keyword and the candidate character cell based on the transformed character cell and the query content in the query keyword includes: calculating the similarity between the transformed character cell and the query content in the query keyword, wherein the similarity is used as the matching value between the query content in the query keyword and the candidate character cell; or, determining the number of masked characters in the transformed character cell; if the number of masked characters is greater than a third threshold, then a set value is used as the matching value between the query content in the query keyword and the candidate character cell.

[0016] In an optional implementation, the third threshold is determined by: MT(k) =

[0017] log(len(Q) / k); where MT(k) represents the third threshold; Q represents the query content in the query keywords; len represents the length of Q; log() represents the logarithmic function; and k represents a set integer.

[0018] In an optional implementation, the set value is determined as follows: exp(MT-Ev(O)); where MT represents the third threshold; exp() represents an exponential function with the natural constant as the base; O represents any candidate character cell; and Ev(O) represents the difference between the number of masked characters in the transformed character cell corresponding to candidate character cell O and the third threshold.

[0019] In an optional implementation, the transformation process of the candidate character cells to obtain transformed character cells includes: calculating the confidence level of each character in the candidate character cells; and replacing characters with similar-looking words with confidence levels less than a second threshold to obtain transformed character cells.

[0020] In the above implementation, some characters with low confidence can be replaced with similar-looking words, so that the transformed character cell can more accurately represent the actual content presented in the document to be queried, thereby further improving the accuracy of the matching value calculation.

[0021] In an optional implementation, replacing characters with confidence levels less than a second threshold with preset similar characters to obtain transformed character cells includes: replacing characters with similar-looking words with confidence levels less than the second threshold to obtain multiple sets of transformed character cells; determining the matching value between the query content in the query keywords and the candidate character cells based on the transformed character cells and the query content in the query keywords includes: calculating the similarity between each set of transformed character cells and the query content in the query keywords; selecting the largest similarity from the similarity between each set of transformed character cells and the query content in the query keywords, and determining it as the matching value between the query content in the query keywords and the candidate character cells.

[0022] In the above implementation, by calculating multiple sets of matching values, the matching values ​​selected from multiple sets of matching values ​​can better express the matching between the query content and the candidate character cells.

[0023] In an optional implementation, calculating the matching value between the query content in the query keyword and the candidate character cells in the semantic matching group includes: calculating the confidence score of each character in the candidate character cells; masking characters with confidence scores less than a second threshold to obtain a first transformed character cell; replacing characters with similar-looking words with characters with confidence scores less than the second threshold to obtain a second transformed character cell; determining a first matching value between the query content in the query keyword and the candidate character cells based on the first transformed character cell and the query content in the query keyword; determining a second matching value between the query content in the query keyword and the candidate character cells based on the second transformed character cell and the query content in the query keyword; and selecting the larger matching value from the first matching value and the second matching value as the matching value between the query content in the query keyword and the candidate character cells in the semantic matching group.

[0024] In the above embodiments, the matching value can also be calculated based on two methods respectively, so as to select the matching value with the larger matching value and improve the accuracy of the matching value calculation.

[0025] In an optional implementation, calculating the matching value between the query content in the query keywords and the candidate character cells in the semantic matching group includes:

[0026] The confidence algorithm is used to calculate the matching value between the query content in the query keyword and the candidate character cell in the semantic matching group; wherein, the confidence algorithm is defined as: confidence(Q,O)=sim(Q,transform(O)); where Q represents the query content in the query keyword; O represents any candidate character cell; confidence(Q,O) represents the matching value between the query content in the query keyword and any candidate character cell;

[0027] sim(Q,transform(O)) represents the function that calculates the similarity between Q and transform(O); transform(O) represents the transformation function that transforms O.

[0028] In an optional implementation, determining the target logical row where the valid character cell is located based on the position information of each valid character cell includes: determining multiple initial logical rows based on the position information of each valid character cell; determining the number of valid character cells contained in each initial logical row; and determining the target logical row based on the number of cells in each initial logical row.

[0029] In an optional implementation, determining the target logic row based on the number of units in each of the initial logic rows includes: determining the target logic row as the initial logic row with the largest number of units among the initial logic rows.

[0030] Secondly, the present invention provides a document matching device, comprising: an identification module for identifying a document to be queried, thereby extracting multiple candidate character cells contained in the document to be queried; a matching module for matching query keywords with each candidate character cell to determine a valid character cell; and a determination module for determining the target logical row where the valid character cell is located based on the position information of each valid character cell.

[0031] Thirdly, the present invention provides an electronic device, including: a processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the machine-readable instructions are executed by the processor to perform the steps of the method described in any of the foregoing embodiments. Attached Figure Description

[0032] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 A block diagram illustrating an electronic device provided in an embodiment of this application; Figure 2 A flowchart of the document matching method provided in the embodiments of this application; Figure 3 A schematic diagram of the document to be queried in one example provided in this application embodiment; Figure 4 An optional flowchart of step 220 of the document matching method provided in an embodiment of this application; Figure 5 Another schematic diagram of the document to be queried in one example provided in this application embodiment; Figure 6 Another schematic diagram of the document to be queried in an example provided for an embodiment of this application; Figure 7 A schematic diagram of the functional modules of the document matching device provided in the embodiments of this application. Detailed Implementation

[0034] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0035] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0036] The inventors of this application have learned that in the field of document retrieval, a common practice is to first identify the text in unstructured documents using general character recognition algorithms, and then perform keyword matching. However, this approach has several problems: 1) The text recognition results lack row and column ranges, making it difficult to further enhance semantics; 2) The text recognized by the character recognition algorithm has logical breaks: for example, a single line in a cell may contain multiple blank or space-separated text segments, which may be recognized as multiple independent text parts. Subsequent matching of these text segments can only be achieved through word segmentation and aggregation to achieve logical matching of adjacent text, with a threshold mechanism added to control the matching effect. For example, a 50% threshold means that a keyword and the text to be matched are considered a successful match if they are 50% similar. However, this approach does not treat all the text in the cell as a whole for keyword matching. Therefore, in cases of severe breaks (such as when the column is very narrow, causing the cell content to be split into multiple lines of text), the character recognition algorithm's matching results will be completely unusable.

[0037] The inventors of this application also understand that in the process of matching text using the above method, in order to improve recall, it is also necessary to make the recognition results of the character recognition algorithm more tolerant (for example, recognizing the string PN10 as the string PNT0). Therefore, the above threshold must also take into account the tolerance-related factors and be determined by empirical values. Under different data distribution conditions, it lacks interpretability and the effect varies greatly.

[0038] Based on the above research, this application provides a document matching method that can alleviate the problems of difficulty in adjusting the matching threshold and the inability to add semantic information to enhance the matching effect due to the lack of logical row and column information, thereby improving the effect of keyword document matching.

[0039] For example, the document matching method of this application can be used in Robotic Process Automation (RPA) technology to achieve character matching in documents during the RPA process. RPA technology can simulate employees' daily computer operations using a keyboard and mouse, replacing humans in tasks such as logging into systems, operating software, reading and writing data, downloading files, and reading emails. Using automated robots as virtual labor for enterprises can free employees from repetitive, low-value tasks, allowing them to focus on high-value-added work. This enables enterprises to reduce costs and increase efficiency while undergoing digital and intelligent transformation.

[0040] RPA (Robotic Process Automation) is a software robot that replaces manual tasks in business processes and interacts with computer front-end systems like a human. Therefore, RPA can be seen as a software-based program robot running on a personal PC or server, mimicking user actions on a computer to automate repetitive tasks such as retrieving emails, downloading attachments, logging into systems, and data processing and analysis—faster, more accurate, and more reliable. While both RPA and traditional physical robots address the speed and accuracy issues in human work through specific rules, traditional physical robots are hardware-software hybrids requiring specific hardware support and software to perform tasks. RPA robots, on the other hand, are purely software-based; once the appropriate software is installed, they can be deployed to any PC or server to complete the assigned tasks.

[0041] In other words, RPA is a method and related technologies that utilize "digital employees" to perform business operations in place of humans. Essentially, RPA uses software automation technology to simulate human operation of computer systems, software, web pages, and documents, retrieving business information, executing business actions, and ultimately achieving automated process processing, saving labor costs, and improving processing efficiency. As the description shows, to implement RPA, the target content to be operated on must first be found in the document or screen before automatic operation can be performed. Therefore, keyword retrieval in the document based on input keywords becomes one of the key technologies for implementing RPA.

[0042] The document matching method provided in this application utilizes Optical Character Recognition (OCR) technology. OCR technology refers to the process of translating the shapes of characters on paper into computer text using character recognition methods. That is, it involves scanning text documents, then analyzing and processing the image files to obtain text and layout information.

[0043] To facilitate understanding of this embodiment, the electronic device that performs the document matching method disclosed in this application embodiment will first be introduced.

[0044] like Figure 1 The diagram shown is a block diagram of an electronic device. The electronic device 100 may include a memory 111, a memory controller 112, a processor 113, a peripheral interface 114, an input / output unit 115, and a display unit 116. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device 100. For example, the electronic device 100 may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0045] The aforementioned memory 111, memory controller 112, processor 113, peripheral interface 114, input / output unit 115, and display unit 116 are electrically connected directly or indirectly to each other to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The aforementioned processor 113 is used to execute executable modules stored in the memory.

[0046] The memory 111 can be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory 111 stores programs, and the processor 113 executes these programs upon receiving execution instructions. The methods executed by the electronic device 100 as defined in any embodiment of this application can be applied to or implemented by the processor 113.

[0047] The aforementioned processor 113 may be an integrated circuit chip with signal processing capabilities. The processor 113 may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a digital signal processor (DSP) or an application-specific integrated circuit (ASIC).

[0048] Application Specific Integrated Circuit (ASIC), Field Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. These devices can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0049] The peripheral interface 114 described above couples various input / output devices to the processor 113 and the memory 111. In some embodiments, the peripheral interface 114, the processor 113, and the memory controller 112 can be implemented on a single chip. In other instances, they can be implemented on separate chips.

[0050] The input / output unit 115 described above is used to provide user input data. The input / output unit 115 may be, but is not limited to, a mouse and keyboard, etc.

[0051] The aforementioned display unit 116 provides an interactive interface (e.g., a user interface) between the electronic device 100 and the user, or displays image data for the user's reference. In this embodiment, the display unit can be a liquid crystal display (LCD) or a touch display. If it is a touch display, it can be a capacitive touchscreen or a resistive touchscreen that supports single-point and multi-point touch operations. Supporting single-point and multi-point touch operations means that the touch display can sense touch operations generated simultaneously from one or more locations on the touch display and pass the sensed touch operations to the processor for calculation and processing.

[0052] In this embodiment, the memory 111 of the electronic device 100 stores a computer program for an OCR algorithm that recognizes documents. When the processor 113 calls the OCR algorithm, it can extract character information from the document.

[0053] When the electronic device 100 in this embodiment is used to execute the steps in the document matching method, the display unit 116 of the electronic device 100 can display the matching result of the document to be queried.

[0054] The electronic device 100 in this embodiment can be used to execute various steps in the various methods provided in the embodiments of this application. The implementation process of the document matching method is described in detail below through several embodiments.

[0055] Please see Figure 2 This is a flowchart of the document matching method provided in the embodiments of this application. The following will describe... Figure 2 The specific process shown will be explained in detail.

[0056] Step 210: Identify the document to be queried to extract multiple candidate character cells contained in the document.

[0057] Optionally, a character recognition algorithm, such as an OCR algorithm, can be used to recognize the candidate character cells contained in the document to be queried. The document to be queried can be an unstructured document containing tables, such as a Portable Document Format (PDF) document or an image document. The recognition of the document to be queried can be performed using a parsing tool, such as a PDF parser or an OCR tool. A candidate character cell refers to the smallest bounding rectangle of each string in the table document; this candidate character cell can be denoted as O.

[0058] For example, the candidate character cell may include location information and character content.

[0059] The location information can be the coordinates of the top-left and bottom-right corners of the smallest bounding rectangle of the character content in each candidate character cell. For example, the location information of any candidate character cell can be represented as [O.x0, O.y0, O.x1, O.y1], where (O.x0, O.y0) represents the top-left corner coordinates of the smallest bounding rectangle, and (O.x1, O.y1) represents the bottom-right corner coordinates of the smallest bounding rectangle. For example, the origin can be the top-left corner of the entire document being queried. The character content refers to the specific characters contained in the candidate character cell; for example, it may include numbers, letters, symbols, etc.

[0060] Step 220: Match the query keywords with each candidate character cell to determine the valid character cells.

[0061] For example, the query keywords can be keywords provided based on the actual query requirements. These query keywords can be keywords related to the document being queried.

[0062] The query keyword can be represented as Q. For example, the query keyword Q can include query tags and query content. For instance, the format of the query keyword Q can be "query tags = query content". For example, the query keyword could be "equipment name = injection molding machine", "model number = model number", etc.

[0063] =MA3600 / 2250G, "Price = 360000", etc. Here, the query tag for each query keyword Q can be represented as Q.tag, and the query content for each query keyword Q can be represented as Q.val.

[0064] For any target candidate cell among multiple candidate character cells, the query label of the query keyword is compared with the first candidate cell in the column containing the target candidate cell. The query content of the query keyword is then matched with the target candidate cell to determine if it can be considered a valid character cell. If the character content of the first candidate cell in the column containing the target candidate cell has the same meaning as the query label of the query keyword, and the character content of the target candidate cell is also the same as the query content of the query keyword, then the target candidate cell can be determined as a valid character cell.

[0065] exist Figure 3 In the example shown, twenty-eight candidate character cells were identified in the document to be queried, including seven columns and four rows of candidate character cells. Taking the query keyword "Q" as an example, if the current match is the candidate character cell in the third row and second column with the character content "Haitian Injection Molding Machine", this candidate character cell cannot be considered a valid character cell because its character content is different from "Haitian Injection Molding Machine" and the query content Q.val of the query keyword "Q" is 360000. Furthermore, the character content of the first candidate character cell in the column containing "Haitian Injection Molding Machine" is "model", which is different from the meaning expressed by the query tag Q.tag of the query keyword "Q" being price. For example, if the current match is a candidate cell in the fourth row and fifth column with the character content "2,220,000.00", this candidate cell cannot be considered a valid cell because its character content is different from the query content Q.val of "2,220,000.00" which is 360000. Furthermore, the first candidate cell in the column containing "2,220,000.00" has the character content "total equipment price (RMB / RMB)," which has the same meaning as the query tag Q.tag price of the query keyword Q. For example, if the current match is a candidate cell in the third row and fifth column with the character content "360,000.00", since this character content is the same as "360,000.00" and the value of Q.val is 360000 for the query content with the keyword "Q", and the character content of the first candidate cell in the column containing "360,000.00" is "total price of equipment (RMB / RMB)", which has the same meaning as the price expressed by the query tag Q.tag for the keyword "Q", this candidate cell can be considered a valid cell.

[0066] Step 230: Determine the target logical row where the valid character cell is located based on the position information of each valid character cell.

[0067] Taking the above example, if the selected valid character cell is the candidate character cell in the third row and fifth column, then the determined target logical row can be... Figure 3 The content of the third line shown. By presenting the entire logical row, the complete information of the content to be queried can be expressed more fully.

[0068] For documents containing tables to be queried, such as Figure 4 As shown, step 220 above may include steps 221 to 224.

[0069] Step 221: Divide multiple candidate character cells into multiple logical columns.

[0070] Each logical column includes one or more character cells. For example, each candidate character cell can be labeled with row information and column information, which can be labeled as O.row and O.col, respectively.

[0071] The row and column information of each candidate character cell can be determined based on its position information.

[0072] For any two first candidate character cells and second candidate character cells among all the candidate character cells identified in the document to be queried, if the x-coordinate of the upper left corner of the outer rectangle of the second candidate character cell is greater than the x-coordinate of the lower right corner of the outer rectangle of the first candidate character cell, then the column information of the second candidate character cell is greater than the column information of the first candidate character cell; if the y-coordinate of the upper left corner of the outer rectangle of the second candidate character cell is greater than the y-coordinate of the lower right corner of the outer rectangle of the first candidate character cell, then the column information of the second candidate character cell is greater than the row information of the first candidate character cell.

[0073] by Figure 3For example, taking the top-left corner of the entire document as the origin, the row and column information of the candidate character cell with the smallest top-left corner coordinate of its smallest bounding rectangle can also be recorded as the minimum value. For instance, if the candidate character cell contains "serial number", since both the x-coordinate and y-coordinate of its top-left corner are minimum, the row and column information of this "serial number" candidate character cell can both be one. However, for the candidate character cell containing "product name", the x-coordinate of its top-left corner is greater than the x-coordinate of the bottom-right corner of the candidate character cell containing "serial number", so its column information can be greater than the column information of the candidate character cell containing "serial number". Its y-coordinate is not greater than the y-coordinate of the bottom-right corner of the candidate character cell containing "serial number", therefore its row information can not be greater than the column information of the candidate character cell containing "serial number". Thus, it can be determined that the candidate character cell containing "product name" has one row information and two column information. By analogy, the row information O.row and column information O.col of all candidate character cells can be determined.

[0074] Candidate character cells with the same column information can be identified as belonging to the same logical column.

[0075] like Figure 5 As shown, the column enclosed by the dashed box can be considered a logical column. Figure 5 In the example shown, seven logical columns can be identified.

[0076] Step 222: For each logical column, determine the column semantic label corresponding to the logical column based on the character cells contained in the logical column.

[0077] The column semantic tags of logical columns are used to characterize the semantic information of all candidate character cells in the logical column. Optionally, the column semantic tags of the table document can be collected first, and relevant rules (such as header rules and column data rules below) can be configured for different column semantic tags (e.g., device name, manufacturer). In practical applications, for any logical column, the character content of the candidate character cells in the logical column is matched against the rules to determine the column semantic tag of that logical column.

[0078] Rule matching includes matching header rules and matching column data rules. Matching header rules is more reliable; it can be done first, and if the match fails, then matching column data rules can proceed. If the header rule match is successful, then matching column data rules is unnecessary. Header rules refer to the keywords or expressions that the header must match. Column data rules refer to the expressions that the column data (excluding the header) must match. For example, the header rule configured for the column semantic label "Price" is "^.*Price|Total Price|Original Purchase Value|Amount|Evaluated Net Value.*$"; the column data rule is "((\d{1,3},)+\d{3}|\d+)(.\d{1,2})".

[0079] For any logical column, if the character content of any candidate character cell in the logical column matches the header rules configured for the specified column semantic label, then the column semantic label of the logical column is determined to be the specified column semantic label.

[0080] For example, if the content of a candidate character cell in a logical column is "total price", since the column semantic label "price" corresponds to the configured header rule "^.*price|total price|purchase original value|amount|appraised net value.*$", then the content of the character is considered to conform to the configured header rule corresponding to the specified column semantic label "price", and the column semantic label of the logical column is "price".

[0081] If candidate character cells in some logical columns of a table document cannot match the header rules, column data rules can be matched. For example, for any logical column, if the character content of candidate character cells in that logical column exceeds a preset proportion and matches the column data rules configured for the specified column semantic label, then the column semantic label for that logical column is determined to be the specified column semantic label.

[0082] The preset ratio can be 2 / 3, 4 / 5, etc. For example, if the character content of candidate character cells in a logical column that exceeds the preset ratio matches the column data rules configured for the specified column semantic label "price", then the column semantic label for that logical column is "price".

[0083] Optionally, for any logical column, the character content of the candidate character cell in the logical column is used as the input of the trained column semantic label extraction model to obtain the column semantic label of the logical column output by the column semantic label extraction model.

[0084] Among them, the column semantic label extraction model can be trained in advance using labeled corpora. For example... Figure 3As shown, the first column can be labeled with the semantic label "Serial Number," the second with "Product Name," the third with "Model Number," the fourth with "Quantity," the fifth with "Price," the sixth with "Invoice," and the seventh with "Supplier." The text content of each column can be used as input, and the labeled semantic labels as output, to perform machine learning and train a column semantic label extraction model. This model can be a classification algorithm based on a pre-trained model (such as BERT). The loss function for the classification algorithm can be the cross-entropy loss function typical of classification tasks.

[0085] In practical applications, for a certain logical column, the character contents of all candidate character cells in the logical column can be concatenated in order and then input into the column semantic label extraction model. The output of the column semantic label extraction model is the column semantic label of the logical column.

[0086] In one embodiment, for any logical column, the column semantic label can be determined first by the rule matching method described above. If the column semantic label cannot be determined by rule matching, the column semantic label can be determined by the column semantic label extraction model described above.

[0087] Step 223: Filter the semantic labels in each column based on the query tags in the query keywords to determine the semantic matching groups that match the query tags.

[0088] Here, the query tag for each query keyword Q can be denoted as Q.tag, and its value range can be column semantic tags. The query content for each query keyword Q can be denoted as Q.val.

[0089] The semantic matching group includes logical columns whose column semantic labels are the same as the query keyword Q's query label. Specifically, the column semantic label of each logical column OC can be denoted as OC.tag. Logical columns OC whose OC.tag is the same as Q.tag are grouped together (i.e., semantic matching groups).

[0090] For example, if a query includes multiple query keywords, and if the query tags Q.tag of the multiple query keywords are not repeated, then multiple semantic matching groups are obtained. A semantic matching group contains a query keyword Q and one or more logical columns OC. The semantic matching group can be denoted as MG.

[0091] Step 224: Filter the candidate character cells in the semantic matching group based on the query content in the query keyword to determine the valid character cells that match the query content.

[0092] For example, the query content in the query keyword can be compared with the character content of candidate character cells in the semantic matching group to identify valid character cells in the semantic matching group that are the same as the query content in the query keyword.

[0093] For any semantic matching group, the query content Q.val of the query keyword Q in the semantic matching group can be compared with the character content O.val of each candidate character cell in the semantic matching group to filter out the valid character cells.

[0094] In an alternative implementation, step 224 may include steps 2241 and 2242.

[0095] Step 2241: For each semantic matching group, calculate the matching value between the query content in the query keyword and the candidate character cell in the semantic matching group.

[0096] The query content in the query keywords within the semantic matching group can be calculated with the character content of each candidate character cell in the semantic matching group to determine the matching value between the query content in the query keywords and the candidate character cells in the semantic matching group.

[0097] For example, the similarity between the query content in the query keywords and the character content of the candidate character cells can be calculated to determine the matching value between the query content in the query keywords and the candidate character cells in the semantic matching group. For instance, the vector of the query content in the query keywords and the vector of the character content of the candidate character cells can be extracted, the distance between the two vectors can be calculated, and the distance between the two vectors can be used as the matching value between the query content in the query keywords and the candidate character cells in the semantic matching group.

[0098] For example, similarity calculation algorithms such as Jaccard and Dice can be used to calculate the similarity between the query content in the query keywords and the character content of the candidate character cells.

[0099] Step 2242: Select candidate character cells with matching values ​​greater than the first threshold as valid character cells.

[0100] The first threshold can be a preset value, such as 0.8, 0.7, 0.85, etc.

[0101] Considering that the character content in the candidate character cells identified by the character recognition algorithm may contain errors, the candidate character cells can be transformed to reduce these errors and improve their reliability. Therefore, step 2241 above may include: transforming each candidate character cell in the semantic matching group to obtain transformed character cells; and determining the matching value between the transformed character cells and the query content in the query keyword and the candidate character cells based on the query content in the query keyword.

[0102] For example, during the transformation process, characters in the candidate character cell that may contain errors can be transformed.

[0103] In one implementation, transforming the candidate character cell to obtain a transformed character cell may include: calculating the confidence score of each character in the candidate character cell; and masking characters with confidence scores less than a second threshold to obtain the transformed character cell.

[0104] For example, the confidence level of each character in the character content of the candidate character cell can be determined first.

[0105] In this embodiment, after the character recognition algorithm performs whole-image recognition on the document to be queried in step 210 and obtains the position information of each candidate character cell, the position information of each character in the candidate character cell can be supplemented to enrich the position information of the candidate character cell. The position information of the character can also be represented by coordinates, which can be the coordinates of the upper left and lower right corners of the character's bounding rectangle. The bounding rectangle of the character can be the smallest bounding rectangle of the character.

[0106] In this embodiment, the confidence level of each character in each candidate character cell can be determined, and this confidence level can be denoted as O.CharConf.

[0107] For example, the confidence score of each character can be determined based on the character content of its candidate character cell or the probability of the character appearing in its surrounding environment. For instance, if the character content in a candidate character cell represents the price of an item, then most of the characters in that cell are numerical values, but one character is the letter "o" which resembles the number 0. This indicates a higher probability of misidentification, and the confidence score for that character might be low. Similarly, if the character content in a candidate character cell is mostly Chinese characters, but one character is the letter "T" which resembles a Chinese character, then the probability of misidentification is high, and the confidence score for that character might be low. Conversely, if the character content in a candidate character cell consists entirely of Chinese characters, and these characters form a reasonable phrase, then the probability of correctly identifying each character in that cell is high, and the confidence score for each character might be high.

[0108] The second threshold can be a value that can be set as needed; for example, the second threshold can be 0.95, 0.9, 0.85, etc.

[0109] For example, characters with a confidence level less than the second threshold can be identified as untrusted characters. An unmatchable placeholder character can be used, such as replacing the untrusted character with an invisible one. This prevents untrusted characters from being counted as correct characters during subsequent similarity calculations. For each candidate character cell, the position of the masked untrusted character can also be recorded when replacing it; the number of masked untrusted characters can also be recorded for each candidate character cell.

[0110] Optionally, the similarity between the transformed character cell and the query content in the query keyword is calculated, wherein the similarity serves as the matching value between the query content in the query keyword and the candidate character cell. For example, the similarity between the character content of the transformed character cell and the query content in the query keyword can be calculated.

[0111] For example, similarity calculation algorithms such as Jaccard and Dice can be used to calculate the similarity between the character content of the transformed character cell and the query content in the query keywords.

[0112] Optionally, the number of masked characters in the transformed character cell is determined; if the number of masked characters is greater than a third threshold, the set value is used as the matching value between the query content in the query keyword and the candidate character cell; if the number of masked characters is not greater than the third threshold, the similarity between the transformed character cell and the query content in the query keyword is used as the matching value between the query content in the query keyword and the candidate character cell.

[0113] In this embodiment, a similarity threshold can also be preset, and this similarity threshold can be a value that can be set as needed. This similarity threshold can be denoted as ST.

[0114] If the similarity between the transformed character cell and the query content in the query keyword is less than the similarity threshold, then the similarity can be used as the matching value between the query content in the query keyword and the candidate character cell.

[0115] If the similarity between the transformed character cell and the query content in the query keyword is not less than the similarity threshold, then the number of masked characters in the transformed character cell is determined; if the number of masked characters is greater than the third threshold, then the set value is used as the matching value between the query content in the query keyword and the candidate character cell; if the number of masked characters is not greater than the third threshold, then the similarity between the transformed character cell and the query content in the query keyword is used as the matching value between the query content in the query keyword and the candidate character cell.

[0116] Optionally, the third threshold is determined as follows: MT(k) = log(len(Q) / k);

[0117] Where MT(k) represents the third threshold; Q represents the query content in the query keyword; len represents the length of Q; log() represents the logarithmic function; and k represents a set integer.

[0118] Optionally, the third threshold can also be set to a value as needed, for example, the third threshold can be equal to 2, 3, 4, 5, etc.

[0119] Optionally, the above setting value is determined by the following method: exp(MT-Ev(O));

[0120] Where MT represents the third threshold; exp() represents an exponential function with the natural constant as the base; O represents any candidate character cell; Ev(O) represents the difference between the number of masked characters in the transformed character cell corresponding to candidate character cell O and the third threshold.

[0121] Alternatively, the set value can be a specific value, for example, the set value can be 0.

[0122] In another implementation, transforming the candidate character cell to obtain a transformed character cell may include: calculating the confidence score of each character in the candidate character cell; and replacing characters with similar-looking words with confidence scores less than a second threshold to obtain the transformed character cell.

[0123] Optionally, the above-mentioned replacement of characters with confidence levels less than the second threshold with preset similar characters to obtain transformed character cells includes: replacing characters with confidence levels less than the second threshold with similar-looking words to obtain multiple sets of transformed character cells.

[0124] A model for similar-looking words can be pre-built, which can be denoted as SYNO. The input of this model is a string and the positions of untrusted characters in it. The output is the top k new strings after replacing the untrusted characters with similar words based on their positions in the input.

[0125] For example, in this model, you can input candidate character cell O and the positions of untrusted characters whose confidence is less than the second threshold into the model SYNO to obtain k new transformed character cells.

[0126] The above-mentioned determination of the matching value between the query content in the query keyword and the candidate character cell based on the transformed character cell and the query content in the query keyword includes: calculating the similarity between each group of transformed character cells and the query content in the query keyword; selecting the largest similarity from each group of similarity between the transformed character cells and the query content in the query keyword, and determining it as the matching value between the query content in the query keyword and the candidate character cell.

[0127] For example, the query content Q.val of the query keyword can be input into a similarity calculation function to obtain k similarity values, and the largest similarity value can be taken as the final confidence score.

[0128] In this embodiment, step 2241 may include: calculating the confidence score of each character in the candidate character cell; masking characters with confidence scores less than a second threshold to obtain a first transformed character cell; replacing characters with similar-looking words with characters with confidence scores less than the second threshold to obtain a second transformed character cell; determining a first matching value between the query content in the query keyword and the candidate character cell based on the first transformed character cell and the query content in the query keyword; determining a second matching value between the query content in the query keyword and the candidate character cell based on the second transformed character cell and the query content in the query keyword; and selecting the larger matching value from the first matching value and the second matching value as the matching value between the query content in the query keyword and the candidate character cell in the semantic matching group.

[0129] The method for determining the first transformed character cell and the second transformed character cell can refer to the two character transformation processing methods provided above, and will not be repeated here.

[0130] The method for determining the first matching value described above can refer to the method for determining the matching value calculated from the changed character cells obtained through masking processing. The method for determining the second matching value described above can refer to the method for determining the matching value calculated from the changed character cells obtained through similar-looking word replacement processing, and will not be repeated here.

[0131] In this embodiment, step 2241 may include: using a confidence algorithm to calculate the matching value between the query content in the query keyword and the candidate character cell in the semantic matching group.

[0132] The confidence algorithm is defined as follows: confidence(Q,O)=sim(Q,transform(O)); where Q represents the query content in the query keywords; and O represents any candidate character cell.

[0133] confidence(Q,O) represents the matching value between the query content in the query keyword and any candidate character cell; sim(Q,transform(O); sim(Q,transform(O)) represents the function to calculate the similarity between Q and transform(O); transform(O) represents the transformation function to transform O.

[0134] In some implementations, step 230 may include: determining multiple initial logical rows based on the position information of each valid character cell; determining the number of valid character cells contained in each initial logical row; and determining a target logical row based on the number of cells in each initial logical row.

[0135] The above-mentioned determination of the target logic row based on the number of units in each initial logic row may include: determining the target logic row as the initial logic row with the largest number of units among the initial logic rows.

[0136] The obtained valid character cells can be grouped according to the row information O.row to obtain a logical row OR containing one or more valid character cells O.

[0137] For example, the number of valid character cells in the logical row OR can be denoted as ORN. The logical row OR with the largest ORN can be selected as the target logical row, or it can be used as the final matching result of the document to be queried.

[0138] For example, if multiple ORNs exist with the same maximum logical OR, these multiple ORNs with the same maximum logical OR can be retained and used as the target logical row. Finally, all retained O's are output, including information about the row containing the valid character cell O and information about all O's.

[0139] The information of the row containing the valid character cell O can be defined by the largest bounding rectangle of the position information of all character cells in the same row.

[0140] like Figure 6 As shown, the column semantic label in the third column is "Model Number", and the column semantic label in the fifth column is "Price". The query keywords are: Model Number = MA3600 / 2250G, Price = 360000. It can be determined that the valid character cells are... Figure 6 The solid-line box indicates the matching results; then, a dashed box is used to present the information of the row containing the valid character cell O.

[0141] In this embodiment, a table recognition tool is first used to identify character cells in the document to be queried. Then, keyword-based document matching is performed using row and column information, improving the completeness of the text within the table cells and mitigating the problem of text blocks failing to match in narrow column cases. Furthermore, combining column semantic tags makes column-based matching more accurate. In the finest-grained similarity matching method, finer-grained threshold control is more interpretable and yields better results in document queries.

[0142] Based on the same application concept, this application also provides a document matching device corresponding to the document matching method. Since the principle of the device in this application is similar to that of the aforementioned document matching method embodiment, the implementation of the device in this application can refer to the description in the above method embodiment, and the repeated parts will not be repeated.

[0143] Please see Figure 7This is a functional module diagram of the document matching device provided in this application embodiment. Each module in the document matching device in this embodiment is used to execute the steps in the above method embodiments. The document matching device includes: an identification module 310, a matching module 320, and a determination module 330; wherein the contents of each module are as follows: the identification module 310 is used to identify the document to be queried, so as to extract multiple candidate character cells contained in the document to be queried; the matching module 320 is used to match the query keywords with each candidate character cell to determine the valid character cell; the determination module 330 is used to determine the target logical line where the valid character cell is located based on the position information of each valid character cell.

[0144] Furthermore, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the document matching method described in the above method embodiments.

[0145] The computer program product of the document matching method provided in this application includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the steps of the document matching method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0146] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0147] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0148] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks. It should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0149] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application. It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0150] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A document matching method, characterized in that, include: The document to be queried is identified to extract multiple candidate character cells contained in the document; The query keywords are matched with each candidate character cell to determine the valid character cells; Based on the position information of each valid character cell, the target logical row where the valid character cell is located is determined, including: determining multiple initial logical rows based on the position information of each valid character cell; determining the number of valid character cells contained in each initial logical row; and determining the target logical row based on the number of cells in each initial logical row. The step of matching query keywords with each candidate character cell to determine valid character cells includes: dividing the multiple candidate character cells into multiple logical columns, wherein each logical column includes one or more character cells; for each logical column, determining the column semantic label corresponding to the logical column based on the character cells contained in the logical column; filtering each column semantic label based on the query label in the query keyword to determine the semantic matching group that matches the query label; and filtering the candidate character cells in the semantic matching group based on the query content in the query keyword to determine the valid character cells that match the query content. The step of filtering candidate character cells in the semantic matching group based on the query content in the query keywords to determine valid character cells that match the query content includes: For each semantic matching group, calculate the matching value between the query content in the query keyword and the candidate character cell in the semantic matching group; Candidate character cells with matching values ​​greater than the first threshold are considered valid character cells; The step of calculating the matching value between the query content in the query keyword and the candidate character cells in the semantic matching group includes: calculating the confidence score of each character in the candidate character cells; masking characters with confidence scores less than a second threshold to obtain a first transformed character cell; replacing characters with similar-looking words with characters with confidence scores less than the second threshold to obtain a second transformed character cell; determining a first matching value between the query content in the query keyword and the candidate character cells based on the first transformed character cell and the query content in the query keyword; and determining a second matching value between the query content in the query keyword and the candidate character cells based on the second transformed character cell and the query content in the query keyword. The larger matching value is selected from the first matching value and the second matching value, and used as the matching value between the query content in the query keyword and the candidate character cell in the semantic matching group.

2. The method according to claim 1, characterized in that, The candidate character cell includes character content and position information, wherein the position information includes the coordinates of the upper left corner and the lower right corner of the bounding rectangle of the character content; The step of dividing the multiple candidate character cells into multiple logical columns includes: For any two first candidate character cells and second candidate character cells, if the x-coordinate of the upper left corner of the outer rectangle of the second candidate character cell is greater than the x-coordinate of the lower right corner of the outer rectangle of the first candidate character cell, then it is determined that the column information of the second candidate character cell is greater than the column information of the first candidate character cell. This determines the column information for all candidate character cells; Candidate character cells with the same column information are identified as the same logical column, so as to divide the multiple candidate character cells into multiple logical columns.

3. The method according to claim 1, characterized in that, The calculation of the matching value between the query content in the query keyword and the candidate character cell in the semantic matching group includes: For each candidate character cell in the semantic matching group, the candidate character cells are transformed to obtain transformed character cells; Based on the transformed character cell and the query content in the query keyword, determine the matching value between the query content in the query keyword and the candidate character cell.

4. The method according to claim 3, characterized in that, The transformation process of the candidate character cells to obtain transformed character cells includes: Calculate the confidence score of each character in the candidate character cell; Characters with a confidence level less than the second threshold are masked to obtain transformed character cells.

5. The method according to claim 4, characterized in that, The step of determining the matching value between the query content in the query keyword and the candidate character cell based on the transformed character cell and the query content in the query keyword includes: Calculate the similarity between the transformed character cell and the query content in the query keyword, wherein the similarity is used as the matching value between the query content in the query keyword and the candidate character cell; or, Determine the number of masked characters in the transformed character cell; if the number of masked characters is greater than a third threshold, then use a set value as the matching value between the query content in the query keyword and the candidate character cell.

6. The method according to claim 5, characterized in that, The third threshold is determined in the following manner: MT(k) = log(len(Q) / k); Where MT(k) represents the third threshold; Q represents the query content in the query keywords; len represents the length of Q; log() represents the logarithmic function; and k represents a set integer.

7. The method according to claim 5, characterized in that, The set value is determined in the following way: exp(MT- Ev(O) ); Wherein, MT represents the third threshold; exp() represents an exponential function with the natural constant as the base; O represents any candidate character cell; Ev(O) represents the difference between the number of masked characters in the transformed character cell corresponding to candidate character cell O and the third threshold.

8. The method according to claim 3, characterized in that, The transformation process of the candidate character cells to obtain transformed character cells includes: Calculate the confidence score of each character in the candidate character cell; Characters with a confidence level less than the second threshold are replaced with similar-looking words to obtain transformed character cells.

9. The method according to claim 8, characterized in that, The step of replacing characters with confidence levels less than a second threshold with preset similar characters to obtain transformed character cells includes: Characters with a confidence level less than the second threshold are replaced with similar-looking words to obtain multiple sets of transformed character cells; The step of determining the matching value between the query content in the query keyword and the candidate character cell based on the transformed character cell and the query content in the query keyword includes: Calculate the similarity between each set of transformed character cells and the query content in the query keywords; The highest similarity is selected from the similarity scores between the transformed character cells in each group and the query content in the query keywords, and this is determined as the matching value between the query content in the query keywords and the candidate character cells.

10. The method according to claim 1, characterized in that, The calculation of the matching value between the query content in the query keyword and the candidate character cell in the semantic matching group includes: Using a confidence algorithm, calculate the matching value between the query content in the query keywords and the candidate character cells in the semantic matching group; The confidence algorithm is defined as follows: confidence(Q,O)=sim(Q,transform(O)); Where Q represents the query content in the query keyword; O represents any candidate character cell; confidence(Q,O) represents the matching value between the query content in the query keyword and any candidate character cell; sim(Q,transform(O); sim(Q,transform(O)) represents the function to calculate the similarity between Q and transform(O); transform(O) represents the transformation function to transform O.

11. The method according to claim 1, characterized in that, Determining the target logical row based on the number of units in each of the initial logical rows includes: The target logic row is determined by identifying the initial logic row with the largest number of units among all the initial logic rows.

12. An electronic device, characterized in that, include: The processor and memory, wherein the memory stores machine-readable instructions executable by the processor, wherein when the electronic device is running, the machine-readable instructions are executed by the processor to perform the steps of the method as described in any one of claims 1 to 11.

Images