Text search method and device, computer equipment and computer readable storage medium

By determining the document type and applying appropriate conversion models, the method enhances PDF text search accuracy by removing headers and footers, addressing the interference issues in existing tools.

CN119226434APending Publication Date: 2024-12-31PING AN BANK CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411173442.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-23
Publication Date
2024-12-31

AI Technical Summary

Technical Problem

Existing text search tools for PDF documents suffer from low accuracy due to the interference of images, headers, and footers, especially in multi-page documents, making it difficult to accurately locate and position desired text.

Method used

A method that determines the document type of a PDF document, applies a suitable conversion model (character extraction or OCR) based on the type, removes headers and footers, and performs precise text matching to enhance search accuracy.

Benefits of technology

Improves the accuracy and efficiency of text search in PDF documents by minimizing the impact of headers and footers, ensuring accurate text location and positioning.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119226434A_ABST
    Figure CN119226434A_ABST
Patent Text Reader

Abstract

The invention provides a text search method and device, computer equipment and a computer readable storage medium, and the method comprises the steps: obtaining a target PDF document and a text search request of the target PDF document, the document search request comprising a to-be-searched text; determining a document type of the target PDF document based on a document type identification rule; determining a document conversion model according to the document type of the target PDF document, and performing document conversion processing on the target PDF document based on the determined document conversion model to obtain a first text corresponding to the target PDF document; determining a header text and / or a footer text in the first text, and deleting the header text and / or the footer text to obtain a second text; and determining a target text matched with the to-be-searched text in the second text, and outputting text information and position information of the target text. According to the method and the device, the text searching and positioning precision of the PDF document is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a text search method, device, computer equipment and computer-readable storage medium. Background Art

[0002] At present, most companies have realized electronic office work. Customer information and corresponding product information are generally stored in terminals or servers through electronic documents. In the case of a large amount of information, such as documents corresponding to insurance policies or documents corresponding to financial information such as customer investment and financial management, they all need to contain more information. Multi-page PDF documents are used to meet the information recording needs. However, if text search and positioning are required in multi-page PDF documents to realize information query, text segment search tools are generally used to quickly locate the text to be searched. However, these text segment search tools will be affected by images, headers or footers in the PDF document during the text search process, resulting in low text search accuracy of the text segment search tools for PDF documents. Summary of the invention

[0003] The present application provides a text search method, apparatus, computer device and computer-readable storage medium, aiming to improve the text search accuracy of PDF documents.

[0004] In a first aspect, the present application provides a text search method, the text search method comprising the following steps:

[0005] Obtaining a target PDF document and a text search request for the target PDF document, wherein the document search request includes text to be searched;

[0006] Determine the document type of the target PDF document based on a document type identification rule;

[0007] Determining a document conversion model according to the document type of the target PDF document, and performing document conversion processing on the target PDF document based on the determined document conversion model to obtain a first text corresponding to the target PDF document;

[0008] Determine the header text and / or footer text in the first text, and delete the header text and / or footer text to obtain a second text;

[0009] A target text matching the text to be searched is determined in the second text, and text information and position information of the target text are output.

[0010] In a second aspect, the present application further provides a text search device, the text search device comprising:

[0011] An information acquisition module, used to acquire a target PDF document and a text search request of the target PDF document, wherein the document search request includes a text to be searched;

[0012] A type determination module, used to determine the document type of the target PDF document based on a document type identification rule;

[0013] A document conversion module, configured to determine a document conversion model according to the document type of the target PDF document, and perform document conversion processing on the target PDF document based on the determined document conversion model to obtain a first text corresponding to the target PDF document;

[0014] A text processing module, used for determining a header text and / or a footer text in the first text, and deleting the header text and / or the footer text to obtain a second text;

[0015] The information output module is used to determine the target text matching the text to be searched in the second text, and output the text information and position information of the target text.

[0016] In a third aspect, the present application also provides a computer device, comprising a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of the text search method as described above are implemented.

[0017] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, wherein when the computer program is executed by a processor, the steps of the text search method as described above are implemented.

[0018] The present application provides a text search method, apparatus, computer equipment and computer-readable storage medium. The present application determines the document type of a target PDF document, selects a suitable document conversion model according to the document type of the target PDF document, performs document conversion processing on the target PDF document, obtains a first text, determines header text and / or footer text in the first text, deletes the header text and / or footer text, obtains a second text, and then performs text search in the second text to complete the text search of the target PDF document. By selecting a document conversion model adapted to the document type of the PDF document for document conversion processing, the accuracy of the converted text is improved, and the influence of headers and footers is avoided, thereby improving the text search accuracy of the PDF document. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0020] Figure 1 A flowchart of a text search method provided in one embodiment of the present application;

[0021] Figure 2 A schematic diagram of a second text provided in an embodiment of the present application;

[0022] Figure 3 A schematic diagram of a second text provided for another embodiment of the present application;

[0023] Figure 4 A schematic diagram of a text search device provided in an embodiment of the present application;

[0024] Figure 5 The present invention is a block diagram showing the structure of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0026] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0027] The embodiments of the present application provide a text search method, apparatus, computer equipment, and computer-readable storage medium. The text search method can be applied to a computer device, which can be an electronic device such as a tablet computer, a laptop computer, or a desktop computer. It can also be applied to a server, which can be a separate server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0028] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0029] Please refer to Figure 1 , Figure 1 A flowchart of a text search method provided in an embodiment of the present application.

[0030] like Figure 1 As shown, the text search method includes steps S101 to S105.

[0031] Step S101: Obtain a target PDF document and a text search request for the target PDF document, wherein the document search request includes text to be searched.

[0032] Exemplarily, the target PDF document is used to indicate the PDF document that needs to be text-searched and located. It is understandable that the PDF document includes but is not limited to the PDF document used to record customer information and the PDF document used to record product information; wherein, the PDF document used to record customer information is, for example, the PDF document used to record customer information in a financial investment company, and these PDF documents record financial information including but not limited to customer investment, financial management and funds, as well as the customer's personal information; the PDF document used to record customer information is, for example, the PDF document used to record customer information in an insurance company, and these PDF documents record the policy information of the insurance policy purchased by the customer and the customer's personal information, etc.; and the PDF document used to record product information records the risk information, income situation and other information corresponding to the investment and financial management products; or records the basic information of insurance, such as insurance type, compensation conditions and compensation amount. It should be noted that if you need to find a specific piece of information, such as the investment amount of a certain financial product of a certain customer, you need to search in the corresponding PDF document. By performing text search on the PDF document, there is no need for manual search, thereby improving the efficiency of information query on the PDF document.

[0033] It should be understood that when recording the above information, more information needs to be recorded in the same document. Therefore, the PDF document that completes the information recording is generally multi-page.

[0034] Exemplarily, when a text search request for a target PDF document is obtained, a text search is performed on the obtained target PDF document to determine whether the text to be searched is in the target PDF document, and if the text to be searched is in the target PDF document, the position of the text to be searched in the target PDF document is determined, thereby completing the text search and positioning.

[0035] Step S102: Determine the document type of the target PDF document based on a document type identification rule.

[0036] Exemplarily, the document type of the target PDF document is determined by a preset document identification rule to determine a document conversion model suitable for the target PDF document.

[0037] It should be understood that since there are many ways to generate PDF documents, such as generating them by format conversion from other text documents, or generating them by scanning, so that the generated PDF documents have different information, therefore, if only one method such as a text segment search tool is used to search and locate the text of the PDF document, better search results can be obtained in the PDF document obtained by format conversion, but if the PDF document is obtained by scanning, the search results of the text segment search tool will be less accurate, and some text segments in the scanned PDF document may not be found. Based on this, by determining the document type of the target PDF document to determine an adapted document conversion model, so as to convert the target PDF document into text, the accuracy of text search for the target PDF document can be improved.

[0038] In some embodiments, determining the document type of the target PDF document based on a document type identification rule includes: traversing each page of the target PDF document to determine page text information of each page; and determining the document type of the target PDF document according to the page text information.

[0039] In a specific implementation, since the information that can be recorded in a PDF document of one page is very limited, the policy PDF documents in the insurance application scenario, or the financial PDF documents in the financial scenario, are all multi-page PDF documents. Based on this, the target PDF document is also a multi-page PDF document, but the present application does not limit the specific number of pages of the target PDF document. Specifically, each page in the target PDF document is traversed to obtain the page text information corresponding to each page, and then the document type of the target PDF document is determined according to the page text information corresponding to each page in the target PDF document. Among them, the page text information of the page is used to indicate whether there is a text segment in the page after the traversal of the target PDF document, and if the page text information of the page indicates that there is a text segment in the page, the page text information also includes the number of text segments. Among them, the text segment can be used to indicate each line of text on the page, that is, one line of text corresponds to one text segment; or the text segment is determined according to punctuation marks, for example, if there are 6 periods in the text of a page, the number of text segments of this page is determined to be 6. It should be understood that those skilled in the art can also determine text segments through other rules, and this application does not limit the specific method of determining text segments.

[0040] It should be noted that since the page text information is obtained by traversal, there may be a page that actually contains a text segment, but it may not be recognized during the traversal process that the page contains the text segment, resulting in the page information of the page indicating that it does not contain a text segment.

[0041] In some embodiments, determining the document type of the target PDF document according to the page text information includes: determining a page whose page text information does not include a text segment as a first page, and determining, among multiple pages whose page text information includes text segments, a page with the largest number of text segments as a second page; determining the number of text segments in the second page according to the page text information of the second page; if the number of the first pages is less than or equal to a preset page number threshold, and the number of text segments in the second page is greater than or equal to a preset text segment number threshold, determining the document type of the target PDF document as the first type; if the number of the first pages is greater than the preset page number threshold, or the number of text segments in the second page is less than the preset text segment number threshold, determining the document type of the target PDF document as the second type.

[0042] Exemplarily, after obtaining the traversal result, the page indicated by the page text information as not containing the text segment is determined as the first page, and the page indicated by the page text information as containing the text segment is determined as the second page to be selected, so as to determine the second page in the second page to be selected. Specifically, the second page to be selected containing the largest number of text segments is determined as the second page, and the number of text segments in the second page is determined.

[0043] Exemplarily, if the number of first pages is less than or equal to a preset page number threshold, and the number of text segments in the second page is greater than or equal to a preset text segment number threshold, the target PDF document is determined to be a PDF document of the first type.

[0044] On the contrary, if the number of the first pages is greater than the preset page number threshold, or the number of text segments in the second page is less than the preset text segment number threshold, the target PDF document is determined to be a second type document.

[0045] In a specific implementation process, the preset page number threshold is determined according to the total number of pages of the target PDF document, for example, 10% of the total number of pages; the preset text segment number threshold is 10. It should be understood that the above-mentioned preset page number threshold and preset text segment number threshold are exemplary descriptions, and the present application does not limit the specific values ​​of the page number threshold and the text segment number threshold.

[0046] Step S103: determining a document conversion model according to the document type of the target PDF document, and performing document conversion processing on the target PDF document based on the determined document conversion model to obtain a first text corresponding to the target PDF document.

[0047] Exemplarily, after determining the document type of the target PDF document, a corresponding document conversion model is determined according to the determined document type, so as to perform document conversion processing on the target PDF document through the corresponding document conversion model, thereby improving the conversion accuracy of the target PDF document.

[0048] In some embodiments, determining the document conversion model according to the document type of the target PDF document includes: when the document type of the target PDF document is the first type, determining that the document conversion model includes a character extraction model; when the document type of the target PDF document is the second type, determining that the document conversion model includes an optical character recognition model.

[0049] Exemplarily, if the target PDF document is a PDF document of the first type, a character extraction model is used to directly perform text extraction on the target PDF document; or a format conversion model is used to restore the document format of the target PDF document to obtain other document formats before conversion to PDF format, such as Word document format or text document format, and then text extraction is directly performed to obtain the first text.

[0050] If the target PDF document is a PDF document of the second type, the target PDF document is converted by an optical character recognition (OCR) model to obtain the first text. Specifically, the target PDF document is converted into an image, and optical character recognition is performed on the image to extract the first text.

[0051] By selecting different document conversion models for target PDF documents of different document types, the accuracy of the first text obtained by conversion can be effectively improved, thereby improving the text search precision and accuracy of the target PDF document.

[0052] Step S104: determine the header text and / or footer text in the first text, and delete the header text and / or footer text to obtain a second text.

[0053] Exemplarily, in the process of text conversion, the header and / or footer in the target PDF document will also be converted into the first text. Therefore, after obtaining the first text, the process of using the first text to perform text search and positioning will be affected by the header and / or footer, thereby affecting the final text search and positioning results. The header and / or footer generally do not carry important information. Therefore, by deleting the header text and / or footer text, the accuracy and efficiency of the text search in the first text can be improved.

[0054] In some embodiments, the determining of the header text and / or footer text in the first text and deleting the header text and / or footer text to obtain the second text includes: determining a plurality of candidate texts in the first text, each of the candidate texts corresponding to the first line of text on each page in the target PDF document; determining a text repetition rate of each of the candidate texts according to the plurality of candidate texts; determining the candidate text having a text repetition rate greater than or equal to a preset text repetition rate threshold as the header text; deleting the header text to obtain the second text; and / or performing text matching processing in the first text according to a preset footer text library based on a regular expression; determining the text matching the preset footer text in the preset footer text library as the footer text; deleting the footer text to obtain the second text.

[0055] Exemplarily, in the first text, the selected text corresponding to the first line of each page of the target PDF document is determined, that is, the determined selected text is the first line of text located on a page of the target PDF document, therefore, the selected text is used to indicate the header corresponding to the page, and multiple selected texts are compared with each other to determine the text repetition rate of each selected text, and the selected text with a text repetition greater than or equal to the preset text repetition rate threshold is determined as the header text, and then the header text is deleted from the first text to obtain the second text. Specifically, the text repetition rate of a selected text is determined by determining the selected text that is the same as the selected text or the similarity is less than the preset threshold, and determining the number of these selected texts, dividing the sum of the current selected text and the number of the same or similar selected texts by the determined total number of selected texts, to obtain the text repetition rate of the current selected text. For example, the selected text includes text A, text B, text C and text D, and text A, text B, and text C are the same text, then the text repetition rates of text A, text B and text C are all 75%, and the text repetition rate of text D is 0. It should be noted that the above method of determining the text repetition rate is for illustrative purposes only, and this application does not limit the specific determination method.

[0056] Exemplarily, based on regular expressions, a preset footer text library is used to traverse the first text to perform text matching in the first text through preset footer text in the preset footer text library, and the text that successfully matches in the first text is determined as the footer text, and the footer text is deleted to obtain the second text.

[0057] In a specific implementation, the preset footer text includes but is not limited to page (\d) of (\d) pages, page (\d), \d, -\d-, etc., where \d is used to indicate an integer character. Regular Expression (RE) is a text processing tool that can help match, find and replace specific strings. In this embodiment, the footer text can be found and deleted.

[0058] It should be noted that the first text may include both a header and a footer, or may include one of the header and the footer. The above method can be used to determine the header text and / or the footer text in the first text, thereby deleting the header text and / or the footer text to obtain the second text. In the process of using the second text for text search and positioning, the influence of the header text and / or the footer text can be avoided, thereby improving the accuracy and efficiency of text search and positioning.

[0059] See also Figure 2 and Figure 3 , Figure 2 A schematic diagram of a second text provided in an embodiment of the present application, Figure 3 A schematic diagram of a second text provided for another embodiment of the present application.

[0060] In some embodiments, after deleting the header text and / or the footer text, the method further includes: performing text splicing processing on the text adjacent to the header text to obtain the second text; and / or performing text splicing processing on the text adjacent to the footer text to obtain the second text.

[0061] For example, Figure 2 As shown in FIG. 1 , after deleting the header text and / or the footer text, the text originally adjacent to the header text and / or the footer text in the first text has a gap between the two texts due to the deletion of the header text and / or the footer text, which causes errors in the text search and positioning process or reduces the efficiency of the search and positioning. By splicing the text adjacent to the header text and / or the text adjacent to the footer text, the second text after deleting the header text and / or the footer text will not have a large gap, as shown in FIG. Figure 3 As shown, the accuracy and efficiency of text search and positioning are improved. It should be understood that Figure 2 and Figure 3The horizontal line portion of some texts is used to represent characters or character strings, and this application does not limit the specific content of the second text.

[0062] Step S105: determine a target text matching the text to be searched in the second text, and output text information and position information of the target text.

[0063] Exemplarily, the second text is traversed to determine the target text matching the text to be searched in the second text, complete the text search, and output the position information corresponding to the target text to achieve text positioning. It should be understood that in the case where the target text spans across pages, the output position information includes all the pages where the target text is located. For example, if the target text spans from the second page to the third page, the position information of the target text includes the second page and the third page. It should be understood that the target text may also include more pages, and this application does not limit the number of pages spanned by the target text or the number of pages where the target text is located.

[0064] In some embodiments, determining a target text matching the text to be searched in the second text and outputting text information and position information of the target text includes: traversing the second text based on the text to be searched to obtain multiple matching texts, wherein the text length of the matching texts is the same as the text length of the text to be searched; determining the text similarity between each matching text and the text to be searched; and determining that the matching text whose text similarity is greater than or equal to a preset text similarity threshold is the target text.

[0065] In a specific implementation, a text sliding box is generated according to the text length of the text to be searched, so as to select text in the second text through the text sliding box, wherein the length of the text selected by the text sliding box is the same as the text length of the text to be searched, and the selection starts from the first character of the second text, that is, at the i-th character, the selection position of the text sliding box is i+L-1, and the text located in the text sliding box at this time is the matching text, wherein L is the text length of the text to be searched, and the current position of the text sliding box can be determined by the position of the character i, thereby determining the position information of the target text.

[0066] Exemplarily, after completing the selection of the second text and obtaining multiple matching texts, the text similarity between each matching text and the text to be searched is determined. For example, for the text abcde, starting from the character a, the length of the text sliding box is 2, the text to be searched is ab, and the matching texts obtained include ab, bc, cd, de, and in other embodiments, the matching texts also include ac, bd, ce; by calculating the text similarity between each of the above matching texts and the text to be searched ab, it is obtained that the text similarity between the matching text ab and the text to be searched ab is 100%, the text similarity between the matching texts bc and ac and the text to be searched ab is 50%, and the text similarity between the remaining matching texts and the text to be searched is 0%. If the text similarity threshold is 50%, the matching texts ab, bc, and ac are all target texts, and the text information and position information of the three target texts are output respectively for the user to view and determine; if the text similarity threshold is 80%, only the matching text ab is the target text, and the text information and position information of this target text are output. It should be understood that the length of the above-mentioned text sliding box and the text similarity threshold can be set according to actual conditions and are not limited in this application.

[0067] By using the above method of determining the target text, even if the user misses or incorrectly enters a character when entering the text to be searched, the similar target text can still be searched and located in the second text, and output for the user to review and confirm, thereby avoiding the situation where the text segment search tool cannot find the text when the user misses or enters a character incorrectly, and improving the fault tolerance of text search and positioning.

[0068] The text search method provided in the above embodiment improves the accuracy of the converted text by selecting a document conversion model that is adapted to the document type of the PDF document for document conversion processing, and avoids the influence of headers and footers in the text, thereby improving the text search accuracy of the PDF document, and searches through the confidence of the text during the search process, thereby improving the fault tolerance rate of text searches on PDF documents, thereby improving the user experience.

[0069] See also Figure 4 , Figure 4 is a schematic diagram of a text search device provided in an embodiment of the present application. The text search device can be configured in a server or a terminal to execute the aforementioned text search method.

[0070] like Figure 4 As shown, the text search device 100 includes: an information acquisition module 110 , a type determination module 120 , a document conversion module 130 , a text processing module 140 , and an information output module 150 .

[0071] The information acquisition module 110 is used to acquire a target PDF document and a text search request of the target PDF document, wherein the document search request includes the text to be searched.

[0072] The type determination module 120 is used to determine the document type of the target PDF document based on a document type identification rule.

[0073] The document conversion module 130 is used to determine a document conversion model according to the document type of the target PDF document, and perform document conversion processing on the target PDF document based on the determined document conversion model to obtain a first text corresponding to the target PDF document.

[0074] The text processing module 140 is used to determine the header text and / or footer text in the first text, and delete the header text and / or footer text to obtain the second text.

[0075] The information output module 150 is used to determine the target text matching the to-be-searched text in the second text, and output the text information and position information of the target text.

[0076] Exemplarily, the text processing module 140 includes a first text determination submodule, a text repetition rate calculation submodule, a second text determination submodule, a text deletion submodule, and a text matching submodule.

[0077] The first text determination submodule is used to determine a plurality of texts to be selected in the first text, each of the texts to be selected corresponds to the first line of text on each page in the target PDF document.

[0078] The text repetition rate calculation submodule is used to determine the text repetition rate of each of the candidate texts according to the plurality of candidate texts.

[0079] The text matching submodule is used to perform text matching processing in the first text based on a regular expression and according to a preset footer text library.

[0080] The second text determination submodule is used to determine the candidate text whose text repetition rate is greater than or equal to a preset text repetition rate threshold as the header text, and / or determine the text matching the preset footer text in the preset footer text library as the footer text.

[0081] The text deletion submodule is used to delete the footer text to obtain the second text.

[0082] Exemplarily, the text search device 100 further includes a text splicing module.

[0083] The text splicing module is used to perform text splicing processing on the text adjacent to the header text to obtain the second text, and / or perform text splicing processing on the text adjacent to the footer text to obtain the second text.

[0084] Exemplarily, the type determination module 120 includes a page information determination submodule.

[0085] The page information determination submodule is used to traverse each page of the target PDF document to determine the page text information of each page.

[0086] The type determination module 120 is further configured to determine the document type of the target PDF document according to the page text information.

[0087] Exemplarily, the type determination module 120 further includes a page determination submodule and a text segment determination submodule.

[0088] The page determination submodule is used to determine a page whose page text information does not contain text segments as the first page, and to determine a page with the largest number of text segments as the second page among multiple pages whose page text information contains text segments.

[0089] The text segment determination submodule is used to determine the number of text segments in the second page according to the page text information of the second page.

[0090] The type determination module 120 is further configured to determine the document type of the target PDF document as the first type if the number of the first pages is less than or equal to a preset page number threshold, and the number of text segments in the second pages is greater than or equal to a preset text segment number threshold.

[0091] If the number of the first pages is greater than the preset page number threshold, or the number of text segments in the second page is less than the preset text segment number threshold, the document type of the target PDF document is determined to be the second type.

[0092] Exemplarily, the document conversion module 130 is further configured to, when the document type of the target PDF document is the first type, determine that the document conversion model includes a character extraction model.

[0093] When the document type of the target PDF document is the second type, it is determined that the document conversion model includes an optical character recognition model.

[0094] Exemplarily, the information output module 150 includes a text traversal submodule, a text similarity calculation submodule, and a target text determination submodule.

[0095] The text traversal submodule is used to traverse the second text based on the text to be searched to obtain multiple matching texts, wherein the text length of the matching text is the same as the text length of the text to be searched.

[0096] The text similarity calculation submodule is used to determine the text similarity between each matching text and the text to be searched.

[0097] The target text determination submodule is used to determine that the matching text whose text similarity is greater than or equal to a preset text similarity threshold is the target text.

[0098] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0099] See also Figure 5 , Figure 5 The present invention is a block diagram of a computer device according to an embodiment of the present invention. The computer device may be a server or a terminal.

[0100] like Figure 5 As shown, the computer device includes a processor, a memory, and a network interface connected via a system bus, wherein the memory may include a storage medium and an internal memory.

[0101] The storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any text search method.

[0102] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0103] The internal memory provides an environment for the operation of the computer program in the storage medium. When the computer program is executed by the processor, the processor can execute any text search method.

[0104] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0105] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0106] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:

[0107] Obtaining a target PDF document and a text search request for the target PDF document, wherein the document search request includes text to be searched;

[0108] Determine the document type of the target PDF document based on a document type identification rule;

[0109] Determining a document conversion model according to the document type of the target PDF document, and performing document conversion processing on the target PDF document based on the determined document conversion model to obtain a first text corresponding to the target PDF document;

[0110] Determine the header text and / or footer text in the first text, and delete the header text and / or footer text to obtain a second text;

[0111] A target text matching the text to be searched is determined in the second text, and text information and position information of the target text are output.

[0112] In one embodiment, when the processor determines the header text and / or footer text in the first text and deletes the header text and / or footer text to obtain the second text, it is configured to implement:

[0113] Determine a plurality of texts to be selected in the first text, each of the texts to be selected corresponds to the first line of text on each page in the target PDF document;

[0114] Determine the text repetition rate of each of the to-be-selected texts according to the plurality of to-be-selected texts;

[0115] Determine the candidate text whose text repetition rate is greater than or equal to a preset text repetition rate threshold as the header text;

[0116] Deleting the header text to obtain the second text; and / or

[0117] Based on regular expressions, text matching processing is performed in the first text according to a preset footer text library;

[0118] Determine the text that matches the preset footer text in the preset footer text library as the footer text;

[0119] The footer text is deleted to obtain the second text.

[0120] In one embodiment, when implementing the text search method, the processor is further used to implement:

[0121] Performing text splicing processing on the text adjacent to the header text to obtain the second text; and / or

[0122] The text adjacent to the footer text is concatenated to obtain the second text.

[0123] In one embodiment, when the processor determines the document type of the target PDF document based on the document type identification rule, it is used to implement:

[0124] Traversing each page of the target PDF document to determine page text information of each page;

[0125] The document type of the target PDF document is determined according to the page text information.

[0126] In one embodiment, when the processor determines the document type of the target PDF document according to the page text information, it is used to implement:

[0127] The page whose page text information does not contain text segments is determined as the first page, and among the multiple pages whose page text information contains text segments, the page with the largest number of text segments is determined as the second page;

[0128] Determining the number of text segments in the second page according to the page text information of the second page;

[0129] If the number of the first pages is less than or equal to a preset page number threshold, and the number of text segments in the second page is greater than or equal to a preset text segment number threshold, determining the document type of the target PDF document as a first type;

[0130] If the number of the first pages is greater than the preset page number threshold, or the number of text segments in the second page is less than the preset text segment number threshold, the document type of the target PDF document is determined to be the second type.

[0131] In one embodiment, when determining the document conversion model according to the document type of the target PDF document, the processor is used to implement:

[0132] When the document type of the target PDF document is the first type, determining that the document conversion model includes a character extraction model;

[0133] When the document type of the target PDF document is the second type, it is determined that the document conversion model includes an optical character recognition model.

[0134] In one embodiment, when the processor determines the target text matching the to-be-searched text in the second text and outputs the text information and position information of the target text, it is configured to implement:

[0135] Traversing the second text based on the text to be searched to obtain a plurality of matching texts, wherein the text length of the matching texts is the same as the text length of the text to be searched;

[0136] Determine the text similarity between each of the matching texts and the text to be searched;

[0137] The matching text whose text similarity is greater than or equal to a preset text similarity threshold is determined as the target text.

[0138] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. The computer program includes program instructions. The method implemented when the program instructions are executed can refer to the various embodiments of the text search method of the present application.

[0139] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc., equipped on the computer device.

[0140] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0141] It should also be understood that the term "and / or" used in the specification of this application and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system including the element.

[0142] The serial numbers of the embodiments of the present application are for description only and do not represent the advantages and disadvantages of the embodiments. The above description is only a specific implementation mode of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A text search method, characterized in that: include: Obtaining a target PDF document and a text search request for the target PDF document, wherein the document search request includes text to be searched; Determine the document type of the target PDF document based on a document type identification rule; Determining a document conversion model according to the document type of the target PDF document, and performing document conversion processing on the target PDF document based on the determined document conversion model to obtain a first text corresponding to the target PDF document; Determine the header text and / or footer text in the first text, and delete the header text and / or footer text to obtain a second text; A target text matching the text to be searched is determined in the second text, and text information and position information of the target text are output.

2. The text search method according to claim 1, characterized in that: The step of determining the header text and / or footer text in the first text and deleting the header text and / or footer text to obtain the second text includes: Determine a plurality of texts to be selected in the first text, each of the texts to be selected corresponds to the first line of text on each page in the target PDF document; Determine the text repetition rate of each of the to-be-selected texts according to the plurality of to-be-selected texts; Determine the candidate text whose text repetition rate is greater than or equal to a preset text repetition rate threshold as the header text; Deleting the header text to obtain the second text; and / or Based on regular expressions, text matching processing is performed in the first text according to a preset footer text library; Determine the text that matches the preset footer text in the preset footer text library as the footer text; The footer text is deleted to obtain the second text.

3. The text search method according to claim 2, characterized in that: After deleting the header text and / or the footer text, the method further includes: Performing text splicing processing on the text adjacent to the header text to obtain the second text; and / or The text adjacent to the footer text is concatenated to obtain the second text.

4. The text search method according to any one of claims 1 to 3, characterized in that: The step of determining the document type of the target PDF document based on a document type identification rule includes: Traversing each page of the target PDF document to determine page text information of each page; The document type of the target PDF document is determined according to the page text information.

5. The text search method according to claim 4, characterized in that: The step of determining the document type of the target PDF document according to the page text information includes: Determine a page whose page text information does not include a text segment as a first page, and determine, among a plurality of pages whose page text information includes a text segment, a page with the largest number of text segments as a second page; Determining the number of text segments in the second page according to the page text information of the second page; If the number of the first pages is less than or equal to a preset page number threshold, and the number of text segments in the second page is greater than or equal to a preset text segment number threshold, determining the document type of the target PDF document as a first type; If the number of the first pages is greater than the preset page number threshold, or the number of text segments in the second page is less than the preset text segment number threshold, the document type of the target PDF document is determined to be a second type.

6. The text search method according to claim 5, characterized in that: The step of determining a document conversion model according to the document type of the target PDF document includes: When the document type of the target PDF document is the first type, determining that the document conversion model includes a character extraction model; When the document type of the target PDF document is the second type, it is determined that the document conversion model includes an optical character recognition model.

7. The text search method according to any one of claims 1 to 3, characterized in that: The step of determining a target text matching the text to be searched in the second text and outputting text information and position information of the target text includes: Traversing the second text based on the text to be searched to obtain a plurality of matching texts, wherein the text length of the matching texts is the same as the text length of the text to be searched; Determine the text similarity between each of the matching texts and the text to be searched; The matching text whose text similarity is greater than or equal to a preset text similarity threshold is determined as the target text.

8. A text search device, characterized in that: The text search device comprises: An information acquisition module, used to acquire a target PDF document and a text search request of the target PDF document, wherein the document search request includes a text to be searched; A type determination module, used to determine the document type of the target PDF document based on a document type identification rule; A document conversion module, configured to determine a document conversion model according to the document type of the target PDF document, and perform document conversion processing on the target PDF document based on the determined document conversion model to obtain a first text corresponding to the target PDF document; A text processing module, used for determining a header text and / or a footer text in the first text, and deleting the header text and / or the footer text to obtain a second text; The information output module is used to determine the target text matching the text to be searched in the second text, and output the text information and position information of the target text.

9. A computer device, characterized in that: The computer device comprises a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of the text search method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the text search method according to any one of claims 1 to 7 are implemented.

Citation Information

Cited By

  • Multi-agent collaborative file large model fine tuning data set construction method and system

    CN121597646A