Methods, media, and products for searching technical documents

Abstract

This application discloses a method, medium, and product for searching technical documents. The method includes: acquiring the original technical document; performing structured parsing on the original technical document to identify multiple semantic blocks and their hierarchical relationships; constructing a document structure tree based on the multiple semantic blocks and hierarchical relationships, wherein each node in the document structure tree corresponds to a semantic block, and each non-root node is associated with path information of all its ancestor nodes; constructing a multi-level composite index based on the nodes in the document structure tree, including a first type index and a second type index; responding to a received query statement, performing searches in the first and second type indexes respectively to obtain multiple candidate result nodes; and reordering the multiple candidate result nodes based on their positions in the document structure tree to generate search results.

Description

Technical Field

[0001] This disclosure relates to the field of information retrieval technology, and more specifically, to a method for searching technical documents, a computer-readable storage medium, and a computer program product. Background Technology

[0002] In engineering, R&D, and operations and maintenance fields, technical personnel routinely consult a large number of technical documents, such as product manuals, design specifications, troubleshooting guides, and application programming interface (API) documents. These documents are characterized by their length, high level of specialization, and large amounts of structured data. How to achieve efficient retrieval of these technical documents is a critical problem that urgently needs to be solved in the field of technical document management and retrieval. Summary of the Invention

[0003] In one aspect, this application discloses a method for searching technical documents, comprising: obtaining an original technical document; performing structured parsing on the original technical document to identify multiple semantic blocks contained in the original technical document and the hierarchical relationship between the multiple semantic blocks, wherein the multiple semantic blocks include title blocks and content blocks, the title blocks include hierarchical relationships, and the content blocks include at least one of descriptive content blocks and structured data blocks; constructing a document structure tree based on the multiple semantic blocks and hierarchical relationships, wherein each node in the document structure tree corresponds to a semantic block, and each non-root node is associated with path information of all its ancestor nodes; constructing a multi-level composite index based on the nodes in the document structure tree, the multi-level composite index including: a first type index for supporting retrieval based on semantic similarity; and a second type index for supporting retrieval based on structured conditions; responding to a received query statement, performing retrieval in the first type index and the second type index respectively to obtain multiple candidate result nodes; and reordering the multiple candidate result nodes based on their positions in the document structure tree and generating search results.

[0004] In other respects, this application discloses a computer-readable storage medium storing instructions and a computer program product including instructions. These instructions, when executed by one or more processors, cause the processors to perform the methods described in this application. Attached Figure Description

[0005] When read in conjunction with the accompanying drawings, various aspects of this disclosure are best understood through the following detailed description. It should be noted that, in accordance with standard practice in the art, the features are not drawn to scale. In fact, for clarity of discussion, the dimensions of the features may be arbitrarily increased or decreased.

[0006] Figure 1 This is an overall flowchart of a method for searching technical documents according to an embodiment of this application.

[0007] Figure 2 The illustration shows a flowchart of document structured parsing and tree-like representation according to an embodiment of this application.

[0008] Figure 3 The illustration shows a flowchart of hybrid retrieval and reordering according to one embodiment of this application.

[0009] Figure 4 This is a schematic diagram of a computing device that can be implemented according to an embodiment of the present invention. Detailed Implementation

[0010] The following disclosure provides numerous different embodiments or examples for implementing various features of the provided subject matter. Specific examples of components and arrangements are described below to simplify this disclosure. Of course, these are merely examples and not limiting.

[0011] Furthermore, the various operations will be described as multiple discrete operations in a manner most conducive to understanding the illustrative embodiments; however, the order of description should not be construed as implying that these operations must depend on the order. In particular, these operations do not need to be performed in the order presented.

[0012] In fields such as engineering, research and development, and operations and maintenance, technical personnel need to consult a large number of technical documents on a daily basis, such as product manuals, design specifications, troubleshooting guides, and application programming interface (API) documents. These technical documents are characterized by their length, high level of specialization, and large amount of structured data.

[0013] In the field of technical document management and retrieval, traditional search methods mainly rely on full-text indexes or simple metadata searches. These search methods have the following significant drawbacks when dealing with long, highly specialized technical documents.

[0014] First, in terms of semantic granularity, traditional methods typically treat the entire document as a single text stream, ignoring the document's internal logical structure, such as chapters, paragraphs, tables, and parameter lists. When a user searches for a specific parameter or a more specific function, the system often returns the entire long document, causing the user to spend a significant amount of time on a secondary search.

[0015] Secondly, in terms of multimodal data processing, technical documents contain a large amount of structured data (such as technical parameter tables and specification lists) and unstructured data (such as functional descriptions and operation guides). Existing search methods struggle to simultaneously achieve semantic understanding and accurate value range retrieval, often resulting in the ability to find keywords but not precise parameters. For example, when a user searches for "components with operating voltage greater than 24V," a keyword-based search might return all paragraphs containing the word "voltage," without the ability to filter by numerical range.

[0016] Furthermore, in terms of search intent matching, simple keyword inverted indexes cannot understand contextual semantics, while pure vector search, although capable of handling semantic similarity, often produces spurious relevance results when dealing with specific model numbers, code snippets, or technical terms due to dimensionality compression. For example, when a user searches for the specific error code "ERR_CODE_0x7F", vector search may return semantically similar but different code results, while keyword search may fail to achieve an exact match due to word segmentation issues involving underscores and hexadecimal symbols.

[0017] In summary, existing search methods generally suffer from technical challenges when dealing with technical document retrieval, namely the disconnect between semantic understanding and exact matching, and the separation between document structure and content retrieval. They cannot simultaneously meet users' demands for high recall, high accuracy, and high readability of search results.

[0018] This application provides a technical document search method based on structured document parsing and multi-level indexing to achieve refined, semantic, and structured retrieval of technical documents, solving the problems of low recall, poor accuracy, and loss of structured information in existing technologies.

[0019] Figure 1 This is a schematic flowchart of a method 100 for searching technical documents according to an embodiment of this application. Method 100 can be performed by a system or device for searching technical documents (e.g., described below). Figure 1 In the illustrated embodiment, method 100 may include operations 110 to 160 for performing technical document search based on structured document parsing and multi-level indexing. In other embodiments, the method for technical document search may include more or fewer or different operations, and this disclosure is not limited in this respect.

[0020] At operation 110, the system retrieves the original technical documents. According to embodiments of this application, the original technical documents can be technical materials in various formats, including but not limited to product manuals in PDF format, design specifications in Word format, online help documents in HTML format, technical specifications in XML format, and API documents in Markdown format. These technical documents are typically characterized by their length, high level of specialization, and the presence of a large amount of structured and semi-structured content.

[0021] At operation 120, the system performs structured parsing on the original technical document to identify multiple semantic blocks contained in the original technical document and the hierarchical relationships between the multiple semantic blocks.

[0022] According to embodiments of this application, the system can intelligently analyze documents using a pre-trained document structure analysis model. This model can be fine-tuned based on, for example, a visual-language pre-trained model such as LayoutLMv3 or ERNIE-Layout, capable of simultaneously processing the document's text content and visual layout information. The document structure analysis model can take document page images and text as input and intelligently identify multiple semantic blocks contained within the document.

[0023] According to embodiments of this application, multiple semantic blocks include title blocks and content blocks. Title blocks define the skeletal structure of the document. The system identifies title blocks and determines their hierarchical level, such as H1, H2, H3, H4, etc. These hierarchical relationships constitute the logical structure of the document. Content blocks carry the specific content of the document, including at least one of descriptive content blocks and structured data blocks. For example, descriptive content blocks can be continuous descriptive text paragraphs, bulleted lists, numbered lists, etc.; structured data blocks can be parameter tables, technical specification tables, configuration parameter lists, etc. Furthermore, in some embodiments, content blocks may also include code blocks or formula blocks to carry content in special formats such as program code or mathematical formulas. For table blocks, the system can further perform table structure recognition, reconstruct the row, column, and header information of the table, and handle complex table structures such as those spanning multiple rows and columns, laying the foundation for subsequent construction of parameter field indexes.

[0024] After identifying all semantic blocks, the system can determine the hierarchical relationships between them based on the hierarchical relationships defined by the heading blocks. For example, an H2-level heading block belongs to the nearest preceding H1-level heading block; a paragraph or table block belongs to the nearest preceding heading block at a higher level. In this way, the system can establish a complete hierarchical relationship between all semantic blocks.

[0025] Through the above parsing process, the system can transform the originally flat and unstructured raw technical documents into structured data with rich semantic annotations and clear hierarchical relationships, laying the foundation for subsequent document structure tree construction and multi-level composite indexes.

[0026] At operation 130, a document structure tree is constructed based on multiple semantic blocks and hierarchical relationships. Each node in the document structure tree corresponds to a semantic block, and each non-root node is associated with the path information of all its ancestor nodes.

[0027] Specifically, the system first determines the document's skeletal structure based on the hierarchical relationships within the heading blocks, using the top-level heading block as the root node or first-level node, and each level of sub-heading blocks as child nodes of their corresponding parent heading block, thus establishing a complete heading hierarchy tree. Subsequently, the system can attach each content block to its corresponding heading node according to its hierarchical relationship, making each paragraph block, table block, etc., a child node of its respective heading block. Through this process, all semantic blocks are organized into a complete document structure tree, where heading blocks typically constitute internal nodes and content blocks typically constitute leaf nodes. After the document structure tree is constructed, the system can associate the path information of all its ancestor nodes with each non-root node; that is, the content of all heading blocks traversed from the root node to the current node is concatenated to form the semantic path of that node. This path information is equivalent to the node's "logical address" in the document, providing a foundation for subsequent index construction and contextual retrieval.

[0028] The following will combine Figure 2 The flowchart shown further illustrates an example process of document structured parsing and tree-like representation according to an embodiment of this application.

[0029] like Figure 2 As shown, the document structure parsing process is a multi-stage process, starting from obtaining the original technical document and continuing until the complete semantic blocks and their hierarchical relationships are identified.

[0030] First, the system retrieves the original technical documents. These documents can be in various formats, including but not limited to product manuals in PDF format, design specifications in Word format, online help documents in HTML format, technical specifications in XML format, and API documents in Markdown format. These technical documents are typically lengthy, highly specialized, and contain a large amount of structured and semi-structured data.

[0031] As shown in box 201, the system first parses the original technical document's format. For different document formats, the system can employ corresponding parsing strategies. For example, for PDF documents, the system can extract layout information such as text position, font size, and table areas; for HTML documents, the system can parse the document's tag structure, such as heading tags, paragraph tags, and table tags; for Word documents, the system can identify style information such as heading styles and list styles. The purpose of this process is to uniformly convert documents of different formats into an internal representation that can be used for further analysis.

[0032] As shown in box 202, the system can intelligently analyze documents using a pre-trained document structure analysis model. This model can be fine-tuned based on vision-language pre-trained models such as LayoutLMv3 and ERNIE-Layout, and can simultaneously process the text content and visual layout information of the document. The document structure analysis model takes document page images and text as input and intelligently identifies multiple semantic blocks contained in the document.

[0033] like Figure 2 As shown in the three parallel boxes 203, 204, and 205, the document structure analysis model identifies different types of semantic blocks in parallel. During title block recognition, the model identifies the title text in the document and determines its hierarchical level, such as H1, H2, H3, H4, etc. The hierarchical relationship between these title blocks constitutes the logical structure of the document. During paragraph block recognition, the model identifies continuous descriptive text, including ordinary paragraphs, bulleted lists, numbered lists, etc. During table block recognition, the model identifies table areas and further performs table structure recognition, reconstructing the row, column, and header information of the table, and handling complex table structures such as those spanning multiple rows and columns, laying the foundation for subsequent construction of parameter field indexes. Furthermore, in some implementations, the model can also identify code blocks or formula blocks, used to carry content in special formats such as program code or mathematical formulas.

[0034] As shown in box 206, after identifying all semantic blocks, the system can determine the hierarchical relationships between the semantic blocks based on the hierarchical relationships defined by the heading blocks. For example, an H2 level heading block belongs to the nearest preceding H1 level heading block; a paragraph block or table block belongs to the nearest preceding heading block at a higher level. In this way, the system can establish a complete hierarchical relationship between all semantic blocks.

[0035] Next, as shown in box 207, the document structure tree construction process is executed. It begins with the heading skeleton. The system can determine the document's skeletal structure based on the heading blocks and their hierarchical relationships identified in operation 120. Specifically, the system identifies the top-level heading blocks in the document, such as H1-level headings, and uses them as the root node or first-level child node of the document structure tree. Then, for each H1 heading block, the system finds all H2 heading blocks belonging to that heading block and uses these H2 heading blocks as child nodes of the H1 heading block. This process continues, with the system processing lower-level heading blocks such as H3 and H4 layer by layer, establishing a complete hierarchical tree structure of headings.

[0036] After the title skeleton is constructed, the system attaches each content block to its corresponding title node. Based on the hierarchical relationship determined in operation 120, each content block belongs to the nearest preceding title block at a higher level. For example, for a paragraph block describing the technical specifications of a hydraulic system, if it is located under the H2 title "3.1 Technical Specifications" and that H2 title belongs to the H1 title "Chapter 3 Hydraulic Systems," then this paragraph block will become a child node of the H2 title node "3.1 Technical Specifications." Similarly, for a table block containing pressure parameters, if it is also located under the H2 title "3.1 Technical Specifications," then this table block will also become a child node of the H2 title node.

[0037] like Figure 2 As shown in the two branch boxes 208 and 209, by constructing the internal nodes and leaf nodes of the document structure tree, the system organizes all semantic blocks into a complete document structure tree. In this tree, each node corresponds to a semantic block. Heading blocks typically constitute the internal nodes of the document structure tree; they can have child nodes and are used to carry content blocks belonging to that heading or lower-level heading blocks. Content blocks (such as paragraph blocks, table blocks, list blocks, code blocks, etc.) typically constitute the leaf nodes of the document structure tree; they have no child nodes and directly carry the specific content of the document.

[0038] As shown in box 210, after the document structure tree is constructed, the system associates the path information of all its ancestor nodes with each non-root node. Specifically, for each non-root node in the tree, the system traces upwards from its parent node, the parent node's parent node, and so on, until the root node, concatenating the title content of all ancestor nodes along the way to form the semantic path of that node. For example, for a parameter table block describing "maximum pressure," its ancestor nodes may include H2 title "3.1 Technical Specifications" and H1 title "Chapter 3 Hydraulic System." Therefore, the semantic path of this table block is the concatenation of "Chapter 3 Hydraulic System" -> "3.1 Technical Specifications." This path information constitutes the complete semantic context of the node, equivalent to the node's "logical address" in the document.

[0039] According to embodiments of this application, the path information can be associated by storing it as metadata in the node's data structure, or by dynamically obtaining it through pointers or references. Regardless of the implementation method, the end result is that the system can obtain the complete context information of each non-root node at any time.

[0040] By constructing the document structure tree and associating path information, the originally flat and isolated document content can be organized into a knowledge network with rich hierarchical relationships and contextual information. This lays the foundation for subsequent multi-level composite index construction and context-based reordering. For example, when constructing a semantic vector index, the system can concatenate the text content of a node with its semantic path information and encode them together. This results in a vector that includes not only the semantic content of the node itself but also its positional semantics within the document, enabling a more accurate understanding of the node's role and meaning within the overall document. During the retrieval phase, when a leaf node is hit, the system can quickly locate its parent and sibling nodes using its associated path information, thereby assembling a knowledge block with complete context to present to the user.

[0041] Continue to refer to Figure 1 In operation 140, the system constructs a multi-level composite index based on nodes in the document structure tree. According to embodiments of this application, the index constructed by the system based on nodes in the document structure tree is not a single index, but rather consists of multiple complementary index layers, aiming to cover the retrieval needs of technical documents from different dimensions.

[0042] For example, for descriptive content blocks in the document structure tree, including paragraph blocks, list blocks, and heading blocks themselves, the system can construct a first-type index, namely, a semantic vector index. Descriptive content blocks contain rich semantic information and are the primary targets for users' exploratory searches. During index construction, the system can utilize optimized dense vector encoding models, such as BGE-large-EN or GTE models, to encode the text content of each semantic block into a fixed-length feature vector. In some implementations, the system can also concatenate the semantic path information of the corresponding node in the document structure tree with the text content and encode it together, so that the generated vector not only contains the semantic content of the node itself but also its positional semantics within the document, thereby more accurately distinguishing similar content appearing in different chapters. After encoding, the system can store these feature vectors in a vector database, forming a semantic vector index. This index can achieve semantic similarity-based retrieval by calculating the cosine similarity between the query vector and the document vector, understanding synonyms and contextual semantics, and solving the retrieval problem of "similar meanings but different vocabulary".

[0043] For structured data blocks in the document structure tree, mainly table blocks and parameter list blocks, the system can construct a second type of index, namely, a parameter field index. Structured data blocks contain the most valuable structured information in technical documents, such as product specifications, performance parameters, and configuration options. For example, during index construction, the system can first perform deep parsing of table blocks to restore the row, column, and header information of the table, handling complex table structures such as those spanning multiple rows and columns. Next, the system can convert the table content into structured key-value pair data. For example, a table containing four columns—"parameter name," "minimum value," "maximum value," and "unit"—can be converted into a set of tuple records containing parameter name, minimum value, maximum value, and unit. Then, the system can utilize the dynamic field mapping function of a search engine to construct a field index for these key-value pairs, similar to that of a relational database. According to embodiments of this application, this index can support range queries for numeric fields, such as "finding components with an operating voltage greater than 24V," and also support exact matching queries for enumerated fields, such as "finding components with an IP67 protection rating." In this way, the parameter field index can solve the problem that traditional search cannot accurately handle technical parameters.

[0044] According to some embodiments of this application, the multi-level composite index may also include a third type of index, namely, a keyword inverted index. This index can be constructed for all semantic blocks in the document structure tree, introducing domain-specific dictionaries and custom rules on top of traditional word segmentation. For example, for technical terms such as product models (e.g., "A320-200") and error codes (e.g., "ERR_CODE_0x7F"), not only is regular word segmentation performed, but they are also indexed as complete phrases, while retaining the ability to index special characters and formats in code blocks and formula blocks. This index is used to support precise keyword matching retrieval, ensuring accurate hits on technical terms.

[0045] Through the above process, the system can construct a multi-level composite index covering semantics, keywords, and structured conditions. These three levels of index are independent yet complementary, together forming a comprehensive description of the technical document content, laying the foundation for subsequent hybrid retrieval and re-ranking.

[0046] At operation 150, in response to the received query, the system searches the first type of index and the second type of index respectively, obtaining multiple candidate result nodes. In embodiments where the multi-level composite index includes more levels of indexes, the system can search in each level of index separately to obtain the corresponding candidate result nodes. For example, in some embodiments, the multi-level composite index also includes a third type of index, which includes a keyword inverted index to support exact keyword matching retrieval. In these embodiments, in response to the received query, the system searches the first type of index, the second type of index, and the third type of index respectively, obtaining the corresponding multiple candidate result nodes.

[0047] At operation 160, the system reorders multiple candidate result nodes based on their positions in the document structure tree and generates search results.

[0048] According to embodiments of this application, reordering multiple candidate result nodes may include: calculating the comprehensive relevance score of each candidate result node to the query statement; for each candidate result node, obtaining its associated nodes in the document structure tree, wherein the associated nodes include one or more of the following: the parent node, child node, or sibling node of the candidate result node; assembling the candidate result nodes and their associated nodes into a knowledge block; calculating the comprehensive relevance score of the knowledge block based on the comprehensive relevance scores of each candidate result node associated with the knowledge block; and sorting the knowledge blocks according to their comprehensive relevance scores.

[0049] According to some embodiments of this application, calculating the comprehensive relevance score of each candidate result node to the query statement may include: normalizing the relevance scores of the candidate result node from different indexes to obtain multiple normalized relevance scores; and weighting and merging the multiple normalized relevance scores according to preset weights to obtain the comprehensive relevance score of the candidate result node.

[0050] The following will combine Figure 3 The flowchart shown describes in detail an example process of hybrid retrieval and reordering according to an embodiment of this application.

[0051] like Figure 3 As shown, the hybrid retrieval and re-ranking process according to embodiments of this application may include multiple stages such as query intent parsing, score normalization, weighted merging, structural context association, and comprehensive ranking and result generation.

[0052] First, as shown in box 301, when the system receives a query from a user, it will perform query intent parsing. The system can analyze the structure of the query, identifying its semantic components, precise terms, and structured conditions. For example, for the query "Find components in the hydraulic system of an A320 aircraft with pressure greater than 3000 psi", the system can identify the semantic component as "hydraulic system components", the precise term as "A320", and the structured condition as "pressure greater than 3000 psi". This parsing result can provide a basis for subsequent multi-path recall and weight configuration.

[0053] Next, after the three-layer retrieval operations shown in boxes 302 to 304, at box 305, the system collects candidate result nodes from different indexes. Each candidate result node is accompanied by its own relevance score and index source, such as a cosine similarity score from the semantic vector index, a BM25 score from the keyword inverted index, and a structured matching score from the parameter field index. These scores constitute the original data for re-ranking.

[0054] The system then enters a reordering process, which may include operations as shown in boxes 306 to 309.

[0055] At box 306, the system normalizes the relevance scores from different indices for each candidate result node, mapping them to the same scale. Since the scores from different indices have different properties and dimensions—for example, the cosine similarity of semantic vector indices is typically between 0 and 1, the BM25 score of keyword inverted indices can range from 0 to over 10 with no fixed upper limit, and the structured matching results of parameter field indices are 0 or 1—these scores cannot be directly compared and merged. Therefore, the system can use methods such as Min-Max normalization, Z-Score normalization, or ranking normalization to uniformly transform various scores into the range of 0 to 1, making them comparable. For example, for the BM25 score, the system can find the maximum and minimum values ​​of this score in the current result set, converting each BM25 score into (original score - minimum value) / (maximum value - minimum value); for the semantic similarity score, since it is already within the range of 0 to 1, it can be used directly or slightly adjusted.

[0056] At box 307, the system performs a weighted merging of the normalized scores based on preset weights, calculating the comprehensive relevance score for each candidate result node. For example, the weighted merging formula can be expressed as: Final Score = w1 × Semantic Normalization Score + w2 × Keyword Normalization Score + w3 × Structured Matching Normalization Score, where w1, w2, and w3 are the weights of each item, and their sum is 1. The weight settings can be dynamically adjusted according to business needs to adapt to different search scenarios. For example, in a general exploratory search scenario, a higher semantic weight can be set, such as w1=0.6, w2=0.2, w3=0.2, to prioritize semantic relevance; in a precise terminology search scenario, a higher keyword weight can be set, such as w1=0.2, w2=0.7, w3=0.1, to ensure accurate matching of specialized terms; and in a parameter specification search scenario, a higher structured weight can be set, such as w1=0.2, w2=0.2, w3=0.6, to meet the filtering requirements of structured conditions. Through this weighted merging, the system can comprehensively consider the correlations across different dimensions.

[0057] At box 308, the system uses the document structure tree to perform contextual association on candidate result nodes. For each candidate result node, the system queries the document structure tree to obtain its associated nodes in the tree. Associated nodes can include parent nodes (i.e., the parent title block that directly contains this node), child nodes (i.e., the child nodes that directly contain this node), or sibling nodes (i.e., other nodes that share the same parent node as the current node). For example, if a table block is matched, the system will obtain its parent title block as the parent node and related paragraph blocks under the same title as sibling nodes; if a paragraph block is matched, the system will obtain its parent title block as the parent node and related table blocks under the same title as sibling nodes. Then, the system assembles the candidate result nodes and their associated nodes into a knowledge block to be sorted, expanding the originally isolated match points into information units with complete context. This knowledge block contains rich information such as the matched node itself, chapter title, related descriptions, and related parameters, providing users with a complete semantic background.

[0058] At box 309, the system sorts knowledge blocks based on their overall relevance to the query statement. The system can comprehensively consider the overall relevance scores of all nodes associated with a knowledge block, as well as the structural integrity of the knowledge block, to calculate the overall relevance score for each knowledge block. The overall relevance score of a knowledge block can be calculated by aggregating the overall relevance scores of its constituent nodes, such as taking the maximum value, average value, or weighted sum. Simultaneously, the system can consider the structural integrity of the knowledge block, such as whether it contains both a title and content, or whether it contains multiple related nodes, to apply appropriate weighting. Then, the system sorts the knowledge blocks according to their overall relevance scores, with knowledge blocks scoring higher appearing first.

[0059] At box 310, the system can generate hierarchical search results and present them to the user. When generating search results, the system can display knowledge blocks in a collapsible or expandable hierarchical format. Each search result can include the content of the matched node and its path information in the document's structure tree, enabling users to clearly understand the result's location and context within the document. For table-type results, the system can also highlight matched rows; for paragraph-type results, the system can highlight matched keywords. This hierarchical display method allows users to obtain complete relevant information from a single result without multiple clicks, greatly improving search efficiency.

[0060] In summary, the embodiments of the present invention, by performing structured parsing of the original technical documents and constructing a document structure tree, preserve the inherent logical structure and hierarchical relationships of the documents, enabling the retrieval system to perform refined retrieval at the semantic block level. This avoids the problem of users having to search twice due to the return of the entire document in traditional methods, significantly improving the recall and accuracy of retrieval results. By constructing a multi-level composite index including semantic vector index, parameter field index, and keyword inverted index, unified retrieval of semantic understanding, structured condition filtering, and precise keyword matching is achieved, solving the problem of the separation between semantic search and structured retrieval in the prior art. Furthermore, through the document structure tree-based reordering mechanism and knowledge block assembly, isolated hit nodes are expanded into information units containing complete context and presented to users in a hierarchical form, significantly improving the readability and completeness of retrieval results. Users can obtain comprehensive relevant information from a single result without multiple clicks.

[0061] Figure 4 This is a schematic diagram of a computing device that can be implemented according to embodiments of the present invention, and can be used to implement the system or device for technical document retrieval described above. For example... Figure 4 As shown, computing device 400 may include bus 402 or other communication mechanism for transmitting information, and one or more hardware processors 404 coupled to bus 402 for processing information. The one or more hardware processors 404 may include, for example, one or more general-purpose microprocessors.

[0062] like Figure 4As shown, in some embodiments, computing device 400 may further include main memory 406 coupled to bus 402. Main memory 406 is used to store information and instructions executed by one or more processors 404, such as random access memory (RAM), cache, and / or other dynamic storage devices. Main memory 406 may also be used to store temporary variables or other intermediate information during the execution of instructions executed by one or more processors 404. These instructions, when stored in storage media accessible to one or more processors 404, can cause computing device 400 to become a dedicated machine customized to perform the operations specified in the instructions. Storage device 408 may include non-volatile and / or volatile storage media. Non-volatile storage media may include, for example, optical discs or magnetic disks. Volatile storage media may include dynamic memory. Common forms of storage media may include, for example, floppy disks, hard disks, solid-state drives, magnetic tape, or any other magnetic data storage media, CD-ROMs, any other optical data storage media, any physical media with a perforated pattern, RAM, DRAM, PROM, and EPROM, FLASH-EPROM, NVRAM, any other memory chip or cartridge, or their networking versions.

[0063] like Figure 4 As shown, in some embodiments, computing device 400 may further include one or more communication interfaces 410 coupled to bus 402. Communication interface 410 may provide bidirectional data communication coupling to one or more network links connected to one or more networks. As another example, communication interface 410 may be a local area network (LAN) card to provide data communication connectivity to a LAN-compatible (or WAN component communicating with a WAN) network. Wireless links may also be implemented.

[0064] The execution of certain operations can be distributed across processors rather than residing within a single machine, but rather deployed across multiple machines. In some example embodiments, the processor or processor-implemented engine may reside in a single geographic location (e.g., in a home environment, office environment, or server farm). In other example embodiments, the processor or processor-implemented engine may be distributed across multiple geographic locations.

[0065] Each of the processes, methods, and algorithms described herein may be embodied in code modules executed by one or more computer systems or computer processors including computer hardware, and may be fully or partially automated by these code modules. The processes and algorithms may be implemented, partially or fully, in dedicated circuit systems.

[0066] When the functions disclosed herein are implemented as software functional units and sold or used as stand-alone products, they may be stored in a processor-executable, non-volatile, computer-readable storage medium. Specific technical solutions (all or part) disclosed herein, or aspects contributing to the prior art, may be embodied in the form of a software product. The software product may be stored in a storage medium and includes several instructions that cause a computing device (which may be a personal computer, server, network device, etc.) to perform all or some steps of the methods of the embodiments of this application. The storage medium may include a flash drive, portable hard disk drive, ROM, RAM, magnetic disk, optical disk, other media operable to store program code, or any combination thereof.

[0067] Specific embodiments further provide a system including a processor and a computer-readable storage medium storing instructions executable by the processor to cause the system to perform operations corresponding to steps in any method of the embodiments disclosed above. Specific embodiments further provide a computer-readable storage medium storing instructions executable by one or more processors to cause the one or more processors to perform operations corresponding to steps in any method of the embodiments disclosed above.

[0068] The embodiments disclosed herein can be implemented via a cloud platform, server, or server cluster (collectively referred to below as the "Service System") that interacts with a client. The client can be a terminal device or a client registered by a user at the platform, wherein the terminal device can be a mobile terminal, a personal computer (PC), or any device that can have the platform application installed.

[0069] The various features and processes described above can be used independently of each other or combined in various ways. All possible combinations and sub-combinations are intended to fall within the scope of this disclosure. Additionally, certain method or process blocks may be omitted in some embodiments. The methods and processes described herein are not limited to any particular order, and their associated blocks or states may be executed in other suitable orders. For example, described blocks or states may be executed in an order other than that specifically disclosed, or multiple blocks or states may be combined into a single block or state. Example blocks or states may be executed sequentially, in parallel, or in some other manner. Blocks or states may be added to or removed from the disclosed example embodiments. The exemplary systems and components described herein may be configured differently than described. For example, components may be added to, removed from, or rearranged compared to the disclosed example embodiments.

[0070] The various operations of the exemplary methods described herein can be performed at least in part by an algorithm. The algorithm may be included in program code or instructions stored in memory (e.g., the aforementioned computer-readable storage medium). This algorithm may include a machine learning algorithm. In some embodiments, the machine learning algorithm may not explicitly turn the computer into an execution function but may learn from training data to produce a predictive model of the execution function.

[0071] The various operations of the exemplary methods described herein can be performed, at least in part, by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Whether temporarily or permanently configured, these processors can constitute an engine of processor implementation that operates to perform one or more of the operations or functions described herein.

[0072] Similarly, the methods described herein may be implemented at least in part by a processor, wherein one or more specific processors are instances of hardware. For example, at least some operations of the methods may be performed by one or more processors or an engine implemented by a processor. Furthermore, one or more processors may also be operable to support the execution of relevant operations in a “cloud computing” environment or as the execution of relevant operations in a “Software as a Service” (SaaS) context. For example, at least some operations may be performed by a group of computers (as an example of a machine containing processors), wherein these operations are accessible via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., application programming interfaces (APIs)).

[0073] The execution of certain operations can be distributed across processors rather than residing within a single machine, and can be deployed across multiple machines. In some example embodiments, the processor or processor-implemented engine may reside in a single geographic location (e.g., in a home environment, office environment, or server farm). In other example embodiments, the processor or processor-implemented engine may be distributed across multiple geographic locations.

[0074] Throughout this specification, multiple instances may be implemented as components, operations, or structures described as a single instance. Although individual operations of one or more methods are illustrated and described as separate operations, one or more of these individual operations may be performed simultaneously, and not necessarily in the order illustrated. Structures and functions presented as separate components in the example configuration may be implemented as composite structures or components. Similarly, structures and functions presented as single components may be implemented as single components. These and other variations, modifications, additions, and improvements fall within the scope of this document.

[0075] As used herein, "or" is inclusive rather than exclusive unless explicitly indicated by the context. Therefore, in this document, "A, B, or C" means "A, B, A and B, A and C, B and C, or A, B, and C" unless explicitly indicated by the context. Furthermore, "and" is combined and separate unless explicitly indicated by the context. Therefore, in this document, "A and B" means "A and B, combined or separate" unless explicitly indicated by the context. Additionally, multiple instances of resources, operations, or structures described herein may be provided as a single instance. Furthermore, the boundaries between various resources, operations, engines, and data storage devices are somewhat arbitrary and specific operations are illustrated within the context of a particular illustrative configuration. Other functional assignments are foreseeable and fall within the scope of various embodiments of this disclosure. Generally, structures and functions presented as individual resources in example configurations may be implemented as combined structures or resources. Similarly, structures and functions presented as single resources may be implemented as single resources. These and other changes, modifications, additions, and improvements fall within the scope of the embodiments of this disclosure as expressed in the appended claims. Therefore, this specification and drawings should be considered illustrative rather than restrictive.

[0076] The terms “comprising” or “including” are used to indicate the presence of a subsequently claimed feature, but do not preclude the addition of other features. Unless otherwise specifically stated or otherwise understood in the context in which they are used, conditional language such as “may,” “can,” “may,” and “can” is generally intended to convey that certain embodiments include certain features, components, and / or steps that are not included in other embodiments. Therefore, this conditional language is generally not intended to imply that one or more embodiments require features, components, and / or steps in any way, or that one or more embodiments must include logic for determining whether such features, components, and / or steps are included in or performed in any particular embodiment, with or without user input or prompts.

[0077] Although the general outline of the subject matter has been described with reference to specific exemplary embodiments, various modifications and changes may be made to these embodiments without departing from the broad scope of embodiments of this disclosure. Where more than one embodiment is disclosed, these embodiments of the subject matter may be referred to individually or collectively herein as the term "invention," this is for convenience only and is not intended to automatically limit the scope of this application to any single disclosure or concept.

[0078] The embodiments illustrated herein are described in detail to enable those skilled in the art to practice the disclosed teachings. Other embodiments may be used and derived therefrom, such that structural and logical substitutions and changes may be made without departing from the scope of this disclosure. Therefore, “implementation” is not intended to be limiting, and the scope of the various embodiments is defined only by the appended claims and their full scope.

Claims

1. A method for technical document search, characterized by, include: Obtain the original technical documentation; The original technical document is subjected to structured parsing to identify multiple semantic blocks contained in the original technical document and the hierarchical relationship between the multiple semantic blocks, wherein the multiple semantic blocks include title blocks and content blocks, the title blocks include the hierarchical relationship, and the content blocks include at least one of descriptive content blocks and structured data blocks; A document structure tree is constructed based on the multiple semantic blocks and the hierarchical relationship. Each node in the document structure tree corresponds to a semantic block, and each non-root node is associated with the path information of all its ancestor nodes. A multi-level composite index is constructed based on the nodes in the document structure tree. The multi-level composite index includes: a first type of index for supporting retrieval based on semantic similarity; and a second type of index for supporting retrieval based on structured conditions. In response to the received query, searches are performed in both the first type of index and the second type of index to obtain multiple candidate result nodes; and Based on the positions of the candidate result nodes in the document structure tree, the candidate result nodes are reordered, and search results are generated. The first type of index includes a semantic vector index, and the method further includes: constructing the semantic vector index by encoding the descriptive content block into a feature vector; the second type of index includes a parameter field index, and the method further includes: constructing the parameter field index by extracting key-value pairs from the structured data block and constructing a field index for the key-value pairs, wherein the parameter field index supports range queries for numeric fields or exact match queries for enumerated fields. The reordering of the multiple candidate result nodes includes: The comprehensive relevance score of each candidate result node to the query statement is calculated by the following steps: the relevance scores of the candidate result node from different indexes are normalized to obtain multiple normalized relevance scores; and the multiple normalized relevance scores are weighted and merged according to preset weights to obtain the comprehensive relevance score of the candidate result node. For each candidate result node, obtain its associated nodes in the document structure tree, wherein the associated nodes include one or more of the following: the parent node, child node, or sibling node of the candidate result node; The candidate result nodes and their associated nodes are assembled into knowledge blocks; Based on the comprehensive relevance scores of each candidate result node associated with the knowledge block, the comprehensive relevance score of the knowledge block is calculated; and The knowledge blocks are ranked according to their overall relevance scores.

2. The method of claim 1, wherein, The structured parsing of the original technical document includes: By using a pre-trained document structure analysis model, the title blocks in the original technical document are identified, and the content blocks corresponding to each level in the title blocks are determined, thereby determining the hierarchical relationship.

3. The method according to claim 1, characterized in that, The content block also includes code blocks or formula blocks.

4. The method according to claim 1, characterized in that, The multi-level composite index further includes a third type of index, which includes a keyword inverted index to support precise keyword matching retrieval, and the method includes: In response to the received query statement, searches are performed in the first type index, the second type index, and the third type index respectively to obtain the plurality of candidate result nodes.

5. The method according to claim 1, characterized in that, Generating search results includes displaying the knowledge block in a hierarchical form that can be collapsed or expanded, wherein the knowledge block includes the content block of the node associated with the knowledge block and the path information of the node in the document structure tree.

6. A computer-readable storage medium storing instructions, characterized in that, When executed by one or more processors, the instructions cause the one or more processors to perform the method according to any one of claims 1-5.

7. A computer program product, the computer program product comprising instructions, characterized in that, When executed by one or more processors, the instructions cause the one or more processors to perform the method according to any one of claims 1-5.

Images