Patent retrieval method based on semantic understanding
Through a patent retrieval method based on semantic understanding, the document parser and graphic information detection model are used to efficiently parse and structure patent documents. The large model and Bge-m3 model are combined to generate embedding vectors, which solves the problems of low accuracy and efficiency of patent retrieval in the existing technology and achieves efficient patent retrieval result matching.
Patent Information
- Application Number
- CN202510794081.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-13
- Publication Date
- 2025-09-26
AI Technical Summary
Existing patent retrieval methods are difficult to capture the deep semantic associations behind query intent, resulting in low user retrieval accuracy and efficiency.
A patent retrieval method based on semantic understanding is adopted. Through the comprehensive application of document parser, graphic information detection model, sorting tree algorithm, large model and Bge-m3 model, efficient parsing and structured processing of patent documents are achieved, embedded vectors are generated and quickly searched in the vector database.
It significantly improves the relevance and efficiency of patent searches and meets users' query needs.
Smart Images
Figure CN120705301A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of patent document retrieval, and specifically is a patent retrieval method based on semantic understanding. Background Art
[0002] Patent database systems are currently widely used to support innovation and technological development. With the advancement of AI technology, particularly natural language processing (NLP) and large language models (LLMs), patent retrieval is undergoing a profound transformation. The new generation of intelligent patent databases not only overcomes the shortcomings of traditional methods but also provides users with a more accurate, fast, and user-friendly platform.
[0003] Traditional patent search methods mainly rely on keyword matching. Although this method is simple and direct, it has limitations in capturing the deep semantic associations behind the query intent, resulting in low accuracy and efficiency for users in the patent search process.
[0004] The present invention proposes a patent retrieval method based on semantic understanding to solve the above technical problems. Summary of the Invention
[0005] The present invention aims to solve at least one of the technical problems existing in the prior art; to this end, the present invention proposes a patent search method based on semantic understanding, which is used to solve the technical problem that the patent search method in the prior art is difficult to capture the deep semantic associations behind the query intent, resulting in low accuracy and efficiency of users in the patent search process.
[0006] To achieve the above objectives, the first aspect of the present invention provides a patent search method based on semantic understanding, comprising:
[0007] S1: Obtain the original texts of several patent documents, the user's query content and the patent history query records;
[0008] S2: Based on the document parser, the original text is recognized and parsed to obtain the text content;
[0009] S3: Extract images from the original text based on the graphic information detection model to obtain some structural information; sort the structural information based on the sorting tree algorithm to obtain a structural information sequence; the graphic information detection model is built based on the YOLO model;
[0010] S4: Integrate structural information sequences and textual content into formatted patent text;
[0011] S5: Extracting historical patent query records based on the Qwen model to obtain several key query information;
[0012] S6: Train the Bge-m3 model based on several query key information to obtain a semantic model; semantically capture the formatted patent text based on the semantic model to obtain an embedding vector; save the embedding vector and the corresponding formatted patent text to the Milvus vector database;
[0013] S7: Input the user's query content into the semantic model to obtain a query vector; input the query vector into the Milvus vector database for retrieval to obtain a patent search result set.
[0014] Preferably, the image extraction of the original text based on the graphic information detection model includes:
[0015] Extract the original text of the patent document and convert it into an image to be recognized. Use the image information detection model to segment the image to be recognized into several fixed-size grid cells, predict the bounding box information and category information of each grid cell, and integrate the bounding box information and category information into structural information. The bounding box information includes the grid cell's coordinates x, coordinates y, width w, height h, and confidence level C.
[0016] Through the loss function Optimize the image information detection model;
[0017] Among them, λ coord is the coordinate loss weight, is the indicator of the presence of an object in the jth bounding box of the i-th grid cell, is the indicator of no object existence in the jth bounding box within the i-th grid cell, represents the target presence indicator of the i-th grid cell, Indicates whether the object in the i-th grid cell belongs to category c, classes is the set of all categories, p i (c) is the predicted probability that the i-th grid cell belongs to category c, is the true probability that the i-th grid cell belongs to category c; i is the number of the grid cell, i = 0, 1, 2, ..., S, S + 1 is the total number of grid cells; j is the number of the bounding box, j = 0, 1, 2, ..., B, B + 1 is the total number of bounding boxes within a single grid cell.
[0018] Preferably, the sorting of the plurality of structural information based on the sorting tree algorithm includes:
[0019] A1: Extract some structural information and mark them as nodes, and integrate some nodes into a node list;
[0020] A2: Start traversing from the last node in the node list;
[0021] A3: Determines whether there is a node that overlaps with the current node on the x-axis. If so, marks the node closest to the current node among the overlapping nodes as the nearest node, marks the current node as a child of the nearest node, and jumps to A4. If not, jumps to A4. The current node refers to the node being traversed.
[0022] A4: Determines whether the current node is the first node in the node list. If so, the current node is marked as the root node. After the traversal is completed, a sorted tree is generated and the process jumps to A5. If not, the current node is marked as a child node of the first node in the node list and the process jumps to A2.
[0023] A5: Perform a pre-order traversal on the sorting tree to obtain a sequence of structural information.
[0024] Preferably, the extraction of patent history query records based on the Qwen large model includes:
[0025] Extract patent history query records and set prompt word templates; based on the prompt word templates, guide the Qwen large model to extract patent history query records and obtain several key query information; among them, the key query information includes query conditions and query results, and the query results include patent number, patent name and patent abstract.
[0026] Preferably, the training of the Bge-m3 model based on several query key information includes:
[0027] Extract several query key information, mark the query conditions and query results in each set of query key information as a training data, and integrate several training data into a training dataset; use the training dataset to train the Bge-m3 model, and mark the trained Bge-m3 model as a semantic model.
[0028] Preferably, the Bge-m3 model is constructed by the loss function Fine-tune the Bge-m3 model; where Q represents the key information of the query, and D + Denotes documents related to the query, D - represents documents that are irrelevant to the query, and Similarity,) is the similarity function.
[0029] Preferably, the semantic capture of the formatted patent text based on the semantic model includes:
[0030] Extract the formatted patent text, concatenate the patent name and patent abstract in the formatted patent text to obtain an embedded data item; input the embedded data item into the semantic model to obtain the corresponding embedding vector.
[0031] Preferably, the step of saving the embedded vector and the corresponding formatted patent text to the Milvus vector database comprises:
[0032] Extract the embedded vector and the corresponding formatted patent text; the formatted patent text includes the patent number, patent name and patent abstract; mark the patent number in the formatted patent text as the primary key of the Milvus vector database, and mark the corresponding patent name, patent abstract and embedded vector as attributes of the primary key; save the primary key and several attributes of the primary key as a data record in the Milvus vector database.
[0033] Preferably, inputting the query vector into the Milvus vector database for retrieval comprises:
[0034] Extract the query vector; input the query vector into the Milvus vector database, and calculate the correlation score between the embedded vector and the query vector in each data record in the Milvus vector database in turn; determine whether the correlation score is greater than a preset score threshold; if so, mark the corresponding data record as a search result; if not, continue to search for the remaining data records; sort several search results using the Bge-m3-rerank reranking model to obtain a patent search result set.
[0035] Preferably, the Bge-m3-rerank reranking model uses the ranking loss function L rank =∑ p,q max(0,ΔS-S p +S q ) fine-tunes the Bge-m3-rerank reranking model; where ΔS is the expected correlation score gap, S p and S q Score the relevance of two documents to the same query.
[0036] Compared with the prior art, the present invention has the following beneficial effects:
[0037] The present invention realizes efficient parsing and structural processing of the original text of patent documents through a series of technical means, including the comprehensive application of document parsers, graphic information detection models, sorting tree algorithms, large models and Bge-m3 models, thereby improving the relevance and efficiency of patent retrieval. First, the document parser and graphic information detection model can accurately extract structural information in text content and images from the original text; then, these structural information are optimized and sorted using the sorting tree algorithm to ensure high-quality integration of formatted patent texts. The large model analysis based on historical query records extracts key information, and the semantic model is constructed in combination with the Bge-m3 model. The deep semantic features of the formatted patent text are captured according to the model semantic model to generate corresponding embedding vectors; the embedding vectors and the corresponding formatted texts are stored in a vector database; the user's query content is input into the semantic model to generate a query vector, and the query vector is quickly searched and matched in the vector database to obtain a retrieval result set; this process significantly improves the retrieval efficiency and accuracy, thereby effectively meeting the user's query needs. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0039] Figure 1 The overall flow chart of the patent search method based on semantic understanding of the present invention;
[0040] Figure 2 Schematic diagram of the principle of image extraction from original text based on the YOLO model in the present invention;
[0041] Figure 3 A schematic diagram of the principle of extracting patent history query records based on the Qwen large model in the present invention;
[0042] Figure 4 A schematic diagram of the principle of semantic capture of formatted patent text based on a semantic model in the present invention;
[0043] Figure 5 Schematic diagram of the principle of storing or retrieving patents based on the Milvus vector database in the present invention. DETAILED DESCRIPTION
[0044] The technical solutions of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.
[0045] See also Figure 1-Figure 5 The first embodiment of the present invention provides a patent search method based on semantic understanding, comprising:
[0046] S1: Obtain the original texts of several patent documents, the user's query content and the patent history query records;
[0047] S2: Based on the document parser, the original text is recognized and parsed to obtain the text content;
[0048] S3: Extract images from the original text based on the graphic information detection model to obtain some structural information; sort the structural information based on the sorting tree algorithm to obtain a structural information sequence; the graphic information detection model is built based on the YOLO model;
[0049] S4: Integrate the structural information sequence and text content into a formatted patent text; wherein the formatted patent text is specifically saved in JSON format;
[0050] S5: Extracting historical patent query records based on the Qwen model to obtain several key query information;
[0051] S6: Train the Bge-m3 model based on several query key information to obtain a semantic model; semantically capture the formatted patent text based on the semantic model to obtain an embedding vector; save the embedding vector and the corresponding formatted patent text to the Milvus vector database;
[0052] S7: Input the user's query content into the semantic model to obtain a query vector; input the query vector into the Milvus vector database for retrieval to obtain a patent search result set.
[0053] In this embodiment, image extraction of the original text is performed based on the graphic information detection model, including:
[0054] Extract the original text of the patent document and convert it into an image to be recognized. Use the image information detection model to segment the image to be recognized into several fixed-size grid cells, predict the bounding box information and category information of each grid cell, and integrate the bounding box information and category information into structural information. The bounding box information includes the grid cell's coordinates x, coordinates y, width w, height h, and confidence level C.
[0055] Through the loss function Optimize the image information detection model;
[0056] Among them, λ coord is the coordinate loss weight, is the indicator of the presence of an object in the jth bounding box of the i-th grid cell, is the indicator of no object existence in the jth bounding box within the i-th grid cell, represents the target presence indicator of the i-th grid cell, Indicates whether the object in the i-th grid cell belongs to category c, classes is the set of all categories, p i (c) is the predicted probability that the i-th grid cell belongs to category c, is the true probability that the i-th grid cell belongs to category c; i is the number of the grid cell, i = 0, 1, 2, ..., S, S + 1 is the total number of grid cells; j is the number of the bounding box, j = 0, 1, 2, ..., B, B + 1 is the total number of bounding boxes within a single grid cell.
[0057] Furthermore, the graphic information detection model is trained using a document structure dataset, which includes PubLayNet or CDLA.
[0058] In this embodiment, the structure information is sorted based on the sorting tree algorithm, including:
[0059] A1: Extract some structural information and mark them as nodes, and integrate some nodes into a node list;
[0060] A2: Start traversing from the last node in the node list;
[0061] A3: Determines whether there is a node that overlaps with the current node on the x-axis. If so, marks the node closest to the current node among the overlapping nodes as the nearest node, marks the current node as a child of the nearest node, and jumps to A4. If not, jumps to A4. The current node refers to the node being traversed.
[0062] A4: Determines whether the current node is the first node in the node list. If so, the current node is marked as the root node. After the traversal is completed, a sorted tree is generated and the process jumps to A5. If not, the current node is marked as a child node of the first node in the node list and the process jumps to A2.
[0063] A5: Perform a pre-order traversal on the sorting tree to obtain a sequence of structural information.
[0064] In this embodiment, the patent history query records are extracted based on the Qwen large model, including:
[0065] Extract patent history query records and set prompt word templates; based on the prompt word templates, guide the Qwen large model to extract patent history query records and obtain several key query information; among them, the key query information includes query conditions and query results, and the query results include patent number, patent name and patent abstract.
[0066] For example, the patent history query record and prompt word template are set as follows:
[0067] Patent history query records:
[0068] Chat log:
[0069] User 1: We are discussing a patent for a fixing device for a mining crusher.
[0070] User 2: The application number of this patent is 20242030379BR.
[0071] User 1: The patent name is "A fixing device for a mining crusher."
[0072] User 2: Price is negotiable.
[0073] User 1: Status is pending transaction.
[0074] User 2: Transaction type is transfer.
[0075] User 1: The added date is 2024-09-03.
[0076] User 2: Patent type ID is 2.
[0077] User 1: The sales type is not licensed.
[0078] Prompt word template:
[0079] {
[0080] "patentId":<patent number>,
[0081] "patentName":<patent name>,
[0082] "abstract":<Patent abstract>
[0083] }.
[0084] In this embodiment, the Bge-m3 model is trained based on several query key information, including:
[0085] Extract several query key information, mark the query conditions and query results in each set of query key information as a training data, and integrate several training data into a training dataset; use the training dataset to train the Bge-m3 model, and mark the trained Bge-m3 model as a semantic model.
[0086] In this embodiment, the Bge-m3 model is combined with the contrastive learning loss function through LLaMa Factory (Large Language Model Factory) Fine-tune the Bge-m3 model; where Q represents the key information of the query, and D + Denotes documents related to the query, D - Represents documents that are irrelevant to the query, and Similarity(,) is the similarity function.
[0087] It should be noted that the purpose of fine-tuning the Bge-m3 model through the contrastive learning loss function L2 is to ensure that the training objective of the model is to maximize the similarity between the query content and relevant documents and minimize the similarity between the query content and irrelevant documents, thereby improving the accuracy of patent retrieval.
[0088] In this embodiment, semantic capture of formatted patent text is performed based on a semantic model, including:
[0089] Extract the formatted patent text, concatenate the patent name and patent abstract in the formatted patent text to obtain an embedded data item; input the embedded data item into the semantic model to obtain the corresponding embedding vector.
[0090] In this embodiment, the embedded vector and the corresponding formatted patent text are saved to the Milvus vector database, including:
[0091] Extract the embedded vector and the corresponding formatted patent text; the formatted patent text includes the patent number, patent name and patent abstract; mark the patent number in the formatted patent text as the primary key of the Milvus vector database, and mark the corresponding patent name, patent abstract and embedded vector as attributes of the primary key; save the primary key and several attributes of the primary key as a data record in the Milvus vector database.
[0092] In this embodiment, the query vector is input into the Milvus vector database for retrieval, including:
[0093] Extract the query vector; input the query vector into the Milvus vector database, and calculate the correlation score between the embedded vector and the query vector in each data record in the Milvus vector database in turn; determine whether the correlation score is greater than a preset score threshold; if so, mark the corresponding data record as a search result; if not, continue to search for the remaining data records; sort several search results using the Bge-m3-rerank reranking model to obtain a patent search result set.
[0094] It should be noted that by inputting the user's query content into the semantic model, obtaining the query vector, and calculating the correlation score between the query vector and the embedded vector, it is possible to quickly match patent information that is semantically relevant to the query content, thereby helping to improve the user's accuracy and efficiency in the patent search process; by sorting several search results through the Bge-m3-rerank reranking model, it is possible to ensure that search results that are more relevant to the user's query content are ranked higher, thereby helping to improve the user's accuracy and efficiency in the patent search process.
[0095] In this embodiment, the Bge-m3-rerank reranking model uses the pairwise ranking loss function L rank =∑ p,q max(0,ΔS-S p +S q ) fine-tunes the Bge-m3-rerank reranking model; where ΔS is the expected correlation score gap, S p and S q Score the relevance of two documents to the same query.
[0096] It should be noted that the Bge-m3-rerank reranking model uses a cross-attention mechanism to re-evaluate the relevance of several retrieval results by using the ranking loss function L rank Fine-tuning the Bge-m3-rerank reranking model can ensure the rationality of the ranking position for any pair of documents. When the relevance of the first document is higher, the relevance score of the first document is correspondingly higher than the relevance score of the second document.
[0097] The above embodiments are only used to illustrate the technical method of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.
Claims
1. A patent search method based on semantic understanding, characterized in that: S1: Obtain the original texts of several patent documents, the user's query content and the patent history query records; S2: Based on the document parser, the original text is recognized and parsed to obtain the text content; S3: Extract images from the original text based on the graphic information detection model to obtain some structural information; sort the structural information based on the sorting tree algorithm to obtain a structural information sequence; the graphic information detection model is built based on the YOLO model; S4: Integrate structural information sequences and textual content into formatted patent text; S5: Extracting historical patent query records based on the big model to obtain several key query information; S6: Construct a semantic model based on several query key information and the Bge-m3 model; capture the semantics of the formatted patent text based on the semantic model to obtain an embedding vector; save the embedding vector and the corresponding formatted patent text to the vector database; S7: Input the user's query content into the semantic model to obtain a query vector; input the query vector into the vector database for retrieval to obtain a patent search result set.
2. A patent search method based on semantic understanding according to claim 1, characterized in that: The image extraction of the original text based on the graphic information detection model includes: The original text of the patent document is extracted and converted into an image to be identified. The image to be identified is segmented into several fixed-size grid cells through an image information detection model, and the bounding box information and category information of each grid cell are predicted, and the bounding box information and category information are integrated into structural information. The bounding box information includes the grid cell's x coordinate, y coordinate, width, height, and confidence level.
3. A patent search method based on semantic understanding according to claim 2, characterized in that: The step of sorting the structural information based on the sorting tree algorithm includes: A1: Extract some structural information and mark them as nodes, and integrate some nodes into a node list; A2: Start traversing from the last node in the node list; A3: Determines whether there is a node that overlaps with the current node on the x-axis. If so, marks the node closest to the current node among the overlapping nodes as the nearest node, marks the current node as a child of the nearest node, and jumps to A4. If not, jumps to A4. The current node refers to the node being traversed. A4: Determines whether the current node is the first node in the node list. If so, the current node is marked as the root node. After the traversal is complete, a sorted tree is generated and the process jumps to A5. If not, the current node is marked as a child node of the first node in the node list and the process jumps to A2. A5: Perform a pre-order traversal on the sorting tree to obtain a sequence of structural information.
4. A patent search method based on semantic understanding according to claim 1, characterized in that: The extraction of patent history query records based on the large model includes: Extract patent history query records and set prompt word templates; based on the prompt word templates, guide the Qwen large model to extract patent history query records and obtain several key query information; among them, the key query information includes query conditions and query results, and the query results include patent number, patent name and patent abstract.
5. A patent search method based on semantic understanding according to claim 4, characterized in that: The semantic model is constructed based on several query key information and the Bge-m3 model, including: Extract several query key information, mark the query conditions and query results in each set of query key information as a training data, and integrate several training data into a training dataset; use the training dataset to train the Bge-m3 model, and mark the trained Bge-m3 model as a semantic model.
6. A patent search method based on semantic understanding according to claim 5, characterized in that: The Bge-m3 model, through the loss function Fine-tune the Bge-m3 model; where Q represents the key information of the query, and D + Denotes documents related to the query, D - Represents documents that are irrelevant to the query, and Similarity(,) is the similarity function.
7. The patent search method based on semantic understanding according to claim 1, characterized in that: The semantic capture of the formatted patent text based on the semantic model includes: Extracting formatted patent text; wherein the formatted patent text includes patent number, patent name and patent abstract; The patent name and patent abstract in the formatted patent text are concatenated to obtain an embedded data item; the embedded data item is input into the semantic model to obtain the corresponding embedding vector.
8. A patent search method based on semantic understanding according to claim 7, characterized in that: Saving the embedded vector and the corresponding formatted patent text to the vector database includes: Extract the embedded vector and the corresponding formatted patent text; mark the patent number in the formatted patent text as the primary key of the vector database, and mark the corresponding patent name, patent abstract and embedded vector as attributes of the primary key; save the primary key and several attributes of the primary key as a data record in the vector database.
9. A patent search method based on semantic understanding according to claim 1, characterized in that: Inputting the query vector into the vector database for retrieval includes: Extract the query vector; input the query vector into the Milvus vector database, and calculate the correlation score between the embedded vector and the query vector in each data record in the Milvus vector database in turn; determine whether the correlation score is greater than a preset score threshold; if so, mark the corresponding data record as a search result; if not, continue to search for the remaining data records; sort several search results using the Bge-m3-rerank reranking model to obtain a patent search result set.
10. A patent search method based on semantic understanding according to claim 9, characterized in that: The Bge-m3-rerank reranking model uses the ranking loss function L rank =∑ p,q max(0,ΔS-S p +S q ) fine-tunes the Bge-m3-rerank reranking model; where ΔS is the expected correlation score gap, S p and S q Score the relevance of two documents to the same query.