Intelligent patent retrieval method and system based on large language model reordering technology

Abstract

The invention discloses an intelligent patent retrieval method and system based on a large language model reordering technology, objects processed by the method comprise a variety of input data including patent public numbers, free texts and text files, key information is extracted, query texts are constructed, retrieval recall is executed, sparse and dense vector retrieval technologies are combined, and the retrieval efficiency is improved. And results are fused through a min sum merge algorithm. The de-noising step uses a sliding window strategy and vector coding to extract information fragments related to the query text. And finally, by training a reordering model and finely adjusting a recommendation list, the ranking of retrieval results is optimized, and the accuracy and correlation of retrieval are improved. According to the intelligent retrieval method, the patent retrieval efficiency and accuracy are improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, specifically to the field of natural language processing and deep learning, and in particular to an intelligent patent retrieval method and system based on large language model re-ranking technology. Background Art

[0002] In the current information age, the pace of technological innovation is accelerating, and with it comes a sharp increase in the number of patent documents. This growth has brought unprecedented challenges: how to efficiently and accurately retrieve valuable information from massive patent data has become a major problem in the field of patent analysis and research and development. Traditional patent search systems often rely on simple keyword matching and static database queries. This method performs poorly in processing complex queries and understanding deep text semantics, and it is difficult to meet users' needs for efficient and accurate retrieval. Summary of the invention

[0003] The present application provides an intelligent patent search method and system based on large language model re-ranking technology, which can receive various forms of input, including existing patent publication numbers, free text or text files, and through advanced text processing and vectorization technology, achieve deep semantic understanding and efficient retrieval of patent documents.

[0004] In a first aspect, an intelligent patent search method based on a large language model re-ranking technology is provided, the method comprising:

[0005] Retrieval and recall: Use query text for semantic retrieval and recall, which is divided into sparse vector retrieval based on bag-of-words model and dense vector retrieval based on vector model;

[0006] Denoising: Based on the retrieved sequence, each candidate patent is denoised using a block and sliding window strategy;

[0007] Training the re-ranking model: Based on the denoising results, the general large language model is used as the base to train the re-ranking model. First, a dataset containing positive and negative example pairs is constructed. The positive examples come from patent citation data, and the negative examples are obtained through different mechanisms. Then, prompt words are constructed to stimulate the potential of the large language model, and the prompt words are concatenated with the query text and the text of the file to be compared to form the model input. A customized vector layer is connected after the model output layer for dimensionality reduction. Finally, the model is trained using the contrast loss function.

[0008] The denoised text pairs are input into the trained large language model to obtain a similarity set, the similarities are sorted to obtain an initial recommendation sequence, and the initial recommendation list is fine-tuned to obtain a final recommendation list; wherein the text pairs include the query text and the file text to be compared.

[0009] Optionally, before the step 1 of searching and recalling, it further includes obtaining multiple types of input data and performing data preprocessing and parsing, wherein the multiple types of input data include patent publication numbers, free texts, and text files;

[0010] In the data preprocessing and parsing results, the corresponding text segment is selected as the query text according to the preset field parameters; if the input is a patent publication number, the independent claim is used by default; if the input is free text, the preprocessed content is the query text; if the input is a text file, the content after recognition and verification is used as the query text for subsequent vector encoding and similarity matching.

[0011] Optionally, when the input data is a patent publication number, it specifically includes: searching and obtaining corresponding patent documents in a patent database by the patent publication number; parsing the obtained patent documents using preset rules to extract key fields; wherein the key fields at least include title, abstract, claims, description, inventor information and IPC classification number; splitting the claim text into items, and matching based on preset rules to distinguish independent claim groups from dependent claim groups, and constructing a relationship tree between them; subdividing the description content into technical field, background technology, invention content, illustrations and specific implementation methods;

[0012] When the input data is free text, it specifically includes: checking punctuation and automatically repairing typos on the input free text to ensure that the text quality meets the needs of subsequent processing;

[0013] When the input data is a text file, it specifically includes: selecting a corresponding parsing strategy according to the file type; wherein the file type includes at least TXT, Word document, text PDF or image PDF; for TXT, Word document or text PDF, directly performing text parsing and reading; if it is an image PDF file, calling OCR technology for text recognition; further performing punctuation checking and automatic repair of typos on the text obtained by OCR.

[0014] Optionally, the query text is used in the retrieval recall for semantic retrieval recall, which is specifically divided into sparse vector retrieval based on the bag-of-words model and dense vector retrieval based on the vector model, including:

[0015] Sparse vectors are constructed using the TF-IDF algorithm and the BM25 algorithm to generate vectors for the title, abstract, claims, and specification for each patent;

[0016] Use the BGE vector model for dense vector retrieval and optimize the contrast loss to improve the model's ability to distinguish different input texts.

[0017] The two search results are fused through the min sum merge algorithm to generate a sequence containing patent publication numbers.

[0018] Optionally, in denoising, based on the retrieved sequence, each candidate patent is denoised using a block and sliding window strategy, specifically including:

[0019] Process the specified paragraphs of the referenced document in blocks to ensure that the semantic integrity is not damaged;

[0020] Use the vector model to vector encode each block and query text, and calculate the similarity. By presetting the maximum window length N, calculate the average similarity between the block and the query text in the window, draw a similarity change curve, and find the peak position.

[0021] Optionally, fine-tuning the initial recommendation list specifically includes:

[0022] Use the "technical problem extraction model" to extract the "technical problem to be solved" field of the present application and the document to be compared, and perform vector encoding;

[0023] Calculate the vector similarity between the application and the document to be compared, and scale them up and down in proportion;

[0024] Obtain the IPC classification number of each file to be compared and weight it according to the matching level;

[0025] All weighted recommendation lists are used for sorting to obtain the final recommendation list.

[0026] In the second aspect, an intelligent patent search system based on large language model re-ranking technology, the system comprises:

[0027] The retrieval recall module is used to perform semantic retrieval recall using query text, which is specifically divided into sparse vector retrieval based on the bag-of-words model and dense vector retrieval based on the vector model;

[0028] The denoising module is used to denoise each candidate patent using a block and sliding window strategy based on the retrieved sequence;

[0029] The module for training a re-ranking model is used to train a re-ranking model based on the denoising results using a general large language model as a base. First, a dataset containing positive and negative example pairs is constructed, where the positive examples come from patent citation data and the negative examples are obtained through different mechanisms. Then, a prompt word is constructed to stimulate the potential of the large language model, and the prompt word is concatenated with the query text and the text of the file to be compared to form the model input. A customized vector layer is connected after the model output layer for dimensionality reduction. Finally, the model is trained using a contrast loss function.

[0030] The fine-tuning module is used to input the denoised text pairs into the trained large language model to obtain a similarity set, sort the similarities to obtain an initial recommendation sequence, and fine-tune the initial recommendation list to obtain a final recommendation list; wherein the text pairs include the query text and the file text to be compared.

[0031] In a third aspect, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the intelligent patent retrieval method based on the large language model reranking technology as described in any one of the first aspects above is implemented.

[0032] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the intelligent patent retrieval method based on the large language model reranking technology described in any one of the first aspects above is implemented.

[0033] In a fifth aspect, a computer program product is provided, including a computer program / instruction, which, when executed by a processor, implements the intelligent patent retrieval method based on the large language model reordering technology as described in any one of the first aspects above.

[0034] The beneficial effects brought by the technical solution provided by the embodiment of the present application include at least:

[0035] (1) Sliding window denoising strategy: By introducing a sliding window strategy to denoise the files to be compared, the system can dynamically evaluate the relevance of text blocks and accurately extract information fragments related to the query text. This strategy effectively reduces the interference of noise data and improves the accuracy of text similarity calculation.

[0036] (2) Retrieval and recall mechanism combining sparse and dense vectors: This technology adopts a bidirectional retrieval strategy of sparse vectors (based on TF-IDF and bm25 algorithms) and dense vectors (based on bge vector model), and fuses the results through the min sum merge algorithm. This combination takes advantage of the advantages of the two vector retrieval methods and enhances the system's retrieval capability and accuracy for patent documents.

[0037] (3) Large language model re-ranking method: Using the pre-trained large language model as the basis, combined with a customized vector layer, the model re-ranking training and use are carried out. This method can optimize the ranking of search results based on the deep semantic relationship of the text content, thereby improving the performance of the patent search system and user satisfaction.

[0038] (4) Multi-dimensional application of weighting strategy: This solution not only adds weighting to solve technical problems in the recommendation list, but also considers the matching degree of IPC classification numbers and other bibliographic information. Through this multi-dimensional weighting, it is possible to more accurately screen and recommend patent documents that meet user needs, effectively improving the relevance and accuracy of the recommendation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the implementation methods or the description of the prior art. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.

[0040] Figure 1 A flowchart of the steps provided for the embodiments of the present application;

[0041] Figure 2 A flowchart of the denoising process provided in the embodiment of the present application;

[0042] Figure 3 A block diagram of the intelligent patent search system provided for this application;

[0043] Figure 4 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0045] In the description of the present application: the terms "including", "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to those steps or units that have been explicitly listed, but may also include other steps or units that are not explicitly listed but are inherent to these processes, methods, products or apparatuses, or steps or units added based on further optimization schemes conceived by the present invention.

[0046] In the current information age, the pace of technological innovation is accelerating, and with it comes a sharp increase in the number of patent documents. This growth has brought unprecedented challenges: how to efficiently and accurately retrieve valuable information from massive patent data has become a major problem in the field of patent analysis and research and development. Traditional patent search systems often rely on simple keyword matching and static database queries. This method performs poorly in processing complex queries and understanding deep text semantics, and it is difficult to meet users' needs for efficient and accurate retrieval.

[0047] In response to the above problems, this study proposed an intelligent patent search method based on large language model re-ranking technology. The system can receive inputs in various forms, including existing patent publication numbers, free text or text files, and through advanced text processing and vectorization technology, it can achieve deep semantic understanding and efficient retrieval of patent documents. The system first extracts the corresponding patent text content from the patent database, including information such as title, abstract, claims and instructions, and splits and vectorizes these contents according to preset rules, and then performs similarity search in the established vector database, thereby achieving fast and accurate information retrieval and recommendation. In addition, the system also introduces large model re-ranking technology, and trains the model through citation documents in the patent review process, further improving the relevance and accuracy of the search results. This patent search anti-method based on large language model re-ranking technology enables the system to continuously adapt to new data and retrieval needs, and has strong accuracy, practicality and scalability.

[0048] The significant advantage of this model is its high flexibility and intelligence, which can effectively cope with large-scale and dynamically changing patent data environments, and solve the problems of traditional patent search technology such as shallow semantic understanding, excessive noise and inaccurate recommendations. This is of great significance for accelerating scientific and technological innovation, improving R&D efficiency, and promoting the rational use and protection of intellectual property rights.

[0049] In one embodiment, Figure 1 As shown, an intelligent patent search method based on large language model re-ranking technology is provided, which can be applied to the server. The method also includes data pre-processing before the search and recall:

[0050] This method can handle many types of input data, including patent publication numbers, free text, and text files. The following are the specific steps for different input types:

[0051] Processing patent publication number:

[0052] Database query: First, the system searches and obtains the corresponding patent documents in the patent database through the patent publication number.

[0053] Field parsing: Use preset rules to parse the acquired patent documents and extract key fields such as title, abstract, claims, description, inventor information and IPC classification number.

[0054] Claim parsing: split the claim text into items, match them based on preset rules, distinguish independent claim groups from dependent claim groups, and build a relationship tree between them.

[0055] Specification analysis: The specification content is subdivided into sections such as "technical field", "background technology", "invention content", "illustration of drawings" and "specific implementation methods" to ensure that the information in each section is accurately captured.

[0056] Processing free text:

[0057] Text verification: Punctuation check and automatic correction of typos in the input free text to ensure that the text quality meets the needs of subsequent processing.

[0058] Processing text files:

[0059] File type identification: Select the appropriate parsing strategy based on the file type (such as TXT, Word document, text PDF, or image PDF).

[0060] Text parsing: For TXT, Word documents or text PDF, directly perform text parsing and reading.

[0061] Image recognition: If it is a picture PDF file, optical character recognition (OCR) technology is used for text recognition.

[0062] Text verification: further check punctuation and automatically repair typos on the text obtained by OCR.

[0063] After completing the above steps, the system selects the corresponding text segment as the query text according to the preset field parameters. Specifically, if the input is a patent publication number, the default is (independent claim), if the input is free text, the preprocessed content is (query text), and if the input is a text file, the content after recognition and verification is used as (query text). This is used for subsequent vector encoding and similarity matching.

[0064] Step 1: Retrieval and recall:

[0065] Use query_text for semantic retrieval and recall. Semantic retrieval is divided into two approaches: one is sparse vector retrieval based on the bag-of-words model, and the other is dense vector retrieval based on the vector model. Finally, a fusion algorithm is used to fuse the retrieval of the two approaches. The fusion algorithm used is the min sum merge algorithm commonly used in the industry. This algorithm is a sequence merging algorithm based on rank sorting. It inputs two sequences (the result sequence obtained by sparse retrieval and the result sequence obtained by dense retrieval) and outputs a publication number sequence (the result sequence after merging).

[0066] The sparse search uses the TF-IDF algorithm and the bm25 algorithm to build sparse vectors. Four sparse vectors (four vectors for each patent) are constructed in advance for the title, abstract, claims, and instructions of all patents. Then, the input field is controlled according to the parameters to search within one or more word bags, and the results are returned merged.

[0067] The dense retrieval uses the bge vector model, and the dual-tower model is trained on a pre-constructed dataset. The dataset includes positive example pairs and multiple negative example pairs, and is organized in a certain ratio. The training goal is to improve the vector model's ability to distinguish different input texts, so that similar patents have higher similarity scores and dissimilar patents have lower similarity scores. This similarity can be understood as the similarity that constitutes a novel conflict rather than textual similarity.

[0068] Steps to generate sparse vectors:

[0069] 1. Vocabulary construction: Segment the entire document set, remove stop words, extract all unique terms, and form a vocabulary. This vocabulary defines the dimensions of the subsequent sparse vectors.

[0070] 2. Calculate the TF-IDF and BM25 weights for each word in the document:

[0071] TF-IDF algorithm:

[0072]

[0073] Where t is a vocabulary item, d is a document of a patent title or other fields, is the frequency of term t in document d, N is the total number of documents, is the number of documents containing term t. is the TF-IDF weight of term t in document d.

[0074] BM25 algorithm:

[0075]

[0076] Among them, k 1 and b are adjustment parameters, usually k1 = 1.2 and b = 0.75. |d| is the number of words in document d of a patent title or other field, and avgdl is the average number of words in all documents. What is calculated here is the BM25 weight of term t in document d.

[0077] 3. Generate sparse vectors:

[0078] Create an initial vector for query_text of length equal to the vocabulary size.

[0079] Iterate over each term in query_text. If the term exists in the vocabulary, fill the corresponding position of the initial vector with its TF-IDF or BM25 weight; if the term does not appear in query_text, fill the corresponding position of the vector with 0.

[0080] Steps to generate dense vectors:

[0081] 1. Dataset construction and preprocessing,collect patent data pairs, which include positive pairs and negative pairs constructed by,different methods. Positive pairs are usually public citation pairs,,while negative pairs are similar but unrelated patent pairs constructed by,different methods.

[0082] 2. Model training, architecture: BGE model. The "BGE" model is a two-tower model, where each tower is usually composed of several layers of Transformers. The task of each tower is to convert text (such as patent abstracts or claims) into a fixed-length vector.

[0083] Contrastive loss calculation:

[0084] Contrastive Loss is used to optimize the model parameters, with the goal of making the vectors of positive pairs closer and the vectors of negative pairs farther apart. The calculation formula for contrast loss is as follows:

[0085]

[0086] Among them, P is the set of positive pairs, i and j are two vectors in the positive pair, k is the negative vector paired with i, and m is the margin, which is usually a positive number. Typically Euclidean distance or cosine distance is used.

[0087] Optimization: Use gradient descent to minimize the loss function and update the model weights.

[0088] 3. Vector generation: input query_text into the trained dense vector model to obtain a vector of fixed dimension.

[0089] Step 2: Denoising:

[0090] Using the recalled topN list obtained in step 1, denoise each candidate patent in the list using the block and sliding window strategy.

[0091] In this method, the text segment of the present application (default independent claim) and the text segment of the document to be compared need to be input into a pre-designed large language model to obtain a similarity score. The above steps are repeated for each document to be compared, that is, a similarity score is obtained with the text segment of the present application, and finally a similarity score list is obtained. By sorting the similarity score list, a recommended order is finally obtained.

[0092] Since the text segment inputted by the present application is relatively short (default independent claim) and the text segment inputted by the comparative document is relatively long (model specific implementation method), the lengths of the two text segments do not match and the noise of the text inputted by the comparative document is too large (a large amount of text is irrelevant to the text segment of the present application), which in turn causes inaccurate model prediction results. Therefore, the denoising method described below is used to simplify the input comparative document text segment, and the simplified text segment replaces the original text segment and is input into the model for prediction.

[0093] Retrieval and recall: Use query text for semantic retrieval and recall, which is divided into sparse vector retrieval based on bag-of-words model and dense vector retrieval based on vector model;

[0094] Denoising: Based on the retrieved sequence, each candidate patent is denoised using a block and sliding window strategy;

[0095] Training the re-ranking model: Based on the denoising results, the general large language model is used as the base to train the re-ranking model. First, a dataset containing positive and negative example pairs is constructed. The positive examples come from patent citation data, and the negative examples are obtained through different mechanisms. Then, prompt words are constructed to stimulate the potential of the large language model, and the prompt words are concatenated with the query text and the text of the file to be compared to form the model input. A customized vector layer is connected after the model output layer for dimensionality reduction. Finally, the model is trained using the contrast loss function.

[0096] The denoised text pairs are input into the trained large language model to obtain a similarity set, the similarities are sorted to obtain the initial recommendation sequence, and the initial recommendation list is fine-tuned to obtain the final recommendation list; wherein the text pairs include the query text and the document text to be compared. Figure 2 A flowchart of the denoising process is given, specifically:

[0097] 1. Blocking: Use a block algorithm to block the specified paragraphs of the comparison file. The principle of block division is not to destroy the semantic integrity. Specifically, hard truncation (non-period and semicolon truncation) is avoided as much as possible, and the length of each block is guaranteed to be as consistent as possible and not exceed the preset maximum length M.

[0098] Given a document D, define a function:

[0099]

[0100] Define split locations and costs:

[0101] Identify possible breakpoints:

[0102]

[0103] Where P is the set of all possible breakpoints, each breakpoint includes position i and the associated breakpoint cost c i If the last position of the breakpoint set P is not the end of the document, then the end of the document is added as an additional breakpoint.

[0104] Insert at the beginning:

[0105]

[0106] Where n is the length of the document.

[0107] Add extra breakpoints to satisfy the maximum block length M: If the distance between two consecutive breakpoints is greater than M, add new breakpoints at intervals of M between the two breakpoints.

[0108] Dynamic programming to find the optimal breakpoint:

[0109] Set the dynamic programming array best, where best[i] represents the minimum cost and path to reach the i-th breakpoint.

[0110] initialization:

[0111]

[0112] Recursive relationship:

[0113]

[0114] in, is the distance between positions j and i, c i is the cost of breakpoint i, sen_cost is a constant c, representing the basic segmentation cost.

[0115] Generate blocks based on optimal breakpoints:

[0116] Reverse tracing of the optimal path: Starting from the end of the best array, reverse tracing to find all the optimal breakpoints and generate segmentation intervals.

[0117] Split the document: Split the document D according to the segmentation interval to obtain the final segmentation result.

[0118] 2. Use the vector model to vectorize each block text and the query text query_text.

[0119] Use the pre-trained vector model for each block S i And the query text query_text is vector encoded:

[0120]

[0121] Herev i is the vector representation of the ith block, v q is the vector representation of the query text.

[0122] 3. Calculate the similarity between the vector of each block and the vector of query_text one by one.

[0123] For each block vector v i With the query text vector v q The similarity is calculated as:

[0124]

[0125] Cosine similarity is used here as the similarity metric. Represents vector v i and vector v q The cosine similarity of .

[0126] 4. The preset maximum window length N is generally an integer multiple of the maximum block length and an integer multiple of M.

[0127] 5. Place the window in the first position, that is, the 1st block to the N / Mth block, calculate the average similarity between each block in the window and the query_text, move the window back one block in turn, and calculate the average similarity of the new position again until it moves to the last block.

[0128] Define the window length N and calculate the average similarity of the blocks in each window:

[0129]

[0130] Where k represents the starting block index of the window.

[0131] 6. Draw a curve for all the similarity averages in 5 to obtain a curve graph showing the change of the similarity average as the window moves, and find the peak position of the graph.

[0132]

[0133] Represents the window similarity set obtained in the previous step. The processing method is as follows:

[0134] First, the block with the highest similarity to the target block is determined, and its similarity is used as a benchmark reference value.

[0135]

[0136] Next, set a similarity threshold. .

[0137] Based on this threshold, a new set C is established, which contains all blocks whose similarity with the target block exceeds this threshold.

[0138]

[0139] Then, by analyzing the positional relationship of these blocks in the original text, the blocks with a distance less than or equal to 3 between each other are grouped to form clusters.

[0140]

[0141] Finally, the cluster with the longest length is identified from all the formed clusters, which represents a peak in the graph.

[0142]

[0143] 7. If there is only one peak, the text in the window corresponding to the peak position is set as the denoised text.

[0144] 8. If there are multiple (X) peaks, split the original window into X windows of equal length, corresponding to the X peaks respectively, and remove the text in the sub-window for each peak.

[0145] 9. Concatenate the texts in all sub-windows in the order of the original text to obtain the denoised text under multiple peaks.

[0146] According to the peak position, select the text in the corresponding window as the denoised text. If there are multiple peaks, select the corresponding sub-windows and splice the text.

[0147]

[0148] here, It represents the final denoised text, which is composed of the text in the window corresponding to all detected peaks.

[0149] Step 3: Train the re-ranking model:

[0150] Use the general large language model as the base to train the re-ranking model.

[0151] 1. Dataset construction: Use positive and negative example pairs constructed based on a certain strategy, where positive examples represent relevant patents and negative examples represent irrelevant documents. Each positive example has several negative examples, which form a piece of training data according to a certain negative example ratio. Positive examples come from patent citation data, while negative examples come from the following mechanisms:

[0152] Negative examples based on BM25 recall: These are documents that are not positive examples in the Top N results recalled by the BM25 algorithm, introducing highly relevant negative examples.

[0153] Negative examples at different IPC levels: These negative examples come from patents that are similar to the IPC classification of the query text at different levels, such as the largest similarity level is a large group or a small group, which is intended to introduce similar fields but unrelated specific technical concepts.

[0154] Random negative examples: These negative examples are randomly selected from the entire dataset and have no obvious or direct relevance to the query text. Introducing random negative examples reduces the difficulty.

[0155] Negative sample ratio

[0156] Set the sampling ratio of three types of negative examples to r 1 , r 2 and r 3 , corresponding to the three types of negative examples mentioned above. The entire training data set consists of the following elements:

[0157]

[0158] in, is a positive example document related to the query text, is the negative example document r sampled according to different strategies 1 , r 2 and r 3 Specifies the sampling ratio of each type of negative examples.

[0159] 2. Construct the prompt words for the large language model, and use the specially constructed prompt words to stimulate the potential of the large language model so that the large language model can handle the task. Concatenate the prompt words, the query text, and the text of the file to be compared to obtain the final input to the large language model:

[0160] Concatenation of query text and prompt words:

[0161]

[0162] Input_query represents the sequence obtained by concatenating the query text query_text with specially constructed prompt words special tokens, which is used to input into the large language model.

[0163] The concatenation of the text of the file to be compared and the prompt words:

[0164]

[0165] Input_doc is a sequence of the denoised document text D' to be compared and special tokens with the same prompt word.

[0166] Final model input:

[0167]

[0168] Final input represents the final model input, which consists of the concatenated sequence of the query text enhanced by the prompt words and the text to be compared.

[0169] 3. Adjust the original model structure and add a customized vector layer after the model output layer to reduce the dimension of the model output from a two-dimensional matrix of (batchsize, vocabsize) to a one-dimensional vector of (batchsize). The customized vector layer contains trainable parameters.

[0170] The output layer of the model is followed by a trainable vector layer to convert the output from a two-dimensional matrix to a one-dimensional vector for more accurate similarity calculation.

[0171]

[0172] Where x is the original output of the model, It is a trainable parameter of the vector layer, f represents the conversion function of the vector layer, and v represents the one-dimensional vector output after conversion, which represents the distribution of the predicted word in the dictionary.

[0173] 4. Final input is input into the model, and the loss function is constructed using the predicted error, and then the model is trained by backpropagation.

[0174] Loss function: The loss function is constructed based on contrastive loss so that similar (positive) samples are closer to each other and dissimilar (negative) samples are farther away from each other, allowing the model to effectively distinguish between positive and negative samples.

[0175] Contrastive loss function:

[0176]

[0177] in, and are the vector representations of positive and negative examples in the same group, y i is the label (1 for positive examples and 0 for negative examples), d() represents the distance between the two vectors (usually Euclidean distance or cosine distance), and m is a margin that defines when the distance between negative examples is considered "far enough".

[0178] The constructed large language model is trained using the contrast loss and back propagation algorithm to obtain a trained large language model.

[0179] The application specifically puts the denoised text pair into the trained large language model (input query text q and file text to be compared d, and returns a similarity value.

[0180]

[0181] For each file to be compared retrieved in step 1, it is combined with q one by one and input into the trained large language model to obtain a similarity set. The similarities are sorted to obtain the initial recommendation sequence P.

[0182] Step 4: Fine-tune the initial recommendation list:

[0183] The purpose of fine-tuning the initial recommendation list is to advance the ranking of certain patents with significant potential characteristics, thereby improving the accuracy of the recommendation. The methods used are: weighting of highly relevant technical issues, and weighting of patents with the same main classification number at the same level.

[0184] 1. Use the "Technical Problem Extraction Model" to input the technical field, background technology, and invention content (first paragraph and last paragraph) of the present application and the patent of the document to be compared. The model will generate the "Technical Problem to be Solved" fields of the present application and the document to be compared respectively.

[0185]

[0186] is a model for extracting “technical problems to be solved” from relevant fields in patent document d. Represents the relevant fields (technical field, background technology and invention content) extracted from document d. It is the extracted text of the technical problem to be solved.

[0187] 2. The “technical problem to be solved” field of this application is encoded using a vector model.

[0188]

[0189] d app This is the patent application document. This is the "technical problem to be solved" field of this application. app It is the vector after encoding the “technical problem to be solved” field of this application.

[0190] 3. Use the vector model to encode the “technical issues to be resolved” field of each document to be compared in the recall list.

[0191]

[0192] d i It is a patent document to be compared. This is the "Technical Issues to be Solved" field of the document to be compared. i It is the vector after encoding the "Technical Issues to be Solved" field of the file to be compared.

[0193] 4. Calculate the similarity between the vector of each file to be compared and the vector of this application, and finally obtain a similarity list.

[0194]

[0195] Here, the cosine similarity function is used to measure the patent vector v app and the patent vector v to be compared i The similarity between .

[0196] 5. The similarity list is processed and scaled proportionally. The similarity value with the lowest similarity in the list is adjusted to 1, and the other similarity values ​​are scaled proportionally to obtain a scaled similarity list.

[0197]

[0198] Among them, S is the scaled similarity list, and P is the initial recommendation sequence. The similarity in the initial recommendation list and the scaled similarity of the vector are multiplied and sorted accordingly.

[0199]

[0200] Among them, Sort is a sorting function. The weighted recommendation sequence of "solving technical problems" is obtained.

[0201] 6. Obtain the ipc classification number of each file to be compared, divide it according to the ipc level (department, major category, minor category, major group, minor group), set five preset weighting values ​​a1, a2, a3, a4, a5, corresponding to the reward weighting of different ipc level matching, compare the ipc of each file to be compared in the list with the ipc of this application, including comparison within the pre-collected ipc category group (different ipc categories with strong correlation obtained through large data volume statistics, if the ipc category of the comparison file falls in the ipc similar group corresponding to this application, it is also considered a success), weight the weighted recommendation list of "solving technical problems" again according to the weighted value corresponding to the similar level, and obtain the "ipc classification number weighted" recommendation list.

[0202] Step Description:

[0203] Acquisition and segmentation of IPC classification numbers: The IPC classification numbers of each document to be compared and this application are acquired and segmented according to the level (section, major category, minor category, major group, minor group).

[0204] Setting of preset weighted values: Set weighted values ​​a1, a2, a3, a4, a5 according to the matching degree of IPC level. Each level corresponds to a weighted value.

[0205] IPC matching and weighting:

[0206] Compare the IPC of each document to be compared with the IPC of this application.

[0207] Appropriate weighting values ​​are applied based on the match level.

[0208] Weighted similarity calculation:

[0209] The previously calculated similarity is weighted again based on the IPC matching level to enhance the weight of patents that are highly relevant to the present application.

[0210]

[0211] Represents the weighted similarity of the i-th file to be compared. is the similarity of the i-th file calculated in the previous step. is a weighted value determined according to the matching level between the i-th document and the patent application in the IPC classification, where Indicates the matching level.

[0212] 7. Use a fully weighted list of recommendations Sorted into the final recommendation list.

[0213] In one embodiment, Figure 3As shown, an intelligent patent search system based on large language model re-ranking technology is provided, which specifically includes:

[0214] The retrieval recall module is used to perform semantic retrieval recall using query text, which is specifically divided into sparse vector retrieval based on the bag-of-words model and dense vector retrieval based on the vector model;

[0215] The denoising module is used to denoise each candidate patent using a block and sliding window strategy based on the retrieved sequence;

[0216] The module for training a re-ranking model is used to train a re-ranking model based on the denoising results using a general large language model as a base. First, a dataset containing positive and negative example pairs is constructed, where the positive examples come from patent citation data and the negative examples are obtained through different mechanisms. Then, a prompt word is constructed to stimulate the potential of the large language model, and the prompt word is concatenated with the query text and the text of the file to be compared to form the model input. A customized vector layer is connected after the model output layer for dimensionality reduction. Finally, the model is trained using a contrast loss function.

[0217] The fine-tuning module is used to input the denoised text pairs into the trained large language model to obtain a similarity set, sort the similarities to obtain an initial recommendation sequence, and fine-tune the initial recommendation list to obtain a final recommendation list; wherein the text pairs include the query text and the file text to be compared.

[0218] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. The processor of the computer device is used to provide computing and control capabilities, and the network interface is used to communicate with an external terminal through a network connection. The computer device implements the above-mentioned intelligent patent search method by loading and running a computer program.

[0219] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0220] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which involves all or part of the processes in the above-mentioned embodiment method.

[0221] In one embodiment, a computer program product is also provided, including a computer program / instruction, which involves all or part of the process in the above embodiment method.

[0222] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in M ​​forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (SyMchliMk) DRAM (SLDRAM), memory bus (RaMbus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0223] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. An intelligent patent search method based on large language model re-ranking technology, characterized in that: The method comprises: Retrieval and recall: Use query text for semantic retrieval and recall, which is divided into sparse vector retrieval based on bag-of-words model and dense vector retrieval based on vector model; Denoising: Based on the retrieved sequence, each candidate patent is denoised using a block and sliding window strategy; Training the re-ranking model: Based on the denoising results, the general large language model is used as the base to train the re-ranking model. First, a dataset containing positive and negative example pairs is constructed. The positive examples come from patent citation data, and the negative examples are obtained through different mechanisms. Then, prompt words are constructed to stimulate the potential of the large language model, and the prompt words are concatenated with the query text and the text of the file to be compared to form the model input. A customized vector layer is connected after the model output layer for dimensionality reduction. Finally, the model is trained using the contrast loss function. The denoised text pairs are input into the trained large language model to obtain a similarity set, the similarities are sorted to obtain an initial recommendation sequence, and the initial recommendation list is fine-tuned to obtain a final recommendation list; wherein the text pairs include the query text and the file text to be compared.

2. The method according to claim 1, characterized in that Before the step 1 of searching and recalling, it also includes obtaining multiple types of input data and performing data preprocessing and parsing, wherein the multiple types of input data include patent publication numbers, free texts, and text files; In the data preprocessing and parsing results, the corresponding text segment is selected as the query text according to the preset field parameters; if the input is a patent publication number, the independent claim is used by default; if the input is free text, the preprocessed content is the query text; if the input is a text file, the content after recognition and verification is used as the query text for subsequent vector encoding and similarity matching.

3. The method according to claim 2, characterized in that When the input data is a patent publication number, it specifically includes: searching and obtaining the corresponding patent document in the patent database through the patent publication number; parsing the obtained patent document using preset rules to extract key fields; wherein the key fields at least include the title, abstract, claims, description, inventor information and IPC classification number; splitting the claim text into items, and matching based on preset rules to distinguish independent claim groups from dependent claim groups, and constructing a relationship tree between them; subdividing the description content into technical field, background technology, invention content, illustrations and specific implementation methods; When the input data is free text, it specifically includes: checking punctuation and automatically repairing typos on the input free text to ensure that the text quality meets the needs of subsequent processing; When the input data is a text file, it specifically includes: selecting a corresponding parsing strategy according to the file type; wherein the file type includes at least TXT, Word document, text PDF or image PDF; for TXT, Word document or text PDF, directly performing text parsing and reading; if it is an image PDF file, calling OCR technology for text recognition; further performing punctuation checking and automatic repair of typos on the text obtained by OCR.

4. The method according to claim 1, characterized in that: In retrieval recall, query text is used for semantic retrieval recall, which is divided into sparse vector retrieval based on bag-of-words model and dense vector retrieval based on vector model, including: Sparse vectors are constructed using the TF-IDF algorithm and the BM25 algorithm to generate vectors for the title, abstract, claims, and specification for each patent; Use the BGE vector model for dense vector retrieval and optimize the contrast loss to improve the model's ability to distinguish different input texts. The two search results are fused through the min sum merge algorithm to generate a sequence containing patent publication numbers.

5. The method according to claim 1, characterized in that In denoising, based on the retrieved sequence, each candidate patent is denoised using a block and sliding window strategy, including: Process the specified paragraphs of the referenced document in blocks to ensure that the semantic integrity is not damaged; Use the vector model to vector encode each block and query text, and calculate the similarity. By presetting the maximum window length N, calculate the average similarity between the block and the query text in the window, draw a similarity change curve, and find the peak position.

6. The method according to claim 1, characterized in that Fine-tuning the initial recommendation list includes: Use the "technical problem extraction model" to extract the "technical problem to be solved" field of the present application and the document to be compared, and perform vector encoding; Calculate the vector similarity between the application and the document to be compared, and scale them up and down in proportion; Obtain the IPC classification number of each file to be compared and weight it according to the matching level; All weighted recommendation lists are used for sorting to obtain the final recommendation list.

7. An intelligent patent search system based on large language model re-ranking technology, characterized in that the system include: The retrieval recall module is used to perform semantic retrieval recall using query text, which is specifically divided into sparse vector retrieval based on the bag-of-words model and dense vector retrieval based on the vector model; The denoising module is used to denoise each candidate patent using a block and sliding window strategy based on the retrieved sequence; The module for training the re-ranking model is used to train the re-ranking model based on the denoising results using the general large language model as the base. First, a dataset containing positive and negative example pairs is constructed, where the positive examples come from patent citation data and the negative examples are obtained through different mechanisms. Then, a prompt word is constructed to stimulate the potential of the large language model, and the prompt word is concatenated with the query text and the text of the file to be compared to form the model input. A customized vector layer is connected after the model output layer for dimensionality reduction. Finally, the model is trained using the contrast loss function. The fine-tuning module is used to input the denoised text pairs into the trained large language model to obtain a similarity set, sort the similarities to obtain an initial recommendation sequence, and fine-tune the initial recommendation list to obtain a final recommendation list; wherein the text pairs include the query text and the file text to be compared.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images