Patent classification method and device driven by cooperation of multiple large models and cue words and storage medium

Abstract

The invention discloses a patent classification method and device driven by cooperation of multiple large models and cue words and a storage medium, and relates to the field of natural language processing and patent information management.The method comprises the steps that key information is extracted from a patent text, and the key information comprises the name of the invention, the abstract of the specification, the technical field, the background technology, the content of the invention and the specific implementation mode; a cue word template structure is designed and optimized, so that the large language model can understand and execute patent classification tasks; and constructing a multi-model cooperative classification workflow, and determining the international patent classification (IPC) number of the patent text through step-by-step classification and an iterative optimization strategy. Through verification of a plurality of groups of contrast experiments, the method has the advantages in the aspects of demand and training cost in patent classification in the small sample photovoltaic field, accurate classification of patent texts is realized, the classification efficiency is improved, the training cost of a patent text classification model is reduced, and a feasible method is provided for patent text classification.

Description

Technical Field

[0001] The present invention relates to a patent classification method, and in particular to a patent classification method, device and storage medium driven by multiple large models collaboration and prompt words, and belongs to the field of natural language processing and patent information management. Background Art

[0002] Patent classification plays a key role in intellectual property management. With the increase in the number of patents and the refinement of the classification system, traditional classification methods are difficult to meet existing needs. Existing patent classification methods rely on manual classification or small-scale machine learning models, and have problems such as dimensionality explosion, data sparsity, and limited generalization ability. With the increase in the number of patents, the research on automated patent classification methods is particularly important. In recent years, deep learning technology has been widely used in patent classification, but there are still problems with high training costs and scarce labeled data. To this end, researchers proposed a new paradigm called prompt learning, which effectively reduces the amount of training parameters by designing high-quality prompt words and using pre-trained language models to make accurate predictions with few samples. However, different prompt word construction methods will significantly affect the quality of prompt learning. Therefore, it is particularly critical to design high-quality prompt words for patent classification tasks. Summary of the invention

[0003] In view of this, the present invention aims to solve the deficiencies in the prior art and provide a patent classification method, device and storage medium driven by multiple large models and prompt words. By optimizing the prompt word structure and multi-model collaboration strategy, the accurate classification of patent texts can be achieved, the classification efficiency can be improved, and the training cost of the patent classification model can be reduced. The present invention proposes a patent classification method driven by multi-model collaboration and prompt words, and its innovations are reflected in the following points:

[0004] 1. Propose a prompt word structure suitable for patent text classification to guide the large language model to understand and perform patent classification tasks.

[0005] 2. A multi-model collaborative classification method is proposed to determine the International Patent Classification (IPC) number of the patent text through hierarchical classification, weight adjustment strategy and iterative optimization strategy.

[0006] 3. Construct a patent text classification dataset in the photovoltaic field to verify the effectiveness of the method in this paper.

[0007] The present invention proposes a patent classification method driven by multi-model collaboration and prompt words, comprising the following steps:

[0008] Step 1: Extract key information from the patent text, including the title of the invention, abstract of the specification, technical field, background technology, content of the invention and specific implementation methods;

[0009] Step 2: Propose a prompt word template structure for patent classification, so that the large language model can understand and perform patent classification tasks;

[0010] Step 3: A collaborative classification method of multiple large models is proposed to determine the International Patent Classification (IPC) number of the patent text through step-by-step classification and iterative optimization strategy.

[0011] Furthermore, the step 1 comprises:

[0012] Step 11: Obtain patent application documents, including claims, specification, specification abstract and drawings. The specification, as an important document of the patent application, elaborates on various aspects of the invention, including the name of the invention, technical field, background technology, content of the invention, description of drawings and specific implementation methods;

[0013] Step 12: Extract the core contents of the invention name, specification abstract, technical field, background technology, invention content and specific implementation methods, and construct a comprehensive description of the patent by extracting key elements from the patent text, so as to provide sufficient information support for the design of subsequent prompt words and the input preparation of the generative large language model.

[0014] Furthermore, the patent classification prompt word template in step 2 includes the following structure, and the patent classification prompt word template is composed of the following structures in order:

[0015] Structure 21: Define the role of the big model; the prompt word sets the role of the big model in the classification task, activates the knowledge domain related to the task, understands the information related to the specific task, and avoids generating outputs that are irrelevant to the task;

[0016] Structure 22: Define patent classification tasks; the prompt words define the patent classification tasks to be performed by the large model, so that the large model can focus on extracting and understanding patent classification tasks in specific fields during the classification process, thereby improving the accuracy of the classification results;

[0017] Structure 23: Propose an execution step suitable for patent classification tasks; guide the big model to process patent information step by step, reduce the ambiguity generated by the big model in processing complex tasks, help the big model locate the specific link where the problem occurs, and make the big model classification process controllable;

[0018] Structure 24: Propose an output format that matches the IPC classification number format; specify the output format of the large model, constrain the classification result format of the large model, limit the output length, data type, classification number structure, and separator, so that the output results are consistent;

[0019] Structure 25: Propose a reference rule for patent classification; add patent IPC classification rules suitable for the large model to perform classification tasks in the prompt words, so that the large model references the rules to perform patent classification steps, reduce the risk of misclassification, and reduce the occurrence of factual hallucinations of the large model;

[0020] Structure 26: Provide reference examples of patent classification; provide reliable example context as a benchmark for large model learning during the classification process of the large model, eliminate the ambiguity of task instructions, and improve the generalization ability of the large model when processing similar data;

[0021] Structure 27: patent information extracted in step 12;

[0022] Furthermore, the step 3 of the collaborative classification method of multiple large models includes the following steps:

[0023] Step 31: Set the number of large models in the classification working method based on the patent classification level, and configure n different large models; the characteristic is that the multi-large model strategy integrates the technologies and perspectives of different large models, combines the advantages of multiple large models in processing different types of information, and reduces the bias and limitations of a single large model through an integrated learning method. In the classification method, the large models are redundant with each other, thereby enhancing the robustness and accuracy of the patent classification system;

[0024] Step 32: Set the weight of large model i at the lth classification level Where i=1,2,…,n,1≤l≤5, default setting The method is characterized in that, at the beginning of the classification work method, an unbiased initial classification decision baseline is set, all models are given equal influence, and the weight of the large model is dynamically adjusted according to the classification result of the model;

[0025] Step 33: Set the initial threshold for the current level in the classification work method Where θ is the parameter for controlling the threshold, and θ=0.75 in this method. The characteristic is that the threshold is set to speed up the decision-making process and filter out the classification results with low scores and possible inaccuracy, so as to improve the overall classification efficiency and accuracy. By adjusting the size of the initial threshold by θ, the classification omission caused by too high a threshold is reduced, and the excessive bias towards the misclassification of some frequently appearing but non-target categories is prevented in the initial classification process;

[0026] Step 34: Set the maximum number of iterations k for each level classification max In this method, k max =5, characterized in that the classification working method can force a decision after reaching a certain number of iterations, preventing over-reliance on small changes in the model and avoiding excessive consumption of computing resources and time in difficult-to-determine classification situations, thereby maintaining the operating efficiency and stability of the entire system;

[0027] Step 35: In the iterative classification stage, in the first round of classification, the prompt words described in claim 3 are input into n large models respectively to obtain classification results; starting from the second round of classification, the classification results of the previous round of large models are incorporated into the prompt words of the current classification level, and each large model is ordered to re-evaluate and determine the category of the patent based on the aggregated classification information; the potential category C output in each round of classification is j Calculating weighted scores Where I(·) indicates the function. If the classification result of model i is C j , then I(C i =C j )=1, otherwise 0; if there is category C j The weighted score S j Reach or exceed the current threshold T k , that is, S j ≥T k , then determine the category C j is the final result, and the classification result C of the current level l is j Incorporate into the prompt words of the next level l+1 to provide a basis for further classification;

[0028] Step 36: Propose a threshold adjustment strategy. In the process of class determination at each level, if there is no weighted score S for any class j Meet the previous threshold T k , then according to T k+1 =T k +λ, where λ is the threshold adjustment coefficient, which is used to adjust the threshold T k Adjustment is made; thereafter, step 35 described in claim 4 is re-executed to continue the classification iteration; until at least one category C j The weighted score S j Reach or exceed the adjusted threshold T k , or the maximum number of iterations k for the classification level is reached max The final classification result of the current level is determined by the threshold adjustment strategy, which enables the classification method to effectively find the most suitable classification result even when the patent information is complex or the preliminary category determination is unclear, avoiding missing the correct classification result due to an excessively high threshold, and improving the adaptability and flexibility of the classification process;

[0029] Step 37: Propose a weight adjustment strategy. After determining the category of the current patent classification level l, if the output result of the large model i is inconsistent with the final classification result of the level l, adjust the weight of the large model i in the next classification level l+1. Where μ is the weight change coefficient; the characteristic is that the weight adjustment strategy specifically enhances or weakens the influence of a specific large model to optimize the performance of the entire classification system. Through the weight change coefficient μ, the weight of the large model with inconsistent classification results is dynamically adjusted, which directly affects the contribution of the large model in the subsequent classification level; if the classification result of the large model is consistent with the classification result of the current level, the weight may increase, otherwise it may decrease; the weight adjustment is not only based on the performance of a single level, but is continuously evaluated and adjusted throughout the entire classification process to ensure that the classification work method can adaptively optimize the response to erroneous judgments, thereby improving the classification accuracy and the overall robustness of the classification work method;

[0030] Step 38: Propose a hierarchical classification strategy to classify patent information according to the level of patent categories, starting from the highest section level, gradually classifying downward to the class level, and then to the subclass, main group, and subgroup levels. The characteristic is that this strategy enables the classification work method to conduct detailed analysis and classification of each level to ensure that patent information is accurately processed at each level. This hierarchical and step-by-step refinement method is used to process and analyze complex patent information, thereby improving the accuracy and efficiency of the overall classification of the classification work method.

[0031] The present invention also proposes an electronic device, comprising a processor and a memory, wherein the memory stores a program that can be run on the processor, and is characterized in that when the program is executed by the processor, the steps of the multi-model collaboration and prompt word driven patent classification method are implemented.

[0032] The present invention also proposes a computer storage medium, which stores at least one program, characterized in that the at least one program can be executed by at least one processor to implement the steps of the multi-model collaboration and prompt word driven patent classification method. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0034] Figure 1 , which is a schematic diagram of a patent classification method provided by the present invention that is driven by multiple large models collaborating and prompting words.

[0035] Figure 2, which is a multi-model collaborative strategy for patent classification driven by collaboration of multiple large models and prompt words provided by the present invention.

[0036] Figure 3 , which is a photovoltaic field patent text classification dataset provided by an embodiment of the present invention, is used to verify the effectiveness of a patent classification method driven by multi-model collaboration and prompt words. DETAILED DESCRIPTION

[0037] In order to further clarify the purpose, characteristics and advantages of the present invention, the present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0038] The following examples will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form.

[0039] It should be pointed out that, for those skilled in the art, the present invention can be implemented in many other ways different from those described herein without departing from the concept of the present invention, and those skilled in the art can make similar improvements without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific implementation disclosed below.

[0040] Example 1

[0041] See also Figure 1 This embodiment provides a patent classification method driven by multi-model collaboration and prompt words, including the following steps:

[0042] Step 1: Extract key information from the patent text, including the title of the invention, abstract of the specification, technical field, background technology, content of the invention and specific implementation methods;

[0043] Step 2: Design and optimize the prompt word template structure so that the large language model can understand and perform patent classification tasks;

[0044] Step 3: Construct a multi-model collaborative classification working method to determine the International Patent Classification (IPC) number of the patent text through step-by-step classification and iterative optimization strategy.

[0045] See also Figure 3 The photovoltaic patent text classification dataset provided in this embodiment is used to verify the effectiveness of a patent classification method driven by multi-model collaboration and prompt words, as follows:

[0046] The experimental data set comes from the Zhihuiya platform. As of May 2024, the keyword "photovoltaic power generation" is used for retrieval to screen out patents with a legal status of "authorized" and a simple legal status of "valid";

[0047] The selected patent types include "invention application", "utility model" and "authorized invention", and a total of 82,775 patents were retrieved;

[0048] The acquired patent text dataset contains the invention title, abstract, technical field, background technology and invention content, as well as the corresponding IPC classification number.

[0049] See also Figure 3 The distribution of each category in the experiment,5 groups of samples containing 100 patent texts were selected, and the effectiveness of the proposed method was evaluated by comparing the classification accuracy of the model classification working method with that of a single model;

[0050] The experiment interacts with the GPT-4, GLM-4 and Qwen-plus large language models through the API interface;

[0051] The experimental parameters are set as follows: GPT4 is selected as the single model, the number of models n = 3, the initial threshold parameter θ = 0.75, the classification level 1 ≤ l ≤ 3, and the maximum number of iterations k max =5, weight adjustment coefficient u = e -α , threshold adjustment coefficient λ = -δ×k, where α and δ are the attenuation coefficients that control the attenuation speed of the weight and threshold. The classification accuracy formula is:

[0052]

[0053] The experimental results are shown in Table 1. The multi-model collaboration and prompt word driven patent classification method proposed in the present invention is significantly better than the classification effect of a single model in all five groups of experiments. The classification accuracy of a single model is between 55% and 57%, while the accuracy of the method in this paper is between 72% and 73%. The experimental results show the effectiveness of the multi-model collaboration strategy in the patent text IPC classification task when processing small sample data sets.

[0054] Table 1 Comparison of classification effects between single model and multi-model collaborative strategy

[0055]

[0056] Ablation Experiment Study See Table 2. The effectiveness of our proposed method is verified through ablation study. This study compares three different configurations of the multi-model collaborative classification working method;

[0057] After excluding the weight adjustment step, the classification accuracy dropped significantly, with scores ranging from 65% to 66%. Dynamically adjusting model weights during the iteration process is critical to maintaining high classification accuracy;

[0058] The lack of an iterative process further degraded the performance, with the accuracy dropping to between 61% and 62%. Iterative optimization enabled the model to achieve more accurate and stable classification results;

[0059] The patent classification method proposed in this paper, which is driven by multi-model collaboration and prompt words, consistently achieved the highest classification accuracy in all five groups of experiments, ranging from 72% to 73%. By integrating the knowledge of pre-trained large language models, optimized prompt structures, and multi-model collaboration mechanisms, the IPC classification accuracy of patent texts was significantly improved.

[0060] Table 2 Classification effects of different multi-model collaborative strategy classification methods

[0061]

[0062] Example 2

[0063] This embodiment provides an electronic device, including a processor and a memory, wherein the memory stores a program that can be run on the above-mentioned processor, and when the program is executed by the above-mentioned processor, the steps of a patent classification method driven by multi-model collaboration and prompt words in the above-mentioned embodiment are implemented.

[0064] Example 3

[0065] This embodiment provides a computer-readable storage medium, which stores at least one program, and the at least one program can be executed by at least one processor to implement the steps of a patent classification method driven by multiple models and prompt words in the above embodiment. The above storage medium includes: a U disk, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk, etc., which can store program codes.

[0066] Obviously, the above embodiments are merely examples for clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from them are still within the protection scope of the invention.

Claims

1. A patent classification method driven by multiple large models and prompt words is characterized by: The following steps are involved: Step 1: Extract key information from the patent text, including the title of the invention, abstract of the specification, technical field, background technology, content of the invention and specific implementation methods; Step 2: Propose a prompt word template structure for patent classification, so that the large language model can understand and accurately perform patent classification tasks; Step 3: A collaborative classification method of multiple large models is proposed to determine the International Patent Classification (IPC) number of the patent text through hierarchical classification, weight adjustment strategy and iterative optimization strategy.

2. The method according to claim 1, characterized in that: The step 1 comprises the following steps: Step 11: Obtain patent application documents, including claims, description, abstract and drawings; Step 12: Extract the core content of the invention title, specification abstract, technical field, background technology, invention content and specific implementation methods.

3. The method according to claim 1, characterized in that Step 2 proposes a template structure for patent classification prompt words. The patent classification prompt word template is composed of the following structures in order: Structure 21: Define the role of the big model; the prompt word sets the role of the big model in the classification task, activates the knowledge domain related to the task, and understands the information related to the specific task; Structure 22: Define the classification task; the prompt word defines the patent classification task to be performed by the large model, so that the large model can focus on extracting and understanding the patent classification tasks in a specific field during the classification process, thereby improving the accuracy of the classification results; Structure 23: Propose an execution step suitable for patent classification tasks; guide the big model to process patent information step by step, reduce the ambiguity generated by the big model in processing complex tasks, help the big model locate the specific link where the problem occurs, and make the big model classification process controllable; Structure 24: Propose an output format that matches the IPC classification number format; specify the output format of the large model, constrain the classification result format of the large model, limit the output length, data type, classification number structure, and separator, so that the output results are consistent; Structure 25: Propose a reference rule for patent classification; add patent IPC classification rules suitable for the large model to perform classification tasks in the prompt words, so that the large model references the rules to perform patent classification steps, reduce the risk of misclassification, and reduce the occurrence of factual hallucinations of the large model; Structure 26: Provide reference examples of patent classification; provide reliable example context as a benchmark for large model learning during the classification process of the large model, eliminate the ambiguity of task instructions, and improve the generalization ability of the large model when processing similar data; Structure 27: Patent information extracted from claim 2.

4. The method according to claim 1, characterized in that: The step 3, a method for collaborative classification of multiple large models, comprises the following steps: Step 31: Set the number of large models in the classification working method based on the level of patent classification, and configure n different large models; the characteristic is that the multi-large model strategy integrates the technologies and perspectives of different large models, combines the advantages of multiple large models in processing different types of information, and reduces the bias and limitations of a single large model through an integrated learning method. In the classification working method, the large models are redundant with each other, thereby enhancing the robustness and accuracy of the patent classification system; Step 32: Set the weight of large model i at the lth classification level Where i=1,2,…,n,1≤l≤5, default setting The method is characterized in that, at the beginning of the classification work, an unbiased initial classification decision baseline is set, all models are given equal influence, and the weight of the large model is dynamically adjusted according to the classification results of the model; Step 33: Set the initial threshold for the current level in the classification work method Among them, θ is the parameter for controlling the threshold; the characteristic is that the threshold is set to speed up the decision-making process and filter out the classification results with low scores and possible inaccuracy, so as to improve the overall classification efficiency and accuracy. By adjusting the size of the initial threshold by θ, the classification omission caused by too high a threshold is reduced, and the excessive bias towards the misclassification of some frequently appearing but non-target categories is prevented in the initial classification process; Step 34: Set the maximum number of iterations k for each level classification max In this method, k max =5, characterized in that the classification working method can force a decision after reaching a certain number of iterations, preventing over-reliance on small changes in the model and avoiding excessive consumption of computing resources and time in difficult-to-determine classification situations, thereby maintaining the operating efficiency and stability of the entire system; Step 35: In the iterative classification stage, in the first round of classification, the prompt words described in claim 3 are input into n large models respectively to obtain classification results; starting from the second round of classification, the classification results of the previous round of large models are incorporated into the prompt words of the current classification level, and each large model is ordered to re-evaluate and determine the category of the patent based on the aggregated classification information; the potential category C output in each round of classification is j Calculating weighted scores Where I(·) indicates the function. If the classification result of model i is C j , then I(C i =C j )=1, otherwise 0; if there is category C j The weighted score S j Reach or exceed the current threshold T k , that is, S j ≥T k , then determine the category C j is the final result, and the classification result C of the current level l is j Incorporate into the prompt words of the next level l+1 to provide a basis for further classification; Step 36: Propose a threshold adjustment strategy. In the process of class determination at each level, if there is no weighted score S for any class j Meet the previous threshold T k , then according to T k+1 =T k +λ, where λ = -δ×k is the threshold adjustment coefficient, for the threshold T k Adjustment is made; thereafter, step 35 described in claim 4 is re-executed to continue the classification iteration; until at least one category C j The weighted score S j Reach or exceed the adjusted threshold T k , or the maximum number of iterations k for the classification level is reached max The final classification result of the current level is determined by the threshold adjustment strategy, which enables the classification method to effectively find the most suitable classification result even when the patent information is complex or the preliminary category determination is unclear, avoiding missing the correct classification result due to an excessively high threshold, and improving the adaptability and flexibility of the classification process; Step 37: Propose a weight adjustment strategy. After determining the category of the current patent classification level l, if the output result of the large model i is inconsistent with the final classification result of the level l, adjust the weight of the large model i in the next classification level l+1. where u=e -αk is the weight change coefficient; its characteristic is that the weight adjustment strategy specifically enhances or weakens the influence of a specific large model to optimize the performance of the entire classification system. Through the weight change coefficient μ, the weight of the large model with inconsistent classification results is dynamically adjusted, which directly affects the contribution of the large model in the subsequent classification level; if the classification result of the large model is consistent with the classification result of the current level, the weight may increase, otherwise it may decrease; the weight adjustment is not only based on the performance of a single level, but is continuously evaluated and adjusted throughout the entire classification process to ensure that the classification work method can adaptively optimize the response to erroneous judgments, thereby improving the classification accuracy and the overall robustness of the classification work method; Step 38: Propose a hierarchical classification strategy to classify patent information according to the level of patent categories, starting from the highest section level, gradually classifying downward to the class level, and then to the subclass, main group, and subgroup levels. The characteristic is that this strategy enables the classification work method to conduct detailed analysis and classification of each level to ensure that patent information is accurately processed at each level. This hierarchical and step-by-step refinement method is used to process and analyze complex patent information, thereby improving the accuracy and efficiency of the overall classification of the classification work method.

5. The method according to claim 1, characterized in that The method shows high classification accuracy and low training cost when processing small sample data sets.

6. An electronic device comprising a processor and a memory, wherein the memory stores a program that can be run on the processor, characterized in that When the program is executed by the processor, the steps of a patent classification method driven by collaboration of multiple large models and prompt words as described in any one of claims 1-5 are implemented.

7. A computer storage medium storing at least one program, characterized in that: The at least one program can be executed by at least one processor to implement the steps of a patent classification method driven by collaboration of multiple large models and prompt words as described in any one of claims 1-5.

Images