Method and device for acquiring and evaluating high-value patent division threshold

Abstract

The invention provides a method for acquiring a high-value patent division threshold, which comprises the following steps of: acquiring a patent sample set, and acquiring evaluation dimensions and evaluation index data of each patent sample according to a patent value evaluation hierarchical structure model; obtaining the comprehensive value of each patent sample according to the high-value patent evaluation model, and selecting the minimum and maximum comprehensive values as an initial comprehensive threshold range; traversing each comprehensive value in the initial comprehensive threshold range, obtaining the prediction accuracy of the machine learning model under each comprehensive value, and constructing an accuracy curve; obtaining the slope of the accuracy curve by adopting a central difference method, and taking the comprehensive value corresponding to the slope stable oscillation interval as a second comprehensive threshold range; fitting the comprehensive value in the second comprehensive threshold range with the corresponding prediction accuracy to obtain a fitting equation, and substituting the maximum prediction accuracy on the accuracy curve into the fitting equation to obtain a comprehensive value threshold; the invention further provides a high-value patent evaluation method and device.

Description

Technical Field

[0001] The present invention relates to the field of high-value patent evaluation, and in particular to a method and device for acquiring and evaluating a high-value patent classification threshold. Background Art

[0002] Cultivating and encouraging high-value patents is of great significance for promoting innovation and progress and supporting and leading the industry's high-quality and sustainable development.

[0003] Based on the clarification of the multi-dimensional attributes of high-value patents, domestic and foreign scholars have built a patent value assessment model to identify and predict high-value patents, and then conduct research on the corresponding cutting-edge hotspots containing such patents. However, high-value patents involve many dimensions, and the content of patents in different fields is different. There is no unified standard for how to define and evaluate patent value. For example, the tobacco industry in my country has a considerable number of patents, but there are still obvious deficiencies in the evaluation of high-value patents, which is not conducive to scientific and technological management and the transformation of scientific and technological achievements.

[0004] The existing patent evaluation methods include expert evaluation method, financial value evaluation method, five high-value invention patent identification standard screening methods, artificial intelligence model prediction method, quantitative analysis method, qualitative and quantitative combined analysis method, etc. Among them, the expert evaluation method and the financial value evaluation method are highly respected for their high accuracy, but the process is complicated, time-consuming and costly. As for the five high-value invention patent identification standard screening methods, the evaluation criteria are too broad to meet actual needs. For example, the tobacco industry has a considerable number of patents in my country. According to the five high-value invention patent identification standard screening methods, more than one-third of the effective invention patents in the tobacco industry will meet the high-value patent standards. Although the artificial intelligence model prediction method has a high accuracy rate and mostly uses neural network and machine learning technologies, the black box characteristics of the neural network model limit the interpretability of its results, and the sample size of high-value patents in the tobacco industry is insufficient, which makes it difficult to meet the training needs of deep learning models; the advantages of the quantitative analysis method are fast and convenient. Although it cannot be qualitatively evaluated, it can provide an objective allocation mechanism for multi-indicator comprehensive evaluation. Among them, the entropy weight method determines the weight of each indicator by calculating the degree of discreteness of the information entropy evaluation index, and uses the weighted average method to calculate the comprehensive value of the patent, which not only improves the efficiency of the evaluation, but also ensures the scientificity and rationality of the evaluation results. However, the quantitative analysis method relies on the setting of the score threshold. Currently, the setting of the score threshold is mostly empirical value, which lacks scientificity; for example, patent CN202410659695.8 discloses the use of statistical analysis and machine learning technology, combined with the distribution of patent scores, to select a reasonable score as the score threshold for high-value patents. It only gives a general direction for obtaining the threshold, and does not give a specific technical solution.

[0005] Therefore, a high-value patent evaluation system was constructed and empirical research was conducted, aiming to provide a feasible idea and easy-to-operate method for rapid batch evaluation of high-value patents, in order to make up for the shortcomings of research in this area.

[0006] In order to solve the above problems, people have been seeking an ideal technical solution. Summary of the invention

[0007] The purpose of the present invention is to address the deficiencies of the prior art and thus provide a method and device for obtaining and evaluating a high-value patent classification threshold, construct a high-value patent evaluation model based on the entropy weight method and the hierarchical analysis method, and scientifically obtain the classification threshold based on the high-value patent evaluation model, machine learning and the central difference method; and after obtaining the comprehensive value of the patent to be evaluated, the value of the patent to be evaluated is judged based on the obtained classification threshold and the comprehensive value, which is scientific and reasonable.

[0008] In order to achieve the above object, the technical solution adopted by the present invention is: A first aspect provides a method for obtaining a high-value patent classification threshold, comprising the following steps: Obtain a patent sample set, and obtain the evaluation dimension and corresponding evaluation index data of each patent sample in the patent sample set according to a pre-built patent value evaluation hierarchical structure model; Obtaining the comprehensive value of each patent sample according to the evaluation dimension, the evaluation index and the high-value patent evaluation model, and selecting the minimum comprehensive value and the maximum comprehensive value corresponding to the patent sample set as the initial comprehensive threshold range; Traverse each comprehensive value within the initial comprehensive threshold range in the set order, obtain the prediction accuracy of the machine learning model under each comprehensive value, and construct an accuracy curve; The slope of the accuracy curve is obtained by using the central difference method, and the comprehensive value corresponding to the stable oscillation interval of the slope is used as the second comprehensive threshold range; The comprehensive value within the second comprehensive threshold range is fitted with the corresponding prediction accuracy to obtain a fitting equation, and the maximum prediction accuracy on the accuracy curve is substituted into the fitting equation to obtain the comprehensive value threshold.

[0009] Further, the steps of traversing each comprehensive value within the initial comprehensive threshold range in a set order and obtaining the prediction accuracy of the machine learning model under each comprehensive value include: Determine the initial comprehensive threshold range and set the initial value of the maximum prediction accuracy to 0; Traverse each comprehensive value within the initial comprehensive threshold range in a set order, use each comprehensive value as a patent threshold, and generate a value label for each patent sample in the patent sample set according to the threshold, wherein the value label is a high-value patent or a common patent; Divide the patent sample set into a training set and a test set according to the generated value labels; Train the machine learning model based on the training set, and use the test set to test the prediction accuracy of the machine learning model; When the prediction accuracy is greater than the maximum prediction accuracy, the prediction accuracy is assigned to the maximum prediction accuracy, and then the next comprehensive value is traversed; otherwise, the next comprehensive value is traversed directly; Until the traversal ends.

[0010] Furthermore, after obtaining the prediction accuracy of the machine learning model under each comprehensive value, the prediction accuracy change difference corresponding to any comprehensive value and its adjacent previous comprehensive value is calculated, and the comprehensive value at which the prediction accuracy change difference is greater than the set change difference for the first time is selected as the limit point I, and the comprehensive value at which the prediction accuracy change difference is greater than the set change difference for the last time is selected as the limit point II; The central difference method is used to obtain the slope of the accuracy curve between the limiting point I and the limiting point II.

[0011] Furthermore, the patent value evaluation hierarchical structure model includes two layers of structure. The first layer of structure includes at least four evaluation dimensions, namely, strategic value dimension, technical value dimension, legal value dimension and economic value dimension; the second layer of structure includes multiple evaluation indicators corresponding to each evaluation dimension; Among them, the evaluation indicators corresponding to the strategic value dimension include patent awards and the technical field to which they belong; The evaluation indicators corresponding to the technical value dimension include patent type, number of patent citations, IPC classification span, technical integration, number of cooperative applicants, number of pages in the specification, and number of embodiments; The evaluation indicators corresponding to the legal value dimension include the number of authorized independent claims, the number of authorized claims, the remaining validity period of the patent, and the confirmation of rights after reexamination; The evaluation indicators corresponding to the economic value dimension include the number of times the patent is licensed, the number of times the patent is transferred, the number of times the patent is pledged, the number of patent families, and the number of countries in the patent family.

[0012] The second aspect provides a high-value patent evaluation method, comprising the following steps: According to the pre-constructed patent value evaluation hierarchical structure model, multiple first-level evaluation dimensions of the patent to be evaluated and multiple evaluation indicators corresponding to each evaluation dimension are obtained; the evaluation dimensions at least include strategic value dimension, technical value dimension, legal value dimension and economic value dimension; Obtaining the comprehensive patent value of the patent to be evaluated according to the evaluation dimensions, the evaluation indicators and the high-value patent evaluation model; wherein the high-value patent evaluation model is constructed based on the entropy weight method and the hierarchical analysis method; The evaluation result of the patent is obtained according to the comprehensive value of the patent to be evaluated and the comprehensive value threshold, wherein the comprehensive value threshold is obtained by using the method for obtaining the high-value patent classification threshold described in the first aspect.

[0013] The third aspect provides a device for obtaining a high-value patent classification threshold, including: Patent sample collection; A dimension and evaluation index acquisition module is used to obtain multiple first-level evaluation dimensions and multiple evaluation indicators corresponding to each evaluation dimension for each patent sample in the patent sample set according to a pre-built patent value evaluation hierarchical structure model; the evaluation dimensions at least include strategic value dimension, technical value dimension, legal value dimension and economic value dimension; High-value patent evaluation model construction module, used to construct a high-value patent evaluation model based on entropy weight method and hierarchical analysis method; An initial comprehensive threshold range acquisition module obtains the comprehensive value of each patent sample according to the evaluation dimension, the evaluation index and the high-value patent evaluation model, and selects the minimum comprehensive value and the maximum comprehensive value corresponding to the patent sample set as the initial comprehensive threshold range; The prediction accuracy acquisition module traverses each comprehensive value of the initial comprehensive threshold range in a set order, obtains the prediction accuracy of the machine learning model under each comprehensive value, and constructs an accuracy curve; The second comprehensive threshold range acquisition module uses the central difference method to obtain the slope of the accuracy curve, and takes the comprehensive value corresponding to the stable oscillation interval of the slope as the second comprehensive threshold range; A fitting equation acquisition module is used to fit the comprehensive value of the second comprehensive threshold range with the corresponding prediction accuracy to obtain a fitting equation; The threshold calculation module is used to bring the maximum prediction accuracy on the accuracy curve into the fitting equation to obtain the division threshold.

[0014] A fourth aspect provides a high-value patent evaluation device, comprising: A dimension and evaluation index acquisition module, used to obtain multiple primary evaluation dimensions of the patent to be evaluated and multiple evaluation indicators corresponding to each evaluation dimension according to a pre-built patent value evaluation hierarchical structure model; the evaluation dimensions at least include strategic value dimension, technical value dimension, legal value dimension and economic value dimension; High-value patent evaluation model construction module, used to construct a high-value patent evaluation model based on entropy weight method and hierarchical analysis method; A comprehensive value threshold acquisition module, used to acquire a comprehensive value threshold based on the method for acquiring the high-value patent classification threshold described in the first aspect; A comprehensive value acquisition module, used to obtain the comprehensive patent value of the patent to be evaluated according to the evaluation dimensions, the evaluation indicators and the high-value patent evaluation model; The value assessment module is used to obtain the patent evaluation result based on the comprehensive value of the patent to be evaluated and the comprehensive value threshold.

[0015] A fifth aspect provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the method described in the first aspect are implemented, or the steps of the method described in the second aspect are implemented.

[0016] The sixth aspect provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method described in the first aspect are implemented, or the steps of the method described in the second aspect are implemented.

[0017] The seventh aspect provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the method described in the first aspect, or implements the steps of the method described in the second aspect.

[0018] The present invention has outstanding substantive features and significant progress compared to the prior art. Specifically, (1) The present invention constructs a high-value patent evaluation system for my country's tobacco industry consisting of strategic value, technical value, legal value, and economic value, and determines a method for calculating the comprehensive value of patents using the entropy weight method.

[0019] (2) The present invention determines the weights of evaluation indicators through entropy weight method and hierarchical analysis method, and constructs a high-value patent evaluation model to achieve the objectivity and scientificity of the evaluation process; (3) The present invention sets different thresholds to divide different numbers of high-value patents and common patents as sample sets, tests the accuracy of the sample sets through a machine learning model, and uses the central difference method to determine the comprehensive threshold of high-value patents, thereby achieving the scientificity and rationality of threshold setting.

[0020] (4) After obtaining the comprehensive value of the patent to be evaluated, the present invention conducts value judgment on the patent to be evaluated based on the obtained segmentation threshold and comprehensive value. It has been verified that the high-value patent data set screened by the high-value patent evaluation system is highly consistent with the results obtained by the five high-value invention patent recognition standards and the award-winning invention patents in the tobacco industry, and has high scientificity and rationality. In the high-value patent data set, the average value of technical value is the highest, the average value of economic value is the lowest, and the average values ​​of the four dimensional values ​​and comprehensive value are significantly better than those of the entire patent sample set. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1It is a flowchart of the method for obtaining the high-value patent classification threshold described in Example 1 of the present invention.

[0022] Figure 2 It is a schematic diagram of the construction process of the high-value patent evaluation model in Example 1 of the present invention.

[0023] Figure 3 It is a schematic diagram of the process of obtaining the prediction accuracy of the machine learning model under various comprehensive values ​​in Example 1 of the present invention.

[0024] Figure 4 It is a partial schematic diagram of the SVM prediction accuracy curve obtained in Example 1 of the present invention.

[0025] Figure 5 It is a partial schematic diagram of the accuracy slope curve obtained in Example 1 of the present invention.

[0026] Figure 6 is the fitting result obtained in Example 1 of the present invention.

[0027] Figure 7 It is a flowchart of the method for obtaining the high-value patent classification threshold described in Example 2 of the present invention.

[0028] Figure 8 It is a flow chart of the high-value patent evaluation method in Example 3 of the present invention.

[0029] Fig. 9 A structural block diagram of a device for acquiring a high-value patent classification threshold in Example 4 of the present invention.

[0030] Fig.10 A structural block diagram of the high-value patent evaluation device in Example 5 of the present invention.

[0031] Fig.11 A structural block diagram of a computer device in Embodiment 6 of the present invention. DETAILED DESCRIPTION

[0032] The technical solution of the present invention is further described in detail below through specific implementation methods.

[0033] Example 1 This embodiment provides a method for obtaining a high-value patent classification threshold, such as Figure 1 As shown, the following steps are included: Step 1: Obtain a patent sample set, and obtain the evaluation dimension and corresponding evaluation index data of each patent sample in the patent sample set according to a pre-built patent value evaluation hierarchical model.

[0034] Among them, the patent sample set mainly comes from the patent search system or from the intellectual property big data comprehensive service system of a certain industry.

[0035] Taking the classification of high-value patents in the tobacco industry as an example, the patent sample set mainly comes from the tobacco industry intellectual property big data comprehensive service system. From 1985 to 2023, my country's tobacco industry has 47,052 valid patents. Among them, there are 11,677 valid invention patents, 31,338 utility model patents, and 4,037 design patents. Patent record information includes application number, title, application date, publication date, number of citations and patents in the same family. The evaluation scope of high-value patents is limited to the valid invention patents and utility model patents owned by my country's tobacco industry as of December 31, 2023, which are used as the entire patent sample set, excluding design patents.

[0036] Among them, the construction of the patent value evaluation hierarchical model adopts the comprehensive use of questionnaire surveys and expert argumentation methods to study and determine the evaluation dimensions and related indicators of the high-value patent evaluation system.

[0037] At present, the selection of patent evaluation indicators is mainly based on the four dimensions of economic value, market value, legal value and technical value, and strategic value is rarely used as a dimension for evaluating patent value. my country's tobacco industry has successively formulated a number of scientific and technological development plans. Patents that meet the requirements of the plans should have a high strategic value. Therefore, it is necessary to use strategic value as one of the dimensions of patent evaluation. In addition, technical value is based on the perspective of technological innovation and importance, and legal value is based on the scope of patent protection and the stability of the law. Both are necessary evaluation dimensions. Economic value and market value are based on the economic input and market output of the applicant to protect the patent technology, respectively. They are important components of reflecting economic benefits, so they are merged into economic value.

[0038] As shown in Table 1, a hierarchical model for the value evaluation of high-value patents in the tobacco industry was constructed from four evaluation dimensions, and the specific evaluation indicators of each dimension were determined based on the results of the "High-value Patent Evaluation Survey in the Tobacco Industry" conducted in 2023 for various directly affiliated units of the tobacco industry and the results of expert discussions.

[0039] Table 1 Evaluation hierarchical model of high-value patents in the tobacco industry Specifically, the evaluation indicators corresponding to the strategic value dimension include patent awards and the technical fields to which they belong; The evaluation indicators corresponding to the technical value dimension include patent type, number of patent citations, IPC classification span, technical integration, number of cooperative applicants, number of pages in the specification, and number of embodiments; The evaluation indicators corresponding to the legal value dimension include the number of authorized independent claims, the number of authorized claims, the remaining validity period of the patent, and the confirmation of rights after reexamination; The evaluation indicators corresponding to the economic value dimension include the number of times the patent is licensed, the number of times the patent is transferred, the number of times the patent is pledged, the number of patent families, and the number of countries in the patent family.

[0040] Furthermore, the evaluation indicators are divided into continuous variables and categorical variables. Except for the patent award (awarded as 1, not awarded as 0), the technical field (traditional tobacco field as 2, new tobacco field as 1), and confirmation of rights after reexamination (yes as 1, no as 0), the rest are continuous variables. Among them, the continuous variables are normalized for dimensionless processing, and the categorical variables are normalized and positively processed by taking the logarithm. The higher the value, the higher the patent value.

[0041] Step 2: Obtain the comprehensive value of each patent sample according to the evaluation dimension, the evaluation index and the high-value patent evaluation model, and select the minimum comprehensive value and the maximum comprehensive value corresponding to the patent sample set as the initial comprehensive threshold range.

[0042] Among them, the high-value patent evaluation model is constructed based on the entropy weight method and the hierarchical analysis method, such as Figure 2 As shown, the specific construction steps are as follows: Step 2.1, after obtaining the evaluation dimension and corresponding evaluation index data of each patent sample in the patent sample set, the weight of each evaluation index is calculated based on the entropy weight method and the hierarchical analysis method, where the sum of the weights of the evaluation index corresponding to each evaluation dimension is 1; Still taking the aforementioned tobacco industry patent sample set as an example, data is extracted from all patent samples according to the evaluation indicators in Table 1, and the entropy weight method is used to process the data to calculate the information entropy value and weight of specific indicators in each dimension. The results are shown in Table 2.

[0043] Table 2 Calculation results of value indicators As shown in Table 2, the maximum weight of the strategic value index is the patent award (98.401%), and the minimum is the technical field (1.599%). The difference between the two weights is large, mainly due to the small number of industry patent awards. So far, there are 41 industry award-winning patents, accounting for only 0.1% of the total sample, resulting in a large difference in the weights of the two indicators. The maximum weight of the technical value index is the number of cooperative applications (24.689%), and the minimum is the number of pages in the specification (1.444%). The number of patent citations (23.239%) is second only to the weight of the number of cooperative applications, indicating that the number of cooperative applications and the number of patent citations are more important to the technical value, and the number of pages in the specification is less important to the technical value. The maximum weight of the legal value index is the confirmation of rights after reexamination (69.607%), and the minimum is the number of authorized claims (1.547%), indicating that the importance of confirmation of rights after reexamination is the highest, which is closely related to its low proportion. The maximum weight of the economic value index is the number of countries in the patent family (25.455%), and the minimum is the number of countries in the patent family (7.597%). The number of times a patent is licensed and pledged is slightly lower than the weight of the number of countries in the patent family, indicating that the more countries a patent is deployed in overseas, the more times it is licensed and pledged, the higher its economic value. Among them, the number of countries in the patent family has the highest level of importance.

[0044] Step 2.2, obtain the weighted value evaluation model of each evaluation dimension based on the weight of each evaluation indicator and the dimension value calculation formula.

[0045] The dimension value calculation formula is: (1) In the formula, V ji is the jth patent in the patent sample set. i The value of a single patent in each evaluation dimension ( i =1,2,3,4), V j1 For strategic value, V j2 For technical value, V j3 For legal value, V j4 For economic value; m Indicates the number of evaluation indicators corresponding to the evaluation dimension; a jk For the k The normalized value of the evaluation index; w k express a jk The corresponding weight.

[0046] Among them k The normalized value of the evaluation indexa k The acquisition method can adopt data normalization methods such as min-max normalization method, Z-score normalization method, standard deviation normalization method, etc.

[0047] Taking the number of embodiments of the technical value dimension as an example, the embodiment data values ​​in the patent sample set include (1, 2, 3, 4, 5, 10). The maximum and minimum values ​​of the embodiment data values ​​are obtained, and the minimum value is 1 and the maximum value is 10. The min-max normalization method is used to normalize the data with an embodiment data value of 4, and the result is: ;in, x jk represents the original value of the j-th patent in the patent sample set on the k-th evaluation index; max( x k ) and min( x k ) represent the minimum and maximum values ​​of the patent sample set on the kth evaluation index respectively.

[0048] The reason why the data of each evaluation indicator needs to be normalized and standardized is that if data of different dimensions are used directly for comparison or modeling, the model will be biased towards indicators with a larger numerical range, thus affecting the accuracy and fairness of the model.

[0049] All evaluation indicators are standardized using the above calculation expressions, and the data of all indicators are converted to the same scale range (0-1), eliminating dimensional differences and making different indicators comparable. This not only improves the comparability and consistency of the data, but also enhances the accuracy and stability of subsequent analysis and model training. Through standardization, it can ensure that the contribution of each indicator in the model is balanced, and there will be no deviation due to differences in numerical ranges.

[0050] It should be noted that after the data of each evaluation indicator is normalized, the standardized data needs to be further checked to ensure that all data are between 0 and 1 and the conversion is correct.

[0051] Specifically, according to the weights of the value indicators in Table 2, the weighted value evaluation model of each evaluation dimension is obtained: V 1=0.984 a 专利获奖情况 +0.016 a 所属技术领域； (2) V 2=0.158 a 专利类型 +0.232 a专利被引用数 +0.208 a IPC分类跨度 +0.087 a 技术融合度 +0.247 a 合作申请数 +0.014 a 说明书页数 +0.053 a 实施例数 (3) V 3=0.272 a 授权独立权利要求数 +0.015 a 授权权利要求数 +0.017 a 专利剩余有效期 +0.696 a 复审后 (4) V 4=0.251 a 专利被许可次数 +0.176 a 专利被转让次数 +0.243 a 专利被质押次数 +0.076 a 专利同族数 +0.255 a 专利同族国家数 (5) Step 2.3, take the average of the weighted value evaluation models of all evaluation dimensions to obtain the high-value patent evaluation model: (6) Where: V The comprehensive value of the patent.

[0052] The value of each dimension of each patent in the patent sample set is calculated according to formulas (2)-(5), and the comprehensive value of each patent is obtained according to formula (6).

[0053] The descriptive statistics of the calculation results of the value indicators of each dimension are shown in Table 3.

[0054] Table 3 Descriptive statistics of the calculation results of value indicators of each dimension As can be seen from Table 3, the maximum and minimum values ​​of strategic value differ greatly, while the average value is small, indicating that there are large differences in the patent value assessment under this dimension and that the strategic value of most patents is not high. The maximum value of technical value is 0.4725, the average value is only 0.0625, and the standard deviation is 0.0815, indicating that the overall technical level is uneven, and the technical content of most patents is evaluated low. The average value of legal value is 0.0115, and the standard deviation is 0.0250. The distribution is relatively concentrated and the fluctuation is small, indicating that the value assessment of most patents in terms of legal protection is relatively close. Among the four dimensions, the economic value has the smallest average value, and the maximum value is only 0.3646, with a standard deviation of 0.0138, indicating that the direct economic benefits of patents are generally not high, and there are certain differences between different patents.

[0055] The minimum comprehensive value is 0.0004, and the maximum is 0.3699, that is, the initial comprehensive threshold range of the patent is [0.0004, 0.3699]. This shows that after comprehensive consideration of various factors, the span of patent value is large, with both lower-value patents and higher-value innovations.

[0056] Step 3: traverse each comprehensive value of the initial comprehensive threshold range in the set order, obtain the prediction accuracy of the machine learning model under each comprehensive value, and construct an accuracy curve.

[0057] The machine learning model adopts one of the algorithm models such as support vector machine SVM (machine learning binary classification model), naive Bayes, decision tree, random forest, K-nearest neighbor algorithm, neural network, etc.

[0058] like Figure 3 As shown, if the SVM model is selected, the steps of traversing each comprehensive value of the initial comprehensive threshold range in a set order and obtaining the prediction accuracy of the SVM model under each comprehensive value include: Determine the initial comprehensive threshold range and set the initial value of the maximum prediction accuracy to 0; Traverse each comprehensive value of the initial comprehensive threshold range in a set order, use each comprehensive value as a patent threshold, and generate a value label for each patent sample in the patent sample set according to the patent threshold, wherein the value label is a high-value patent or a common patent; wherein the traversal compensation can be set to 0.001; Among them, when the comprehensive value of the patent sample is greater than or equal to the patent threshold, a high-value patent label is generated for the patent sample, otherwise a common patent label is generated for the patent sample; Divide the patent sample set into a training set and a test set according to the generated value labels; The support vector machine (SVM) model is trained based on the training set, and the prediction accuracy of the support vector machine (SVM) model is tested using the test set; When the prediction accuracy is greater than the maximum prediction accuracy, the prediction accuracy is assigned to the maximum prediction accuracy, and then the next comprehensive value is traversed; otherwise, the next comprehensive value is traversed directly; Until the traversal ends.

[0059] According to the SVM prediction accuracy under different thresholds corresponding to the initial high-value patent threshold range, the SVM prediction accuracy curve is drawn, such as Figure 4 As shown, Figure 4 It is only the SVM prediction accuracy corresponding to some thresholds. Figure 4 It can be seen that the prediction accuracy of SVM continues to improve and approaches 100% as the patent value threshold increases.

[0060] Step 4: Use the central difference method to obtain the slope of the accuracy curve, and use the comprehensive value corresponding to the stable oscillation interval of the slope as the second comprehensive threshold range.

[0061] Specifically, on the SVM prediction accuracy curve corresponding to the initial high-value patent threshold range, the slope is calculated for every threshold point to obtain the prediction accuracy slope of the SVM at each threshold, and the accuracy slope curve is drawn according to the SVM prediction accuracy slopes at different thresholds corresponding to the initial high-value patent threshold range. Figure 5 It is a partial schematic diagram of the accuracy slope curve.

[0062] Depend on Figure 5 It can be seen that the slope of the patent threshold prediction accuracy reaches the maximum when the high-value patent threshold is 0.049, and then rapidly decreases, oscillating between [0.054, 0.069], with a small change range. The smaller the slope, the more gradual the change in the patent threshold prediction accuracy, that is, the higher the reliability of the SVM model. Therefore, [0.054, 0.069] is used as the second comprehensive threshold range.

[0063] Step 5, fitting the comprehensive value of the second comprehensive threshold range with the corresponding prediction accuracy to obtain a fitting equation, and substituting the maximum prediction accuracy on the accuracy curve into the fitting equation to obtain the comprehensive value threshold.

[0064] Fitting methods include, but are not limited to, the method of least squares.

[0065] like Figure 6 The fitting results are shown in Figure 2. Figure 6 It can be seen that the fitting correlation coefficient R 2 It is 0.9961, indicating that the SVM prediction accuracy in this interval is linearly correlated with the change of the threshold.

[0066] Depend on Figure 4It can be seen that the maximum prediction accuracy of SVM is 1.00. Substituting y=1.00 into the fitting equation y=6.2019x+0.534, we can get the high-value patent threshold x=0.075, that is, the comprehensive value threshold of high-value patents is 0.075.

[0067] Example 2 The difference between this embodiment and embodiment 1 is that: Figure 7 As shown, after obtaining the prediction accuracy of the machine learning model under each comprehensive value, calculate the prediction accuracy change difference between any comprehensive value and its adjacent previous comprehensive value, select the comprehensive value where the prediction accuracy change difference is greater than the set change difference for the first time as the limit point I, and select the comprehensive value where the prediction accuracy change difference is greater than the set change difference for the last time as the limit point II; The central difference method is used to obtain the slope of the accuracy curve between the limiting point I and the limiting point II.

[0068] During the test, it was found that the limit point I was 0.04 and the limit point II was 0.28. That is, the initial high-value patent threshold range was narrowed to [0.04, 0.28]. The SVM prediction accuracy curve corresponding to the initial high-value patent threshold range after the narrowing is as follows: Figure 4 The central difference method is used to obtain the slope of the accuracy curve between the limiting point I and the limiting point II, as shown in Figure 5 shown.

[0069] Narrowing the initial high-value patent threshold range can reduce the amount of calculation required for obtaining the slope of the accuracy curve using the central difference method. For example, if the step size is still 0.001, only 241 training sessions are required to obtain the SVM prediction accuracy under different thresholds.

[0070] Example 3 This embodiment provides a high-value patent evaluation method, such as Figure 8 As shown, the following steps are included: According to the pre-constructed patent value evaluation hierarchical structure model, multiple first-level evaluation dimensions of the patent to be evaluated and multiple evaluation indicators corresponding to each evaluation dimension are obtained; the evaluation dimensions at least include strategic value dimension, technical value dimension, legal value dimension and economic value dimension; Obtaining the comprehensive patent value of the patent to be evaluated according to the evaluation dimensions, the evaluation indicators and the high-value patent evaluation model; wherein the high-value patent evaluation model is constructed based on the entropy weight method and the hierarchical analysis method; The evaluation result of the patent is obtained according to the comprehensive value of the patent to be evaluated and the comprehensive value threshold, wherein the comprehensive value threshold is obtained by using the method for obtaining the high-value patent classification threshold described in Example 1 or Example 2.

[0071] Based on the high-value patent evaluation method proposed in this embodiment, each patent sample in the aforementioned patent sample set was evaluated, and finally 1,001 high-value patents were obtained, accounting for 2.4% of the total number of all patent samples.

[0072] Specifically, among the 1,001 high-value patents screened out, there are 993 invention patents and 8 utility model patents. Invention patents account for as high as 99.2%, which is consistent with the general view in academia that the value of invention patents is higher than that of utility model patents. Among the 1,001 high-value patents screened out, 97.4% of the patents meet the five high-value invention patent identification standards, which also reflects the scientific nature of the evaluation method. In addition, all award-winning core invention patents in all industries were accurately identified as high-value patents, with a match rate of 100%, further verifying the scientific nature and rationality of the evaluation method.

[0073] The descriptive statistics of the 1001 high-value patents screened out are shown in Table 4.

[0074] Table 4 Descriptive statistics of the value indicators of each dimension of the selected high-value patent dataset In general, high-value patents have the highest average technical value and the lowest economic value. The standard deviation of strategic value is the largest, indicating that the high-value patent set has a large degree of dispersion at the strategic level. The highest mean of technical value reflects its greatest contribution to the overall value, and the smaller standard deviation indicates that technological innovation has a relatively consistent impact on patent value. The distribution of legal value is relatively scattered, reflecting the diversity of patent legal protection. The average and standard deviation of economic value are both low, indicating that the market transformation potential of most patents needs to be improved, and the economic value performance of high-value patents is relatively consistent. The standard deviation of comprehensive value is small, indicating that the overall value of patents fluctuates less, but the gap between the maximum and minimum values ​​shows that some patents have higher value. Compared with Table 3, it can be seen that the mean of the high-value patent data set in the four value dimensions and comprehensive value is significantly better than that of the entire patent sample set.

[0075] Example 4 This embodiment provides a device for obtaining a high-value patent classification threshold, such as Fig. 9 As shown, including: Patent sample collection; A dimension and evaluation index acquisition module is used to obtain multiple first-level evaluation dimensions and multiple evaluation indicators corresponding to each evaluation dimension for each patent sample in the patent sample set according to a pre-built patent value evaluation hierarchical structure model; the evaluation dimensions at least include strategic value dimension, technical value dimension, legal value dimension and economic value dimension; High-value patent evaluation model construction module, used to construct a high-value patent evaluation model based on entropy weight method and hierarchical analysis method; An initial comprehensive threshold range acquisition module obtains the comprehensive value of each patent sample according to the evaluation dimension, the evaluation index and the high-value patent evaluation model, and selects the minimum comprehensive value and the maximum comprehensive value corresponding to the patent sample set as the initial comprehensive threshold range; The prediction accuracy acquisition module traverses each comprehensive value of the initial comprehensive threshold range in a set order, obtains the prediction accuracy of the machine learning model under each comprehensive value, and constructs an accuracy curve; The second comprehensive threshold range acquisition module uses the central difference method to obtain the slope of the accuracy curve, and takes the comprehensive value corresponding to the stable oscillation interval of the slope as the second comprehensive threshold range; A fitting equation acquisition module is used to fit the comprehensive value of the second comprehensive threshold range with the corresponding prediction accuracy to obtain a fitting equation; The threshold calculation module is used to bring the maximum prediction accuracy on the accuracy curve into the fitting equation to obtain the division threshold.

[0076] Example 5 This embodiment provides a high-value patent evaluation device, such as Fig.10 As shown, including: A dimension and evaluation index acquisition module, used to obtain multiple primary evaluation dimensions of the patent to be evaluated and multiple evaluation indicators corresponding to each evaluation dimension according to a pre-built patent value evaluation hierarchical structure model; the evaluation dimensions at least include strategic value dimension, technical value dimension, legal value dimension and economic value dimension; A comprehensive value threshold acquisition module, used to acquire a comprehensive value threshold based on the method for acquiring a high-value patent classification threshold described in Example 1 or 2; A comprehensive value acquisition module, used to obtain the comprehensive patent value of the patent to be evaluated according to the evaluation dimensions, the evaluation indicators and the high-value patent evaluation model; The value assessment module is used to obtain the patent evaluation result based on the comprehensive value of the patent to be evaluated and the comprehensive value threshold.

[0077] Example 6 This embodiment provides a computer device, such as Fig.11 As shown, it includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, it implements the steps of the method described in Example 1 or Example 2, or implements the steps of the method described in Example 3.

[0078] Furthermore, the computer device also includes an input / output interface and a network interface.

[0079] Example 7 This embodiment provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in Embodiment 1 or Embodiment 2 are implemented, or the steps of the method described in Embodiment 3 are implemented.

[0080] Example 8 This embodiment provides a computer program product, including a computer program. When the computer program is executed by a processor, the steps of the method described in Embodiment 1 or Embodiment 2 are implemented, or the steps of the method described in Embodiment 3 are implemented.

[0081] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0082] The present application is described with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0083] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0084] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1The steps for the functions specified in one or more boxes.

[0085] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0086] Computer readable media include permanent and non-permanent, removable and non-removable media and can store information by any method or technology. The information can be computer readable instructions, data structures, program modules or other data.

[0087] Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0088] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0089] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, ordinary technicians in the field should understand that the specific implementation methods of the present invention can still be modified or some technical features can be replaced by equivalents without departing from the spirit of the technical solution of the present invention, which should be included in the scope of the technical solution for protection of the present invention.

Claims

1. A method for obtaining a threshold value for high-value patent classification, characterized in that: The following steps are involved: Obtain a patent sample set, and obtain the evaluation dimension and corresponding evaluation index data of each patent sample in the patent sample set according to a pre-built patent value evaluation hierarchical structure model; Obtaining the comprehensive value of each patent sample according to the evaluation dimension, the evaluation index and the high-value patent evaluation model, and selecting the minimum comprehensive value and the maximum comprehensive value corresponding to the patent sample set as the initial comprehensive threshold range; Traverse each comprehensive value within the initial comprehensive threshold range in the set order, obtain the prediction accuracy of the machine learning model under each comprehensive value, and construct an accuracy curve; The slope of the accuracy curve is obtained by using the central difference method, and the comprehensive value corresponding to the stable oscillation interval of the slope is used as the second comprehensive threshold range; The comprehensive value within the second comprehensive threshold range is fitted with the corresponding prediction accuracy to obtain a fitting equation, and the maximum prediction accuracy on the accuracy curve is substituted into the fitting equation to obtain the comprehensive value threshold.

2. The method for obtaining a high-value patent classification threshold according to claim 1, characterized in that: The steps of traversing each comprehensive value of the initial comprehensive threshold range in a set order and obtaining the prediction accuracy of the machine learning model under each comprehensive value include: Determine the initial comprehensive threshold range and set the initial value of the maximum prediction accuracy to 0; Traverse each comprehensive value within the initial comprehensive threshold range in a set order, use each comprehensive value as a patent threshold, and generate a value label for each patent sample in the patent sample set according to the threshold, wherein the value label is a high-value patent or a common patent; Divide the patent sample set into a training set and a test set according to the generated value labels; Train the machine learning model based on the training set, and use the test set to test the prediction accuracy of the machine learning model; When the prediction accuracy is greater than the maximum prediction accuracy, the prediction accuracy is assigned to the maximum prediction accuracy, and then the next comprehensive value is traversed; otherwise, the next comprehensive value is traversed directly; Until the traversal ends.

3. A method for obtaining a high-value patent classification threshold according to claim 1 or 2, characterized in that: After obtaining the prediction accuracy of the machine learning model under each comprehensive value, calculate the prediction accuracy change difference between any comprehensive value and its adjacent previous comprehensive value, select the comprehensive value where the prediction accuracy change difference is greater than the set change difference for the first time as the limit point I, and select the comprehensive value where the prediction accuracy change difference is greater than the set change difference for the last time as the limit point II; The central difference method is used to obtain the slope of the accuracy curve between the limiting point I and the limiting point II.

4. A method for obtaining a high-value patent classification threshold according to claim 1 or 2, characterized in that: The patent value evaluation hierarchical structure model includes two layers of structure. The first layer of structure includes at least four evaluation dimensions, namely, strategic value dimension, technical value dimension, legal value dimension and economic value dimension; the second layer of structure includes multiple evaluation indicators corresponding to each evaluation dimension; Among them, the evaluation indicators corresponding to the strategic value dimension include patent awards and the technical field to which they belong; The evaluation indicators corresponding to the technical value dimension include patent type, number of patent citations, IPC classification span, technical integration, number of cooperative applicants, number of pages in the specification, and number of embodiments; The evaluation indicators corresponding to the legal value dimension include the number of authorized independent claims, the number of authorized claims, the remaining validity period of the patent, and the confirmation of rights after reexamination; The evaluation indicators corresponding to the economic value dimension include the number of times the patent is licensed, the number of times the patent is transferred, the number of times the patent is pledged, the number of patent families, and the number of countries in the patent family.

5. A high-value patent evaluation method, characterized in that: The following steps are involved: According to the pre-constructed patent value evaluation hierarchical structure model, multiple first-level evaluation dimensions of the patent to be evaluated and multiple evaluation indicators corresponding to each evaluation dimension are obtained; the evaluation dimensions at least include strategic value dimension, technical value dimension, legal value dimension and economic value dimension; Obtaining the comprehensive patent value of the patent to be evaluated according to the evaluation dimensions, the evaluation indicators and the high-value patent evaluation model; wherein the high-value patent evaluation model is constructed based on the entropy weight method and the hierarchical analysis method; The evaluation result of the patent is obtained according to the comprehensive value of the patent to be evaluated and the comprehensive value threshold, wherein the comprehensive value threshold is obtained by using the method for obtaining the high-value patent classification threshold described in any one of claims 1-4.

6. A device for obtaining a threshold value for dividing high-value patents, characterized in that: include: Patent sample collection; A dimension and evaluation index acquisition module is used to obtain multiple first-level evaluation dimensions and multiple evaluation indicators corresponding to each evaluation dimension for each patent sample in the patent sample set according to a pre-built patent value evaluation hierarchical structure model; the evaluation dimensions include at least a strategic value dimension, a technical value dimension, a legal value dimension, and an economic value dimension; High-value patent evaluation model construction module, used to construct a high-value patent evaluation model based on entropy weight method and hierarchical analysis method; An initial comprehensive threshold range acquisition module obtains the comprehensive value of each patent sample according to the evaluation dimension, the evaluation index and the high-value patent evaluation model, and selects the minimum comprehensive value and the maximum comprehensive value corresponding to the patent sample set as the initial comprehensive threshold range; The prediction accuracy acquisition module traverses each comprehensive value of the initial comprehensive threshold range in a set order, obtains the prediction accuracy of the machine learning model under each comprehensive value, and constructs an accuracy curve; The second comprehensive threshold range acquisition module uses the central difference method to obtain the slope of the accuracy curve, and takes the comprehensive value corresponding to the stable oscillation interval of the slope as the second comprehensive threshold range; A fitting equation acquisition module is used to fit the comprehensive value of the second comprehensive threshold range with the corresponding prediction accuracy to obtain a fitting equation; The threshold calculation module is used to bring the maximum prediction accuracy on the accuracy curve into the fitting equation to obtain the division threshold.

7. A high-value patent evaluation device, characterized in that: include: A dimension and evaluation index acquisition module, used to obtain multiple primary evaluation dimensions of the patent to be evaluated and multiple evaluation indicators corresponding to each evaluation dimension according to a pre-built patent value evaluation hierarchical structure model; the evaluation dimensions at least include strategic value dimension, technical value dimension, legal value dimension and economic value dimension; High-value patent evaluation model construction module, used to construct a high-value patent evaluation model based on entropy weight method and hierarchical analysis method; A comprehensive value threshold acquisition module, used to acquire a comprehensive value threshold based on the method for acquiring a high-value patent classification threshold according to any one of claims 1 to 4; A comprehensive value acquisition module, used to obtain the comprehensive patent value of the patent to be evaluated according to the evaluation dimensions, the evaluation indicators and the high-value patent evaluation model; The value assessment module is used to obtain the patent evaluation result based on the comprehensive value of the patent to be evaluated and the comprehensive value threshold.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the processor implements the steps of the method according to any one of claims 1 to 4, or implements the steps of the method according to claim 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 4, or implements the steps of the method according to claim 5.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 4, or implements the steps of the method according to claim 5.

Images