Patent grading and classifying evaluation method, system and equipment and storage medium

Abstract

The invention discloses a patent grading and classification evaluation method, system and device and a storage medium. The method comprises the following steps: step 1, constructing a training data set; 2, constructing a patent evaluation index system as an index of an input layer of the neural network model; 3, selecting a neural network model, and training the training data set; and 4, carrying out grading and classification evaluation on the to-be-evaluated patent by adopting the neural network trained in the step 3, and giving a grading and classification result of the corresponding patent data according to a grading and classification evaluation standard. According to the method, the neural network model is constructed, the historical data is subjected to model training, the patent grading and classification evaluation model is obtained, the evaluation efficiency is improved, meanwhile, the one-sidedness of quantitative evaluation is avoided, and the result uniformity is guaranteed through unified model evaluation.

Description

Technical Field

[0001] The present invention belongs to the field of big data processing technology, and specifically relates to a patent grading and classification evaluation method, system, device and storage medium based on neural network. Background Art

[0002] Patent grading and rating is a tiered and categorized evaluation of authorized patents based on specific standards and objectives. It assesses the type and level of authorized patents to facilitate subsequent high-value patent cultivation and patent operations such as licensing, transfer, pledge, and equity investment.

[0003] Existing patent grading and evaluation processes typically rely on manual evaluation, where a team of experts, drawing on industry experience, manually assesses authorized patents. However, manual evaluation is inefficient and slow, and is subject to subjective factors.

[0004] Patent publication number CN112231549A discloses a "patent value assessment method, system, device, and storage medium." This invention searches for patent information based on the patent's application number or publication number; constructs a patent value assessment model based on this detailed information combined with an assessment coefficient; and uses the patent value assessment model to calculate the patent's assessed value and ultimately assign a score. However, the assessment indicators constructed by this method are quantitative, used to assess the type and level of a patent. The results are relatively one-sided, resulting in low accuracy. Summary of the Invention

[0005] In order to solve the above technical problems, the present invention provides a patent grading and classification evaluation method and system based on neural network. By constructing a neural network model and training the model on historical data, a patent grading and classification evaluation model is obtained, which improves the evaluation efficiency while avoiding the one-sidedness of quantitative evaluation. The uniformity of the results is ensured through unified model evaluation.

[0006] The object of the present invention is achieved through the following technical solution, a patent classification and evaluation method, comprising the following steps:

[0007] Step 1: Build a training dataset;

[0008] Step 2: Construct a patent evaluation index system as the input layer index of the neural network model;

[0009] Step 3: Select a neural network model and train the training data set;

[0010] Step 4: Use the neural network trained in step 3 to perform hierarchical classification evaluation on the patents to be evaluated, and give the hierarchical classification results of the corresponding patent data according to the hierarchical classification evaluation standards.

[0011] Preferably, in step 1, historical patent data that has won the China Patent Award and has been evaluated by experts is used as the training data set.

[0012] Preferably, in step 2, the evaluation index system includes:

[0013] Technological influence, including at least the following sub-indicators: total number of cited patents in the simple same family, total number of cited patents in the INPADOC same family, number of citations within 3 years, and number of cited patents;

[0014] Field coverage and international presence, including at least the following sub-indicators: number of IPC symbols, number of simple family members, and number of INPADOC family members;

[0015] The degree of basic innovation includes at least the following sub-indicators: number of inventors, number of document pages, and number of claims.

[0016] Preferably, in step 3, the neural network model is trained using a deep learning library, and the model training is performed using a ten-fold cross method.

[0017] Preferably, in step 3, the specific process of training the training data set is: based on the expert evaluation results and the award-winning patents, analyzing the correlation between the quantitative indicators in the patent evaluation index system constructed in step 2 and the evaluation results, applying a neural network to comprehensively calculate the indicators, and drawing patent evaluation conclusions.

[0018] Preferably, step 3 at least includes the following steps:

[0019] Step 3.1, data preprocessing;

[0020] Step 3.2, feature selection;

[0021] Step 3.3, neural network architecture design;

[0022] Step 3.4, neural network training;

[0023] Step 3.5: Predict new data and generate a convertible index.

[0024] Preferably, in step 4, the grading classification standard is a decimal between 0-1, specifically as follows: 0.8-1 for core patents, 0.6-0.8 for important patents, 0.4-0.6 for basic patents, 0.2-0.4 for general patents, and 0-0.2 for auxiliary patents.

[0025] In addition to providing a patent grading and classification evaluation method based on a neural network, the present invention further provides a system for implementing the grading and classification evaluation method, which includes:

[0026] The data acquisition module is used to obtain the input training set data and the patent data to be evaluated, pre-process the input data, extract various sub-indicators, and input them into the model training module.

[0027] The model training module pre-sets the deep learning library, receives the pre-processed data, trains the neural network model, and uses the trained neural network model to evaluate the input patent data to be evaluated, and inputs the evaluation results into the grading and classification module.

[0028] The grading and classification module presets grading and classification standards, compares the input evaluation results with the grading and classification standards, and outputs the grading and classification results corresponding to the evaluation results.

[0029] The present invention further provides an electronic device, comprising a memory and a processor.

[0030] The memory is used to store computer programs.

[0031] The processor is used to load the computer program in the memory so that the above-mentioned patent grading and classification evaluation method can be implemented when the above-mentioned electronic device is running.

[0032] The present invention also provides a computer scale storage medium on which a computer program is stored. When the program is executed by a processor, the above-mentioned patent grading and classification evaluation method can be implemented.

[0033] Compared with the prior art, the present invention has the following advantages:

[0034] The present invention provides a patent grading and classification evaluation method, system, device and storage medium based on a neural network. The present invention constructs a training data set with high-quality labeled data based on long-term accumulated expert evaluation results and Chinese patent award data, and the training results are highly accurate. Based on the high-quality patent data, the optimal hyperparameters are determined during the training of the neural network, such as the structure of the neural network, the number of layers, the ratio of the training set to the test set, and the learning rate. These hyperparameters are obtained by debugging on a specific data set and have high accuracy. The present invention adopts a ten-fold crossover method to verify the prediction results of the model, which can prevent overfitting of the model.

[0035] The present invention relies on patent awards and expert evaluation results to calculate the correlation between indicators and conclusions, and selects indicators with stronger correlation; applies neural networks to learn indicators to avoid the problem of insufficient dimensions in simple linear calculations; and evaluates patents, normalizes the results to 0-1, and determines the grading and classification results of different patents. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a flow chart of the patent classification and evaluation method based on neural network in the present invention;

[0037] Figure 2 A schematic diagram of a predictive neural network architecture when a hierarchical classification evaluation method is used to evaluate the convertibility of patents in an embodiment of the present invention;

[0038] Figure 3 The patented convertible predictive neural network performance indicators proposed for neural network training in the embodiments of the present invention. DETAILED DESCRIPTION

[0039] The present invention will be further described in detail below with reference to the accompanying drawings and examples. It will be understood that the specific embodiments described herein are intended only to illustrate the present invention and are not intended to limit the present invention. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions relevant to the present invention, not all structures.

[0040] The technical solution of the present invention proposes a patent grading and classification evaluation method based on a neural network. This method analyzes the correlation between the patent's quantitative indicators and the evaluation results based on the patent's award-winning status and expert evaluation results. The neural network is then used to perform a comprehensive calculation of the indicators to draw a patent evaluation conclusion. This method can be applied to the grading and classification evaluation of relevant patent technologies before patent operations such as patent licensing, transfer, pledge, and equity investment, to assist in decision-making. Specifically, it includes the following:

[0041] Step 1: Construct a training dataset: Select patents that have won the China Patent Award as core patents; sort out the expert evaluation results compiled in the early stage and distinguish them into five types: core, important, basic, general, and auxiliary;

[0042] Step 2: Construct a patent evaluation index system as the input layer index of the neural network model:

[0043] Construct a patent evaluation index system (quantitative evaluation index). It is mainly divided into three aspects: the first is the influence of patent technology, including the total number of cited patents in the simple same family, the total number of cited patents in the INPADOC same family, the number of citations within 3 years, and the number of cited patents; the second is the coverage of patent fields and international layout, including the number of IPC classification numbers, the number of simple same family members, and the number of INPADOC same family members; the third is the degree of basic innovation of patents, including the number of inventors, the number of pages of documents, and the number of claims. In the process of determining the index system, according to the patent awards and expert evaluation results in step 1, the correlation between the indicators and the classification results is analyzed, and the indicators without significant correlation are eliminated, while the indicators with significant correlation are retained. Through repeated testing, the most appropriate index system is obtained.

[0044] Step 3: Select a neural network model and train the training data set:

[0045] The neural network model can be trained using Keras or other deep learning libraries in Python. The input layer includes multiple sub-indicators, and the output layer corresponds to two logical values ​​0 and 1 (corresponding to whether it is a high-value patent, where 1 indicates a high-value patent). The model is trained using the ten-fold crossover method, and the comprehensive recall rate and precision rate are used as indicators to evaluate the model performance.

[0046] Step 4: Use the neural network trained in step 3 to perform hierarchical classification evaluation on the patents to be evaluated, and give the hierarchical classification results of the corresponding patent data according to the hierarchical classification evaluation standard: the hierarchical classification standard used in the patent hierarchical classification evaluation is: the result is a decimal between 0-1, where 0.8-1 is a core patent, 0.6-0.8 is an important patent, 0.4-0.6 is a basic patent, 0.2-0.4 is a general patent, and 0-0.2 is an auxiliary patent.

[0047] Taking the evaluation of patent transferability as an example, the following describes in detail a patent classification and evaluation method based on a neural network provided in the invention technical solution. Figure 1 As shown, the patent classification and evaluation method based on neural network includes the following steps:

[0048] Step 1: Construct a training dataset: In this embodiment, a patent dataset of the shipbuilding industry is obtained through a patent platform, which contains a total of 6,646 patents.

[0049] Step 2: Construct a patent evaluation index system as the input layer index of the neural network model:

[0050] Construct a patent evaluation index system (quantitative evaluation indicators), including the current number of applicants (patent rights), the number of inventors, the number of claims, the number of document pages, the number of cited patents, the number of citations within 3 years and the IPC classification number.

[0051] Step 3: Select a neural network model and train the training data set:

[0052] The neural network model can be trained using Keras or other deep learning libraries in Python. The input layer includes multiple sub-indicators, and the output layer corresponds to two logical values ​​0 and 1 (corresponding to whether it is a high-value patent, where 1 indicates a high-value patent). The model is trained using the ten-fold crossover method, and the comprehensive recall rate and precision rate are used as indicators to evaluate the model performance. Specifically, in this embodiment, step 3 includes the following steps:

[0053] Step 3.1, data preprocessing:

[0054] The training dataset contains fields such as publication (announcement) number, title, abstract, patent type, technical efficacy phrase, number of current applicants (patentees), agents, number of inventors, IPC classification number, number of claims, document page count, claims, independent claims, application year and month, publication (announcement) year and month, strategic emerging industries classification, specification, IPC main classification number (subclass), IPC main classification number (major group), number of cited patents, number of citations within three years, license type, and legal status / event. These fields contain both valid and invalid features. Furthermore, some fields contain empty information, making them difficult to process. Data preprocessing requires data cleaning, setting them to zero or removing them as invalid examples. The following rules are established to determine the label of each patent example based on the legal status / event field information: when the legal status / event field of a patent example contains "license" or "rights transfer," its label value is 1, indicating conversion; otherwise, it is non-conversion. The above method yielded a dataset of 6,646 patent convertibles. We divided this dataset into a training set and a test set. The training set, with 5,500 samples, was used to train the neural network for our proposed patent convertible identification. The test set, with 1,146 samples, was used to test the performance of the trained neural network model.

[0055] Step 3.2, feature selection:

[0056] In this embodiment, in order to facilitate the training of the neural network, seven relevant numerical features are selected: the number of current applicants (patent rights), the number of inventors, the number of claims, the number of document pages, the number of cited patents, the number of citations within 3 years and the IPC classification number.

[0057] Step 3.3, neural network architecture design:

[0058] Since patent conversion prediction is a binary classification problem, the architecture of the neural network is designed as follows Figure 2 As shown in the figure, the patent transferability prediction neural network is a fully connected neural network consisting of an input layer, two hidden layers, and an output layer. The input layer fully corresponds to the seven selected features. The first hidden layer contains 64 neurons, and the second hidden layer contains 32 neurons. The output layer contains two neurons, corresponding to the logical values ​​(logit) of 1 and 0, respectively. Assuming that the logical value of 1 is large, the output of the neural network is 1, thus achieving transferability prediction.

[0059] Step 3.4, neural network training:

[0060] The proposed patent transferable prediction neural network is implemented using a multi-layer perceptron in Pytorch, and trained and tested. The basic principles are:

[0061] Multi-Layer Perceptron (MLP) is a basic deep learning model that can be implemented in PyTorch by defining a network class that inherits from nn.Module.

[0062] The following are the basic steps of the multi-layer perceptron:

[0063] ①Import related libraries:

[0064] import torch

[0065] import torch.nn as nn

[0066] import torch.optim as optim

[0067] ②Define the MLP network class:

[0068] class MLP(nn.Module):

[0069] def__init__(self,input_dim,hidden_dim,output_dim):

[0070] super(MLP,self).__init__()

[0071] self.fc1=nn.Linear(input_dim,hidden_dim)

[0072] self.fc2=nn.Linear(hidden_dim,output_dim)

[0073] def forward(self,x):

[0074] x = self.fc1(x)

[0075] x = torch.relu(x)

[0076] x = self.fc2(x)

[0077] x = torch.sigmoid(x)

[0078] return x

[0079] In the above code, the MLP class inherits from nn.Module and overrides the __init__ and forward methods.

[0080] In the __init__ method, two fully connected layers (nn.Linear) are created: self.fc1, which connects the input layer to the hidden layer, and self.fc2, which connects the hidden layer to the output layer. In the forward method, the forward propagation process of the data in the network is defined.

[0081] ③Create a network instance:

[0082] input_dim=XX#input dimension

[0083] hidden_dim=XX #hidden layer dimension

[0084] output_dim=1#output dimension (output 1 value for binary classification problem)

[0085] net=MLP(input_dim,hidden_dim,output_dim)

[0086] Create a network instance by calling the MLP class and passing in parameters such as input dimension, hidden layer dimension, and output dimension.

[0087] ④Define the loss function and optimizer:

[0088] criterion = nn.BCELoss() # Use BCELoss as the loss function for binary classification problems

[0089] optimizer = optim.SGD (net.parameters (), lr = 0.01) # Use stochastic gradient descent to optimize, with a learning rate of 0.01

[0090] In the above code, the binary cross entropy loss function (nn.BCELoss) is used as the loss function of the model; the stochastic gradient descent method (optim.SGD) is used to optimize the model parameters, and the learning rate is 0.01.

[0091] ⑤ Conduct model training:

[0092] for epoch in range(num_epochs):

[0093] #Forward propagation

[0094] outputs = net(inputs)

[0095] loss=criterion(outputs,labels)

[0096] #Backpropagation and optimization

[0097] optimizer.zero_grad()# gradient zeroing

[0098] loss.backward()#backward propagation

[0099] optimizer.step()#Update weight

[0100] #Print training information

[0101] if (epoch+1) % 10 == 0:

[0102] print('Epoch[{} / {}],Loss:{:.4f}'.format(epoch+1,num_epochs,loss.item()))

[0103] During training, forward propagation is performed first to calculate the network output and loss value. Backward propagation is then performed to calculate the gradients of each parameter. Finally, the optimizer's step method is called to update the weights. Training information, such as loss values, can be printed as needed during training.

[0104] ⑥ Make model predictions:

[0105] with torch.no_grad():

[0106] predicted = net(test_data)

[0107] predicted_labels=torch.round(predicted)

[0108] In the prediction phase, the prediction result is obtained by calling the network instance and passing in the test data. In the case of binary classification problems, the predicted label can be obtained by rounding the prediction result.

[0109] The training parameters are: learning rate, batch size of 64, and number of training rounds of 1000. After the training, the test set is used to evaluate the designed patent conversion prediction neural network. The test results are as follows: Figure 3 As shown in the figure, the model has an accuracy of 75.6%, a precision of 72.5%, and a recall of 77.4% in the test set, which satisfies the patent convertibility prediction in the field of shipbuilding to a certain extent.

[0110] Step 3.5: Predict new data and generate a conversion index:

[0111] Since the model is a classification model, it is necessary to determine the patent's convertibility index in practice. To this end, this embodiment first normalizes the logical output value of the neural network model to the range [0, 1], and uses the normalized logical output value as the convertibility index to predict unknown patents. The convertibility index is then mapped to the convertibility level, as shown in Table 1. Convertibility index and convertibility level:

[0112] Table 1

[0113]

[0114]

[0115] In this embodiment, the expert evaluation results adopt the patent open license selection and valuation scheme based on expert evaluation, which is as follows:

[0116] Patent open license selection and valuation scheme based on expert evaluation (if the economic value assessment of open licenses is not required in daily management, and only the grading and classification evaluation is conducted from the perspective of patent quality, the scoring method in step 4 can be used as a reference):

[0117] Step 1: Determine the appropriateness of patent open licensing

[0118] The patented technology is not in the exclusive or exclusive licensing status; the corresponding product of the patented technology has good market prospects and is suitable for patent open licensing.

[0119] Step 2: Select an estimation method

[0120] Since the patented technology has not been implemented, estimates can be made by referring to patent licensing statistics in the same industry.

[0121] Step 3: Determine the calculation base

[0122] Clarify the industry's average annual patent payment based on a fixed or convertible amount, and the average license term data.

[0123] Step 4: Calculate the adjustment coefficient

[0124] (1) Establish an indicator system

[0125] Based on the special scenario of patent open licensing, an index system for factors to consider in patent license fee adjustment coefficients is constructed. The index system consists of two levels of indicators, including 4 first-level indicators and 14 second-level indicators. The specific indicators are shown in Table 2:

[0126] Table 2 Index system of factors to consider when adjusting the patent open license royalty rate (applicable to the estimation method based on statistical data of patent licenses in the same industry)

[0127]

[0128]

[0129] (2) Establish indicator weights

[0130] In the special application scenario of patent open licensing, the calculation process of open licensing royalties is usually simplified, and equal weights are assigned to indicators at all levels. Different weight values ​​can also be set for the indicator system based on specific scenarios.

[0131] (3) Assignment

[0132] Experts from specific industries are invited to compare the status of proposed patents with the industry average and assign a value to each. Using the industry average as a reference, if the patents are consistent with the average, the indicator score is 1; if they are 0.2 times better than the average, the score is 1.2; and if they are 0.2 times less than the average, the score is 0.8.

[0133] (4) Specific calculation

[0134] The adjustment coefficient f can be calculated according to the values ​​assigned to indicators at all levels and the weights using the following formula.

[0135]

[0136] Where: α i is the corresponding weight of the secondary indicator; M i It is the score value of the secondary indicator.

[0137] Step 5: When the patentee chooses to pay in lump sum or in installments within the lump sum amount:

[0138] Total open license patent royalties = average annual patent open license royalties × license term = average annual patent amount × adjustment coefficient × license term.

[0139] It should be noted that, in the present invention, the construction of the data set, the construction of the patent evaluation index system, and the selection and training of the neural network are not limited to the situations in the above-mentioned embodiments. The patent grading and classification evaluation method provided in the technical solution of the present invention can be adaptively adjusted according to different grading and classification evaluation purposes, and they are not listed one by one here.

[0140] In addition to providing a neural network-based patent grading and classification evaluation method, the technical solution of the present invention further provides a system for implementing the grading and classification evaluation method, which includes:

[0141] The data acquisition module is used to obtain the input training set data and the patent data to be evaluated, pre-process the input data, extract various sub-indicators, and input them into the model training module.

[0142] The model training module pre-sets the deep learning library, receives the pre-processed data, trains the neural network model, and uses the trained neural network model to evaluate the input patent data to be evaluated, and inputs the evaluation results into the grading and classification module.

[0143] The grading and classification module presets grading and classification standards, compares the input evaluation results with the grading and classification standards, and outputs the grading and classification results corresponding to the evaluation results.

[0144] The technical solution of the present invention further provides an electronic device, including a memory and a processor.

[0145] The memory is used to store computer programs.

[0146] The processor is used to load the computer program in the memory so that the above-mentioned patent grading and classification evaluation method can be implemented when the above-mentioned electronic device is running.

[0147] The technical solution of the present invention also provides a computer scale storage medium on which a computer program is stored. When the program is executed by a processor, the above-mentioned patent grading and classification evaluation method can be implemented.

[0148] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A patent classification and evaluation method, characterized by: The evaluation method comprises the following steps: Step 1: Build a training dataset; Step 2: Construct a patent evaluation index system as the input layer index of the neural network model; Step 3: Select a neural network model and train the training data set; Step 4: Use the neural network trained in step 3 to perform hierarchical classification evaluation on the patents to be evaluated, and provide hierarchical classification results for the corresponding patent data in accordance with the hierarchical classification evaluation standards.

2. A patent classification and evaluation method according to claim 1, characterized in that: In step 1, historical patent data that have won Chinese patent awards and have been evaluated by experts are used as a training data set.

3. A patent classification and evaluation method according to claim 2, characterized in that: In step 2, the evaluation index system includes: Technological influence, including at least the following sub-indicators: total number of cited patents in the simple same family, total number of cited patents in the INPADOC same family, number of citations within 3 years, and number of cited patents; Field coverage and international presence, including at least the following sub-indicators: number of IPC symbols, number of simple family members, and number of INPADOC family members; The degree of basic innovation includes at least the following sub-indicators: number of inventors, number of document pages, and number of claims.

4. A patent classification and evaluation method according to claim 3, characterized in that: In step 3, the neural network model is trained using a deep learning library, and the model training is performed using a ten-fold cross method.

5. A patent classification and evaluation method according to claim 4, characterized in that: In step 3, the specific process of training the training data set is as follows: based on the expert evaluation results and the award-winning patents, the correlation between the quantitative indicators in the patent evaluation index system constructed in step 2 and the evaluation results is analyzed, and the indicators are comprehensively calculated using a neural network to obtain the patent evaluation conclusion.

6. A patent classification and evaluation method according to claim 5, characterized in that: The step 3 at least includes the following steps: Step 3.1, data preprocessing; Step 3.2, feature selection; Step 3.3, neural network architecture design; Step 3.4, neural network training; Step 3.5: Predict new data and generate a convertible index.

7. A patent classification and evaluation method according to claim 6, characterized in that: In step 4, the grading classification standard is a decimal between 0 and 1, specifically as follows: 0.8-1 is a core patent, 0.6-0.8 is an important patent, 0.4-0.6 is a basic patent, 0.2-0.4 is a general patent, and 0-0.2 is an auxiliary patent.

8. A patent classification and evaluation system, characterized by: The system implements the patent grading and classification evaluation method according to any one of claims 1 to 7, and the system includes: The data acquisition module is used to obtain the input training set data and the patent data to be evaluated, pre-process the input data, extract various sub-indicators, and input them into the model training module; The model training module has a pre-installed deep learning library, receives pre-processed data, trains the neural network model, and uses the trained neural network model to perform evaluation based on the input patent data to be evaluated, and inputs the evaluation results into the grading and classification module; The grading and classification module presets grading and classification standards, compares the input evaluation results with the grading and classification standards, and outputs the grading and classification results corresponding to the evaluation results.

9. An electronic device comprising a memory and a processor, wherein: The memory is used to store computer programs, and the processor is used to load the computer programs in the memory. It is characterized in that when the electronic device is running, it can implement the patent grading and classification evaluation method described in any one of claims 1-7.

10. The present invention also provides a computer scale storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, it can implement the patent grading and classification evaluation method described in any one of claims 1-7.

Images