Multi-relational graph comparison enhanced high-value patent identification method and system

Abstract

The invention discloses a multi-relation graph comparison enhanced high-value patent identification method and system, and the method comprises the steps: S100: building an applicant network and an inventor network through common relation connection, calculating the patent text similarity based on text embedding generated by a BERT pre-training model, and building a patent text topic network, constructing a complex heterogeneous relationship of the patent network; s200, using a graph attention network to learn importance relations of different patent neighbors in different relation modes; s300, respectively learning embedded representations in different patent relation modes through an attention network; and S400, constructing a multi-relation perceptual attention mechanism by taking applicant relation embedding as query and combining the inventor and theme relation as perceptual embedding, performing model training on a data set after determining a loss function, performing model performance evaluation and ablation experiment, and obtaining a multi-relation graph comparison enhanced high-value patent recognition algorithm model. The model provided by the invention shows high robustness in the aspect of automatically identifying high-value patents.

Description

Technical Field

[0001] The present invention belongs to the technical field of patent text mining, and more specifically, relates to a high-value patent identification method and system with multi-relationship graph contrast enhancement. Background Art

[0002] Patent documents, as an important carrier of scientific and technological intelligence, are core resources for promoting the transformation of innovative achievements, technological progress and economic development. According to the latest report released by the World Intellectual Property Organization, the number of global patent applications will reach 3.5 million in 2023, but the surge in the number of patent applications has not brought about an improvement in the overall quality of patents. Research shows that the value of global patents is created by only 5% of high-value patents, and there are very few patents that can be successfully transformed into actual results. High-value patents not only help companies accurately locate R&D investment and realize technology industrialization, but also significantly enhance technological competitiveness and market share [4]. However, the evaluation indicators of high-value patents are complex and difficult to identify, which makes it difficult for R&D results between patents and the industry to be quickly and efficiently industrialized. Therefore, accurate identification of high-value patents is an important link in accelerating the transformation of patent technology results.

[0003] There are three existing methods for the above technical difficulties. (1) The evaluation method based on patent indicators is studied, and the value of technological innovation is measured by relying on patent quality. This method provides a new perspective for the quantification of patent value, helps to more accurately identify and utilize high-value patents, and explores the direction for the identification of high-value patents. However, due to the large number and complexity of patent indicators and the difficulty in publicly obtaining high-value patent transaction data, it is difficult for the industry to reach a unified consensus on which indicator system has the greatest impact on patent value; (2) To this end, some scholars have begun to explore the use of complex network methods, constructing complex patent networks through information such as patent applicants and inventors, and performing multi-relationship feature aggregation, ultimately achieving accurate identification of high-value patents. Patent complex network analysis is often limited to analyzing a single type of patent relationship to evaluate the value of a patent. In general, the complex network method provides new ideas for the identification of high-value patents, but to overcome its shortcomings in multi-dimensional data fusion and feature extraction, it still needs to be combined with more advanced analysis technologies, such as deep learning, to improve the comprehensiveness and accuracy of identification. Machine learning has significantly improved the resolution of high-value patent (HVP) identification models due to its nonlinear mapping capabilities, and has largely overcome challenges such as insufficient data and data privacy, opening up new ideas for patent evaluation and automated identification. Early studies mainly used machine learning technology to discover high-value patents. For example, Yang et al. measured technical novelty and used K-means clustering methods to identify technical opportunities in the field of drones. However, traditional machine learning methods still have some shortcomings when dealing with complex patent networks. For example, when faced with patent data with strong domain characteristics and complex relationships, machine learning models often find it difficult to effectively capture the complex associations between patents, especially in multi-dimensional data fusion and deep feature extraction. Therefore, some scholars have proposed deep learning methods, which automatically extract and learn complex features in patent data through multi-layer neural network structures. Especially when faced with multi-dimensional and nonlinear data, the ability of deep learning far exceeds that of traditional machine learning models. Deep learning can capture complex relationships and hidden patterns between patents at different levels of patent texts, and more accurately identify high-value patents.

[0004] Although deep learning has achieved certain results in improving the accuracy of high-value patent identification, it still faces the following challenges: (1) It lacks the ability to integrate and model complex heterogeneous multi-relationships such as patent applicants, inventors, and patent texts; (2) Current high-value patent identification methods based on deep learning mainly focus on a single patent relationship, which leads to deviations in the model's understanding of patent network semantics. Summary of the invention

[0005] In view of the above defects or improvement needs of the prior art, the present invention provides a high-value patent identification method and system with long multi-relationship graph comparison enhancement, which models the complex multi-relationship pattern between patents through meta-paths, and uses graph comparison learning to strengthen the learning process of multi-relationship patent networks, thereby effectively identifying high-value patents. Comparative experimental analysis shows that the model proposed in the present invention shows a high degree of robustness in automatically identifying high-value patents, effectively responds to the challenges of complex patent network environments, and provides new ideas for the identification of high-value patents.

[0006] In order to achieve the above object, according to a first aspect of the present invention, a high-value patent identification method with multi-relationship graph contrast enhancement is provided, comprising:

[0007] S100: Establish the applicant network and inventor network through common relationship connection, calculate the similarity of patent text based on the text embedding generated by the BERT pre-training model, build the patent text topic network, and then build the complex heterogeneous relationship of the patent network;

[0008] S200: Use graph attention networks to learn the importance relationships of different patent neighbors under different relationship patterns, and construct graph contrast learning between different relationships to form collaborative supervision signals, explore the commonalities of different relationship patterns, and enhance the robustness of high-value patent mining;

[0009] S300: Learn the embedded representations of different patent relationship modes through the attention network, and establish supervision signals between different relationships to achieve the coordination and enhancement of patent features in different relationship views;

[0010] S400: Taking the applicant relationship embedding as the query and combining the inventor and subject relationship as the perception embedding, a multi-relationship perception attention mechanism is constructed, which integrates the embedding representations under the patent applicant, inventor and subject relationship model. After determining the loss function, the model is trained on the dataset, followed by model performance evaluation and ablation experiments, and finally a high-value patent recognition algorithm model with multi-relationship graph contrast enhancement is obtained.

[0011] Furthermore, in step S100, a patent-applicant interaction matrix is ​​constructed. and Patent-Inventor Interaction Matrix They are:

[0012]

[0013] in represents the patent relationship matrix under the meta-path relationship patent-applicant-patent, Represents the patent relationship matrix under the meta-path patent-inventor-patent.

[0014] Furthermore, constructing the patent text subject network in step S100 includes:

[0015] S101: Construct fixed-dimensional token vectors and position encodings for patent subject sentence segmentation to determine the order of sentence context;

[0016] S102: Use the multi-head self-attention mechanism to fuse the sentence context information, and its output is:

[0017]

[0018] where Q, K and V represent the query, key and value matrices respectively.

[0019] Furthermore, constructing the patent text topic network in step S100 also includes: using cosine similarity to measure the correlation between different patents:

[0020]

[0021] in, and denote the text topic embedding representation of patent i and patent j respectively, sim(·) denotes the cosine similarity function, A represents the threshold set in this paper to determine whether there is an interaction between patents. PAP represents the adjacency matrix between text topics between patents, A PTP is a normalized topic similarity matrix. The larger the value, the stronger the topic relevance of the corresponding patent.

[0022] Furthermore, the importance of patent neighbors in step S200 includes:

[0023] Given a target patent node i and its neighbor patent node set under relationship r Adaptively learn and calculate the attention coefficient between the target patent and its neighbor patents, where Indicates the importance of neighbor patent j to patent i. After calculating the importance of neighbor patent nodes to the target patent node, normalization is performed through the softmax function to obtain the weight coefficients of the target patent and neighbor patents. The attention coefficients between different patents are as follows:

[0024]

[0025] Where σ represents the ReLu activation function, || represents the concatenation operation, and a T is the node-level attention vector,

[0026] is the neighbor patent node of the target patent, represents the neighbors of patent i under relation r, is a collection of patent relations.

[0027] Furthermore, the importance of patent neighbors in step S200 also includes:

[0028] Setting K attention heads to the importance of neighboring patents, the embedding of target node i can be aggregated through the projected features of neighboring nodes and the corresponding weight coefficients as shown below:

[0029]

[0030] in Represents the embedded representation of patent i after aggregation of patent neighbors in the rth relation mode.

[0031] Furthermore, the image comparison learning in step S200 includes:

[0032] S201: Use a multi-layer perceptron with multiple hidden layers to map these features to a unified contrast space dimension, as follows:

[0033]

[0034]

[0035]

[0036] Among them, W is the weight and b is the coefficient.

[0037] Furthermore, the image comparison learning in step S200 includes:

[0038] S202: In the applicant relationship, the neighbor patents of the target patent are taken as positive samples; in the co-inventor relationship model, the neighbor patents of the target patent are taken as positive samples, including its neighbor patents in the inventor view.

[0039] Furthermore, the image comparison learning in step S200 includes:

[0040] S203: Construct the patent-applicant-patent graph adjacency matrix A PAP and the patent-inventor-patent graph adjacency matrix A PIP Construct the positive and negative sample matrix as:

[0041]

[0042] in The dimensions of both are the same. Indicates the addition of elements, thereby maintaining A PAP and A PIP The synergy between the two.

[0043] Furthermore, the image comparison learning in step S200 includes:

[0044] S204: In Patent-Applicant-Patent A PAP and the patent-topic-patent relationship graph adjacency matrix A PTP Construct the positive and negative sample matrix as:

[0045]

[0046] Furthermore, the graph contrast learning in step S200 includes: using the subject relationship and the inventor relationship to assist the applicant view embedding, forming a synergistic reinforcement under the three relationship modes, thereby constructing the contrast learning loss as follows:

[0047]

[0048] in represents the sum of graph contrast enhancement losses in the patent multi-relation mode. The sim(·) function is used to estimate the similarity of positive and negative pairs between different patent relations. τ is the temperature parameter. is the set of positive sample pairs, is the set of negative sample pairs.

[0049] Further, the multi-relation-aware attention mechanism in step S400 includes designing a multi-relation-aware self-attention mechanism encoder, which is composed of L multi-head self-attention and feed-forward network blocks stacked;

[0050] Taking the relationship between inventor and applicant as the input of Key and Value, the calculation process is as follows:

[0051]

[0052] FFN(O,W1,b1,W2,b2)=GELU(OW1+b1)W2+b2 (15)

[0053] in They respectively represent the relationship-aware signals introduced in this paper.

[0054] Furthermore, the multi-relationship-aware attention mechanism in step S400 includes using a self-attention mechanism to capture patent dependencies, and the calculation process is as follows:

[0055]

[0056]

[0057]

[0058] in, represents the learnable weight matrix, L represents the number of multi-head attention, and the perceptual representation of patent node i is obtained by averaging them in is derived from the representation in;

[0059]

[0060] Where W out is the linear transformation matrix, b out is a trainable parameter.

[0061] Furthermore, the loss function in step S400 is a binary cross entropy loss function,

[0062]

[0063] in, Represents the value of the loss function, y represents the vector of the true label, whose elements are 0 or 1; p represents the probability vector predicted by the model, that is, the original predicted value output by the model without being processed by the sigmoid function, and log represents the natural logarithm function.

[0064] Furthermore, the final optimization loss of the loss function in step S400 is the sum of the binary cross entropy loss function and the multi-relation enhancement contrast loss, that is:

[0065]

[0066] Among them, λ is a hyperparameter for regulating contrastive learning loss, θ is a trainable parameter in the model, and ξ is a regularization coefficient.

[0067] According to a second aspect of the present invention, a high-value patent identification system with multi-relationship graph contrast enhancement is provided, comprising:

[0068] Complex heterogeneous relationship building module, which is used to establish applicant networks and inventor networks through common relationship connections, calculate patent text similarity based on text embedding generated by the BERT pre-training model, build patent text topic network, and then build complex heterogeneous relationships of patent networks;

[0069] Graph contrast learning module, which is used to learn the importance relationship of different patent neighbors using graph attention network under different relationship modes, and construct graph contrast learning between different relationships to form collaborative supervision signals, explore the commonalities of different relationship modes, and enhance the robustness of high-value patent mining;

[0070] Patent relationship graph attention fusion module, which is used to learn the embedding representations of different patent relationship modes through the attention network and establish supervision signals between different relationships to achieve the coordination and enhancement of patent features in different relationship views;

[0071] The multi-relationship-aware attention mechanism fusion module is used to use the applicant relationship embedding as the query and combine the inventor and topic relationship as the perception embedding to construct a multi-relationship-aware attention mechanism, which integrates the embedding representations under the patent applicant, inventor and topic relationship model. After determining the loss function, the model is trained on the dataset, followed by model performance evaluation and ablation experiments, and finally a high-value patent recognition algorithm model with multi-relationship graph contrast enhancement is obtained.

[0072] According to a third aspect of the present invention, there is provided a terminal, comprising:

[0073] Memory for storing computer programs;

[0074] A processor, configured to implement the steps of the processing method according to any one of claims 1 to 15 when executing the computer program.

[0075] According to a fourth aspect of the present invention, there is provided a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the processing method according to any one of claims 1 to 15 are implemented.

[0076] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects compared with the prior art:

[0077] 1. The recognition method of the present invention constructs a contrastive learning enhancement strategy under a multi-relation graph network, effectively models the multi-relation characteristics between patents, and successfully realizes the recognition task of high-value patents. At the same time, a combined feature set based on 17 secondary indicators is constructed, which avoids the complex calculation and manual evaluation of traditional multi-features, and can automatically and scalably realize the recognition of high-value patents with a small number of samples. An empirical study was carried out on a patent data set in the field of humanoid robots. Through comparative experimental analysis, it was found that the method proposed in this paper improved the accuracy (ACC) by 15.12%, the recall rate (Recall) by 7.59%, the normalized discounted cumulative gain (NDCG) by 3.57%, and the area under the curve (AUC) also showed excellent performance, confirming that the MRGCE model is highly robust in automatically identifying high-value patents.

[0078] 2. The identification method of the present invention effectively identifies the value of patents in complex networks by creating different patent relationship patterns, and uses the GNN model to adaptively learn the importance of patents to enhance the synergy of patent features.

[0079] 3. The recognition method of the present invention introduces a multi-relationship-aware attention mechanism that can adaptively fuse the embedded representations of different relationship patterns, greatly improving the accuracy and robustness of patent value recognition.

[0080] 4. The identification method of the present invention explores different learning models and adopts a multi-dimensional value indicator combination method to pre-evaluate high-value patents, effectively responding to the challenges of a complex patent network environment, showing excellent performance in comparison with multiple models, and providing new ideas for the identification of high-value patents. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1 This is a schematic diagram of a high-value patent identification task according to an embodiment of the present invention;

[0082] Figure 2 A schematic diagram of a process flow of a high-value patent identification method with enhanced multi-relationship graph comparison according to an embodiment of the present invention;

[0083] Figure 3 A schematic diagram of constructing a multi-relationship network of a patent embodiment of the present invention;

[0084] Figure 4 A schematic diagram of constructing the relationship between positive and negative samples according to an embodiment of the present invention;

[0085] Figure 5 This is a schematic diagram of the fusion of multi-relationship perception and multi-head attention mechanism according to an embodiment of the present invention;

[0086] Figure 6 This is the influence of different relationship combinations on the recognition performance in the embodiments of the present invention. DETAILED DESCRIPTION

[0087] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0088] Example 1

[0089] like Figure 1 and Figure 2 As shown in the figure, a high-value patent identification method based on multi-relational graph contrast enhancement is proposed, which aims to build a deep graph neural network model from multi-category relational data of patent data to identify high-value patents. First, the present invention constructs three relationship patterns, constructs a patent network of applicants and inventors through common relationships, and constructs a topic network using patent subject text generated by the BERT pre-trained model, thereby extracting the structure of a multi-relationship patent interaction graph. Secondly, under different relationship patterns, the present invention uses a graph attention network to learn the importance relationship of different patent neighbors, and constructs graph comparative learning between different relationships to form collaborative supervision signals, mine the commonalities of different relationship patterns, and enhance the robustness of high-value patent mining. In addition, this study designed a multi-relationship-aware attention mechanism to fuse patent representations under different relationship patterns to enhance the robustness of the model output. Finally, the method of the present invention achieved the best patent value path recognition performance on real patent data in multiple fields.

[0090] Among them, the extraction of patent multi-relationship network is as follows: Figure 3 As shown, it aims to highlight the association between patents, thereby forming patent embedding with strong association. In order to more effectively model different relationship graph patterns, the present invention constructs meta-paths to traverse all patent-applicant-patent relationships and patent-inventor-patent relationships, and extracts patent graph networks under the two relationships respectively.

[0091] In order to realize the above two relationship extraction processes, the present invention constructs the patent-applicant interaction matrix

[0092] and Patent-Inventor Interaction Matrix The above matrix can be obtained by the following formula:

[0093]

[0094] in represents the patent relationship matrix under the meta-path relationship patent-applicant-patent, Represents the patent relationship matrix under the meta-path patent-inventor-patent.

[0095] Considering that the patent subject outlines the core content of the patent, the present invention uses the BERT pre-trained model to capture the contextual relationship of the text subject to extract the subject text features. Since BERT is a variant of Transformer, k Represents the dimension of the BERT model output. The order of the sentence context is determined by constructing a fixed-dimensional Token vector and position encoding for the patent subject sentence segmentation. Then, the sentence context information is fused using the multi-head self-attention mechanism, and its output is:

[0096]

[0097] Where Q, K and V represent query, key and value matrices respectively.

[0098] Through feedforward neural network, layer normalization and residual connection, the embedded representation of the topic sentence is output, and its formula is as follows:

[0099] FFN(x)=max(0,xW1+b1)W2+b2 (3)

[0100] h s =LN(x+sublayer(x)) (4)

[0101] Where FFN(x) represents the linear transformation of the topic sentence features, h s Represents the final sentence features after residual connection and layer normalization.

[0102] For text topic embedding, the present invention uses cosine similarity to measure the correlation between different patents, and its formula is:

[0103]

[0104] in, and denote the text topic embedding representation of patent i and patent j respectively, sim(·) denotes the cosine similarity function, Indicates the threshold set by the present invention, which is used to determine whether there is an interaction boundary between patents. PAP represents the adjacency matrix between text topics between patents, A PTP is a normalized topic similarity matrix, the larger the value, the stronger the topic relevance of the corresponding patent. In this way, the present invention completes the construction of graph networks of three relationship modes, which respectively represent graph structure networks under different semantic relationships.

[0105] Furthermore, the patent relationship graph attention fusion includes that in the graph network of different patent relationship patterns, the value of a patent is deeply affected by neighboring patents. Therefore, the present invention uses a graph attention mechanism to distinguish the importance weights of neighboring patents.

[0106] Given a target patent node i and its neighbor patent node set under relationship r Formula (6) is used to adaptively learn and calculate the attention coefficient between the target patent and the neighboring patents, where

[0107] Indicates the importance of neighbor patent j to patent i. After calculating the importance of neighbor patent nodes to the target patent node, normalize it through the softmax function to obtain the weight coefficients of the target patent and neighbor patents. The calculation formula of the attention coefficient between different patents is as follows:

[0108]

[0109] Where σ represents the ReLu activation function, || represents the concatenation operation, and a T is the node-level attention vector. Note that j is the neighbor patent node of the target patent. represents the neighbors of patent i under relation r. is a collection of patent relations.

[0110] In addition, in order to maintain the stability of self-attention mechanism learning, the present invention sets K attention heads to the importance of neighbor patents. In the experiment, the present invention sets K to 4. Then, the embedding of the target node i can be aggregated by the projected features of the neighbor nodes and the corresponding weight coefficients, as shown below:

[0111]

[0112] in Represents the embedded representation of patent i after aggregation of patent neighbors in the rth relation mode.

[0113] like Figure 4 As shown, the patent multi-relationship graph comparison enhancement includes, in order to strengthen the synergy of learning between different patent relationship model views, the present invention constructs a novel view relationship enhancement graph comparison learning architecture. The architecture assumes that the applicant is the main view structure, and the inventor and subject graph network are auxiliary graph structures. The present invention constructs a self-supervisory signal between the main view and the other two relationship models to maintain the synergy of the same patent in different patent relationship model views, while mining the commonalities of different patent relationship models, and enhancing the robustness of high-value patent mining.

[0114] In order to ensure that the patent node features in different relationships are in the same contrast space dimension, the present invention first uses an MLP (multi-layer perceptron) with two hidden layers to map these features to a unified contrast space dimension. The specific formula is as follows:

[0115]

[0116]

[0117]

[0118] Secondly, in order to set the positive and negative sample relationship matrix under different patent graph comparative learning, the present invention takes the neighbor patents of the target patent as positive samples under the applicant relationship, because these patents are highly correlated with the target patent under the semantics of the meta-path. In the co-inventor relationship mode, the neighbor patents of the target patent are also taken as positive samples, including its neighbor patents in the inventor view. Therefore, the present invention constructs the patent-applicant-patent graph adjacency matrix A PTPand the patent-inventor-patent graph adjacency matrix A PIP Construct the positive and negative sample matrix, and its calculation formula is:

[0119]

[0120] in The dimensions of both are the same. Indicates the addition of elements, thereby maintaining A PTP and A PTP The synergy between the two.

[0121] Similarly, the present invention can also be found in Patent-Applicant-Patent A PTP and the patent-topic-patent relationship graph adjacency matrix A PTP Construct the positive and negative sample matrix, and its calculation formula is:

[0122]

[0123] After constructing the above multi-relationship positive and negative sample matrix, the present invention uses the subject relationship and inventor relationship to assist the applicant view embedding, forming a synergistic reinforcement under the three relationship modes. Since there is an intersection between the positive and negative sample relationships between the three relationships, the present invention ignores the relationship superscript and constructs a contrastive learning loss, the formula of which is as follows:

[0124]

[0125] in represents the sum of graph contrast enhancement losses in the patent multi-relation mode. The sim(·) function is used to estimate the similarity of positive and negative pairs between different patent relations. τ is the temperature parameter. is the set of positive sample pairs, is the set of negative sample pairs.

[0126] By comparing the patent application view with the inventor and subject text patent views and cooperating with each other for mutual supervision, the present invention aims to narrow the distance between positive samples and widen the distance between negative samples. The three relationship modes cooperate with each other to optimize the learning process of patent embedding and explore the potential common relationships in the embedding features of different views, thereby improving the robustness of the model.

[0127] Multi-relation perception multi-head attention mechanism fusion module such as Figure 5 As shown, the input of this module is the embedded representation under different patent relations. Next, the present invention designs a multi-relation-aware self-attention mechanism encoder to capture the dependency of patent value on different relations. The encoder is composed of L stacked multi-head self-attention (MSA) and feed-forward network (FFN) blocks.

[0128] The relationship between inventors and applicants is used as the input of Key and Value, aiming to learn the dependency of patents in different relationship environments. The calculation process is as follows:

[0129]

[0130] FFN(O,W1,b1,W2,b2)=GELU(OW1+b1)W2+b2,#(15)

[0131] in They respectively represent that the present invention introduces relational perception signals to enhance the model's understanding of different relational patent embeddings. Finally, the self-attention mechanism is used to capture the patent dependency relationship in the patent. The calculation process is as follows:

[0132]

[0133]

[0134]

[0135] in, represents the learnable weight matrix, and L represents the number of multi-head attention.

[0136] The perceptual representation of patent node i is obtained by averaging their It is derived from the representation in .

[0137]

[0138] Where W out is the linear transformation matrix, b out is a trainable parameter.

[0139] Furthermore, in order to better carry out model training, the present invention adopts a binary cross entropy loss function for binary classification problems. Its formula is the same as the binary cross entropy loss function mentioned above:

[0140]

[0141] in, Represents the value of the loss function, y represents the vector of the true label, whose elements are 0 or 1. p represents the probability vector predicted by the model, that is, the original predicted value output by the model without being processed by the sigmoid function, and log represents the natural logarithm function.

[0142] The final loss function of the obtained model is: The final optimization loss is the sum of the binary cross entropy loss function and the multi-relationship enhanced contrast loss, that is,

[0143]

[0144] Among them, λ is a hyperparameter for regulating contrastive learning loss, θ is a trainable parameter in the model, and ξ is a regularization coefficient.

[0145] In addition, in order to verify the effect of the model and improve the training and generalization ability of the classification model, the present invention optimizes the parameter values ​​in each model. During each training, the present invention randomly extracts 60% for training, 20% for verification, and 20% for testing to split the data set. In addition, a confusion matrix is ​​used to measure the performance of the model, and the indicators include accuracy, precision and recall. Since the two indicators of accuracy and recall affect each other, the present invention introduces the F1 score into these two indicators.

[0146] Table 1 True values ​​and predicted values ​​in the confusion matrix

[0147]

[0148] TP (True Positives): True positives, that is, positive examples are predicted to be true (predicted to be positive and actually positive);

[0149] FP (False Positives): False positive examples, that is, negative examples are predicted to be true (predicted as positive examples but actually negative examples);

[0150] FN (false Negatives): False negative examples, that is, positive examples are predicted to be false (predicted as negative examples but actually positive examples);

[0151] TN (True Negatives): True negative examples, that is, negative examples are predicted to be false (predicted as negative examples and are actually negative examples).

[0152] Accuracy: The proportion of samples with correct predictions (including positive or negative examples predicted correctly, that is, positive examples predicted as positive TP or negative examples predicted as negative TN) to the total samples.

[0153]

[0154] Recall rate: The ratio of predicted true positive examples (TP) to all actual positive examples. The actual samples are used as the basis for judgment. Among the actual positive examples, the ratio of the predicted correct positive examples to the total actual positive examples is used.

[0155]

[0156] F1-score: Comprehensive consideration of accuracy and recall:

[0157]

[0158] AUC: (Area Under Curve): is an evaluation indicator to measure the quality of the binary classification model:

[0159]

[0160] Among them, TPR is the true positive rate; FPR is the false positive rate.

[0161] Example 2

[0162] In another embodiment of the present invention, an experiment of a high-value patent identification method with multi-relationship graph contrast enhancement is provided.

[0163] Step 1: Dataset

[0164] To achieve this goal, the present invention constructs a humanoid robot technology decomposition table based on expert knowledge in the field of robotics, and formulates a retrieval strategy accordingly. The search formula used was "((humanoid robot OR "Humanoid robot" OR humanoid robot OR Humanoid OR humanoid robot OR "Humanoid robot" OR humanoid robot OR "Humanoidrobot") / tac OR (((frameless torque motor OR harmonic reducer OR contactless torque sensor OR position sensor OR mechanical clutch OR crossed roller bearing OR angular contact bearing) or (frameless torque motor OR planetary roller screw OR force sensor OR position sensor OR ball bearing OR four-point contact bearing) or (coreless cup motor OR precision reducer OR worm gear OR rope drive)) AND (camera OR fisheye camera OR millimeter wave radar) and (controller) and (battery pack OR thermal management OR structural parts)) / tac". This search formula was applied to the patent database HimmPat for retrieval on July 10, 2024, and a total of 10,955 relevant data were obtained. After data processing such as application number merging, 8,025 patent data were finally selected as the data set for model training and testing.

[0165] Constructing a scientific and reasonable evaluation index system is the key to identifying high-value patents. Scholars have conducted in-depth research on this. According to comprehensive analysis, existing research can be divided into three categories. The first category of research constructs a high-value patent identification index system from the three dimensions of high-level technology research and development, high-quality patent confirmation and high-return transformation and application. This index system uses machine learning algorithms such as support vector machine (SVM), neural network and adaptive enhancement to achieve the identification of high-value patents. However, this index system has not fully considered the economic value dimension of patents. To solve this problem, some scholars have constructed patent economic value evaluation indicators based on PCT applications and patent families, patent transfer / licensing, patent pledge, survival period, etc., to capture the economic attribute characteristics of patents from multiple angles, and use them for the training of machine learning or deep learning models to achieve the identification or prediction of high-value patents. However, the high-value characteristics of patents need to be fed back through market applications to a large extent.

[0166] Table 2 High-value patent characteristic index system

[0167]

[0168] To this end, some scholars have proposed market dimension evaluation indicators based on patents, such as market share and expansion degree, as well as subject dimension evaluation indicators based on patent holders, number of inventors, etc.

[52] . This study adopts the combination of multi-dimensional value indicators shown in Table 2, and uses graph neural networks to model the multi-relationship network of patents, thereby achieving the task of identifying high-value patents.

[0169] In order to realize the encoding of patent features, based on the above 17 secondary indicators, the present invention uses the query-based approach of the Pytorch framework to construct the initial vectors of 17 different indicators, and the 17 initial vectors corresponding to each patent are averaged and fused to embed the initial representation of the patent. In view of the low proportion of high-value patents in patent clusters, this study randomly shuffled the data set in the semi-supervised high-value patent identification task and divided it into 60% for training, 20% for verification, and the remaining 20% ​​for testing. In order to verify the effectiveness of the method proposed in this invention, the authoritative humanoid robot data sets were selected: Human_robot_patent and B. In the Human_robot_patent data set, 17 patent-related features were considered, including the number of IPC classifications, the number of citations, the classification of strategic emerging industries, and the classification of patent-intensive industries.

[0170] Table 3 Dataset information

[0171]

[0172] Step 2: Experimental Setup

[0173] This paper compares the proposed method with two machine learning models and seven deep learning baseline models. The experimental environment is the same as this study, and fine parameter optimization is performed. Graph Isomorphism Network (GIN), Multi-Layer Perceptron (MLP), Graph Convolutional Network (GCN), Graph Sample and Aggregation Neural Network (GraphSAGE), RECT_L, eXtreme Gradient Boosting (XGBOOST), Light Gradient Boosting Machine (LightGBM), Probabilistic Random Walk (Node2Vec), and Lightweight Graph Convolutional Network (LightGCN). The baseline models are summarized as follows:

[0174] MLP: A basic feedforward neural network consisting of an input layer, a hidden layer, and an output layer, with each layer containing multiple neurons.

[0175] GIN: A graph neural network consisting of multiple layers of graph convolution, each layer updates the features of nodes to distinguish different graph structures.

[0176] GCN: A graph neural network that learns embedded representations of nodes by applying convolution operations on a graph. It exploits the local connectivity patterns of the graph to aggregate features of neighboring nodes to update the feature representation of each node.

[0177] GraphSAGE: A flexible graph neural network framework designed to efficiently learn features of nodes in graph data. It updates the embedding representation of each node by sampling and aggregating features of neighboring nodes.

[0178] RECT-L: A graph neural network that learns node representations by recursively aggregating node neighbors’ features

[0179] XGBoost: A gradient boosting algorithm for classification and regression problems known for its speed and accuracy.

[0180] LightGBM: A decision tree-based learning algorithm that uses tree-based gradient boosting techniques.

[0181] Node2Vec: A random walk-based graph embedding technique for exploring and learning low-dimensional vector representations of nodes in a graph.

[0182] LightGCN: A simplified graph convolutional network that discards linear transformations and activation functions based on GCN to quickly learn embedded representations of nodes.

[0183] Given that different comparison models have their own unique advantages and limitations in patent data processing and high-value patent (HVP) identification, this study carefully selected the following baseline models for comparison. By comparing these models, it aims to evaluate the performance and efficiency of the MRGCE method proposed in this study in the task of patent value identification.

[0184] Table 4 Comparison of baseline models

[0185]

[0186] The present invention successfully implements the proposed MRGCE model using the Pytorch framework, and carefully adjusts the key parameters to optimize its performance. Specifically, the Adam optimizer is used to iteratively optimize all model parameters, and the batch size is set to 1024, the learning rate is set to 0.001, and RTX4090 is used as the training device. On the Human_robot_patent dataset, the embedding dimension of the model input and output is set to 64. In the experimental verification part, the number of layers of the graph attention network is discussed, and the optimal number of layers l of GCN is found in the range of {1, 2, 3, 4, 5}. In addition, the value of the important coefficient λ in the contrast loss function is adjusted to observe its impact on the performance of the model, and its range of variation is from 0.1 to 0.9. In order to ensure the fairness of the experiment, the Baseline model is tested under the same experimental conditions to ensure that the batch size and learning rate are consistent with the MRGCE model, both of which are 1024 and 0.001.

[0187] Step 3: Method performance analysis

[0188] This paper evaluates the proposed high-value patent recognition model and compares its performance with the baseline model on the Human_robot_patent dataset. Table 5 summarizes the performance of nine baseline models based on multidimensional patent indicators in terms of key evaluation indicators such as accuracy, recall, F1 score and AUC.

[0189] Table 5 Performance of the proposed model and the baseline model on the Human_robot_patent dataset

[0190]

[0191] On the Human_robot_patent dataset, the MRGCE models proposed in this invention all show excellent performance, which fully demonstrates the effectiveness of the high-value patent mining framework with multi-relationship graph contrast enhancement proposed in this invention. Next, this invention will conduct an in-depth analysis of the experimental results to reveal the advantages of the MRGCE model and the principles behind it.

[0192] The XGBoost model is sensitive to outliers in the data, which may affect its performance in patent data, especially during the splitting process of the tree, where the presence of outliers may cause the performance of the model to deteriorate. In contrast, the LightGBM model directly supports categorical features, which simplifies the process of processing categorical data and makes LightGBM more flexible when processing mixed data types. In addition, LightGBM introduces a variety of optimization algorithms, such as the Histogram algorithm, the GOSS algorithm, and the EFB algorithm. The combination of these algorithms effectively reduces the computational complexity and memory consumption and improves the training efficiency of the model. However, the sensitivity of LightGBM to noise is also an issue that needs to be considered. Node2Vec captures the structural information of patents in the graph network through random walks and encodes patent features. However, it relies entirely on unsupervised learning, which may lead to performance bottlenecks when processing complex patent networks.

[0193] The MLP model identifies the value of patents through nonlinear mapping, but this method may ignore the complex relationships between patents. In contrast, graph neural networks such as GCN, GraphSAGE and LightGCN can effectively propagate and aggregate between patent graph structures, but they have limitations in handling multiple relationships between patents and it is difficult to mine the commonalities under different relationship patterns.

[0194] MRGCE achieved the best experimental results in Accuracy, Recall and F1-score, and was only slightly lower than LightGBM in AUC. This study successfully applied the graph neural network method under the multi-relational model to high-value patent identification, providing a new solution for research in this field.

[0195] After rigorous empirical analysis and verification, the method proposed in this study shows significant advantages in evaluating patent validity. Compared with traditional patent evaluation methods, this method not only improves the accuracy and efficiency of evaluation, but also more accurately identifies patents with potential high value. Experimental results show that this method is highly reliable in predicting the commercial success and technological influence of patents, providing strong support for enterprises and R&D institutions in patent investment decisions, thereby promoting the sustainable development of patent technology transfer and innovation.

[0196] Example 3

[0197] In addition, the present invention also analyzes the impact of different patent relationships on the identification of high-value patents, as well as the impact of edge relationship combinations, as shown in Table 6.

[0198] Table 6 Experimental verification of multiple relationships in complex patent networks

[0199]

[0200] Figure 6 The performance impact of the MRGCE model on different patent relationships in the Human dataset is intuitively demonstrated. When only a single relationship is considered, the applicant relationship performs best. This may be because applicants usually have a high degree of independence and importance in patent documents. In contrast, the performance of the patent topic text network built based on the BERT model by embedding patent topics into a low-dimensional space and similarity calculation is slightly insufficient. This process may introduce noise, thereby affecting the recognition accuracy of topic relationships.

[0201] When considering two patent relationships, the combination of the applicant and inventor relationships performs best, indicating that the two relationships complement each other in the model and jointly promote performance improvement. Although the combination of the applicant and subject relationships is slightly inferior, it is still better than the combination of the inventor and subject. This shows that among patent relationships, the applicant relationship has the highest value, followed by the inventor relationship, while the subject relationship has a relatively low value. The MRGCE model proposed in the present invention achieves the best overall performance by comprehensively considering these three relationship patterns. The MRGCE model uses graph contrast enhancement technology to strengthen the commonalities between different relationship patterns, so that the three relationships can optimize each other through self-supervisory signals. In addition, MRGCE also adopts a multi-relationship-aware self-attention mechanism, which can adaptively fuse the feature signals of the three relationship patterns, thereby maximizing the performance of the model.

[0202] Example 4

[0203] In order to explore the influence of attributes on the model recommendation performance of each module, this paper ablates two modules, namely, image contrast learning and multi-relation-aware attention mechanism. The experimental results are shown in Table 7:

[0204] Table 7 Experimental verification of MRGCE model module ablation

[0205]

[0206] Based on the multi-relationship pattern between patents, this paper constructs a contrastive learning enhancement strategy under a multi-relationship graph network, effectively models the multi-relationship characteristics between patents, and successfully achieves the identification task of high-value patents. At the same time, a combined feature set based on 17 secondary indicators is constructed, which avoids the complex calculation and manual evaluation of traditional multi-features, and can automatically and scalably realize the identification of high-value patents with a small number of samples. An empirical study was carried out on the patent dataset in the field of humanoid robots. Through comparative experimental analysis, it was found that the method proposed in this paper improved the accuracy (ACC) by 15.12%, the recall rate (Recall) by 7.59%, the normalized discounted cumulative gain (NDCG) by 3.57%, and the area under the curve (AUC) also showed excellent performance, confirming that the MRGCE model is highly robust in automatically identifying high-value patents.

[0207] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A high-value patent identification method with multi-relationship graph contrast enhancement, characterized in that: include: S100: Establish applicant network and inventor network through common relationship connection, calculate patent text similarity based on text embedding generated by BERT pre-training model, build patent text topic network, and then build complex heterogeneous relationships of patent network; S200: Use graph attention networks to learn the importance relationships of different patent neighbors under different relationship patterns, and construct graph contrast learning between different relationships to form collaborative supervision signals, explore the commonalities of different relationship patterns, and enhance the robustness of high-value patent mining; S300: Learn the embedded representations of different patent relationship modes through the attention network, and establish supervision signals between different relationships to achieve the coordination and enhancement of patent features in different relationship views; S400: Taking the applicant relationship embedding as the query and combining the inventor and subject relationship as the perception embedding, a multi-relationship perception attention mechanism is constructed, which integrates the embedding representations under the patent applicant, inventor and subject relationship model. After determining the loss function, the model is trained on the dataset, followed by model performance evaluation and ablation experiments, and finally a high-value patent recognition algorithm model with multi-relationship graph contrast enhancement is obtained.

2. The high-value patent identification method with multi-relationship graph contrast enhancement according to claim 1 is characterized in that: Step S100: Constructing a patent-applicant interaction matrix and Patent-Inventor Interaction Matrix They are: in represents the patent relationship matrix under the meta-path relationship patent-applicant-patent, Represents the patent relationship matrix under the meta-path patent-inventor-patent.

3. The high-value patent identification method with multi-relationship graph contrast enhancement according to claim 2 is characterized in that: Constructing the patent text subject network in step S100 includes: S101: Construct fixed-dimensional token vectors and position encodings for patent subject sentence segmentation to determine the order of sentence context; S102: Use the multi-head self-attention mechanism to fuse the sentence context information, and its output is: where Q, K and V represent the query, key and value matrices respectively.

4. The high-value patent identification method with multi-relationship graph contrast enhancement according to claim 3 is characterized in that: Constructing the patent text topic network in step S100 also includes: using cosine similarity to measure the correlation between different patents: in, and denote the text topic embedding representation of patent i and patent j respectively, sim(·) denotes the cosine similarity function, represents the threshold set in this paper to determine whether there is an interaction boundary between patents. PAP represents the adjacency matrix between text topics between patents, A PTP is a normalized topic similarity matrix. The larger the value, the stronger the topic relevance of the corresponding patent.

5. A high-value patent identification method with multi-relationship graph contrast enhancement according to any one of claims 1-4, characterized in that: The importance of patent neighbors in step S200 includes: Given a target patent node i and its neighbor patent node set under relationship r Adaptively learn and calculate the attention coefficient between the target patent and its neighbor patents, where Indicates the importance of neighbor patent j to patent i. After calculating the importance of neighbor patent nodes to the target patent node, normalization is performed through the softmax function to obtain the weight coefficients of the target patent and neighbor patents. The attention coefficients between different patents are as follows: Where σ represents the ReLu activation function, l| represents the concatenation operation, is the node-level attention vector, j is the neighbor patent node of the target patent, represents the neighbors of patent i under relation r, is a collection of patent relations.

6. The high-value patent identification method with multi-relationship graph contrast enhancement according to claim 5 is characterized in that: The importance of patent neighbors in step S200 also includes: Set K attention heads to the importance of neighboring patents, and the embedding of target node i is aggregated through the projected features of neighboring nodes and the corresponding weight coefficients, as shown below: in Represents the embedded representation of patent i after aggregation of patent neighbors in the rth relation mode.

7. The high-value patent identification method with multi-relationship graph contrast enhancement according to claim 6 is characterized in that: The image comparison learning in step S200 includes: S201: Use a multi-layer perceptron with multiple hidden layers to map these features to a unified contrast space dimension, as follows: Among them, W is the weight and b is the coefficient.

8. The high-value patent identification method with multi-relationship graph contrast enhancement according to claim 7 is characterized in that: The image comparison learning in step S200 includes: S202: In the applicant relationship, the neighboring patents of the target patent are taken as positive samples; in the co-inventor relationship model, the neighboring patents of the target patent are taken as positive samples, including its neighboring patents in the inventor view.

9. The high-value patent identification method with multi-relationship graph contrast enhancement according to claim 8 is characterized in that: The image comparison learning in step S200 includes: S203: Construct the patent-applicant-patent graph adjacency matrix A PAP and the patent-inventor-patent graph adjacency matrix A PIP Construct the positive and negative sample matrix as: in The dimensions of both are the same. Indicates the addition of elements, thereby maintaining A PAP and A PIP The synergy between the two.

10. The high-value patent identification method with multi-relationship graph contrast enhancement according to claim 9 is characterized in that: The image comparison learning in step S200 includes: S204: In Patent-Applicant-Patent A PAP and the patent-topic-patent relationship graph adjacency matrix A PTP Construct the positive and negative sample matrix as:

11. The high-value patent identification method with multi-relationship graph contrast enhancement according to claim 10 is characterized in that: In step S200, the graph contrastive learning includes: using the subject relationship and the inventor relationship to assist the applicant view embedding, forming a synergistic reinforcement under the three relationship modes, thereby constructing the contrastive learning loss as follows: in represents the sum of graph contrast enhancement losses in the patent multi-relation mode. The sim(·) function is used to estimate the similarity of positive and negative pairs between different patent relations. τ is the temperature parameter. is the set of positive sample pairs, is the set of negative sample pairs.

12. A high-value patent identification method with multi-relationship graph contrast enhancement according to any one of claims 1-4, characterized in that: The multi-relation-aware attention mechanism in step S400 includes designing a multi-relation-aware self-attention mechanism encoder, which is composed of L multi-head self-attention and feed-forward network blocks stacked; Taking the relationship between inventor and applicant as the input of Key and Value, the calculation process is as follows: FFN(O,W1,b1,W2,b2)=GELU(OW1+b1)W2+b2 (15) in They respectively represent the relationship-aware signals introduced in this paper.

13. The high-value patent identification method with multi-relationship graph contrast enhancement according to claim 12 is characterized in that: The multi-relationship-aware attention mechanism in step S400 includes using a self-attention mechanism to capture patent dependencies, and the calculation process is as follows: in, represents the learnable weight matrix, L represents the number of multi-head attention, and the perceptual representation of patent node i is obtained by averaging them in is derived from the representation in; Where W out is the linear transformation matrix, b out is a trainable parameter.

14. The high-value patent identification method with multi-relationship graph contrast enhancement according to claim 13 is characterized in that: The loss function in step S400 is a binary cross entropy loss function. in, represents the value of the loss function, y represents the vector of the true label, whose elements are 0 or 1; p represents the probability vector predicted by the model, that is, the original prediction value output by the model without being processed by the sigmoid function, and log represents the natural logarithm function.

15. The high-value patent identification method with multi-relationship graph contrast enhancement according to claim 14 is characterized in that: The final optimization loss of the loss function in step S400 is the sum of the binary cross entropy loss function and the multi-relation enhancement contrast loss, that is: Among them, λ is a hyperparameter for regulating contrastive learning loss, θ is a trainable parameter in the model, and ξ is a regularization coefficient.

16. A high-value patent identification system with multi-relationship graph contrast enhancement, characterized in that: include: Complex heterogeneous relationship building module, which is used to establish applicant networks and inventor networks through common relationship connections, calculate patent text similarity based on text embedding generated by the BERT pre-training model, build patent text topic network, and then build complex heterogeneous relationships of patent networks; Graph contrast learning module, which is used to learn the importance relationship of different patent neighbors using graph attention network under different relationship modes, and construct graph contrast learning between different relationships to form collaborative supervision signals, explore the commonalities of different relationship modes, and enhance the robustness of high-value patent mining; Patent relationship graph attention fusion module, which is used to learn the embedding representations of different patent relationship modes through the attention network and establish supervision signals between different relationships to achieve the coordination and enhancement of patent features in different relationship views; The multi-relationship-aware attention mechanism fusion module is used to use the applicant relationship embedding as the query and combine the inventor and topic relationship as the perception embedding to construct a multi-relationship-aware attention mechanism, which integrates the embedding representations under the patent applicant, inventor and topic relationship model. After determining the loss function, the model is trained on the dataset, followed by model performance evaluation and ablation experiments, and finally a high-value patent recognition algorithm model with multi-relationship graph contrast enhancement is obtained.

17. A terminal, characterized in that: Memory for storing computer programs; A processor, configured to implement the steps of the processing method according to any one of claims 1 to 15 when executing the computer program.

18. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the processing method according to any one of claims 1 to 15 are implemented.

Images