Patent quality measuring and calculating method and system based on patent text similarity

Abstract

The invention provides a patent quality measurement and calculation method and system based on patent text similarity, and belongs to the technical field of patent quality measurement and calculation, and the method comprises the steps: calculating the word frequency-reverse document frequency of each vocabulary in a to-be-measured patent; a patent text vocabulary high-dimensional matrix is constructed on the basis of a word frequency-reverse inverse document frequency, a vector in the pth row of the matrix represents a word vector of a patent, the length of the vector is the length of a union set of all patent text vocabularies, for vocabularies existing in the patent, the value of a corresponding vector element is the word frequency-reverse inverse document frequency, and the value of a corresponding vector element is the word frequency-reverse inverse document frequency. The value of the vocabulary which does not exist in the patent is zero; the vocabulary vector is standardized by using the module of the vocabulary vector, the similarity between any two patent pairs is the cosine similarity of the two patents, and the larger the value is, the higher the similarity of the two patents is; defining an average reverse similarity index of the patents and an average forward similarity index of the patents based on the similarity; defining relative text similarity coefficients of different technical fields relative to all the technical fields based on average reverse similarity of patents; the product of the ratio of the forward similarity index to the reverse similarity index of the patent and the similarity coefficient is the patent quality.

Description

Technical Field

[0001] This invention belongs to the field of patent quality assessment technology, and in particular relates to a patent quality assessment method and system based on patent text similarity. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] The novelty of a patent refers to the degree to which a patent contains new knowledge that has not appeared in existing historical patents. In other words, the lower the similarity between a patent and an existing patent, the more novel the patent is.

[0004] Judging patent quality solely based on the novelty of knowledge may have the problem of failing to fully and effectively depict the true market value and innovative impact of a patent. In particular, cutting-edge technological innovations may not readily connect with market demands or achieve effective commercialization, potentially leading to a situation where the technology is "too sophisticated for the masses." In such cases, although the patent may demonstrate high novelty, the lack of market acceptance and commercialization capabilities in the short term may result in a low short-term market value. Therefore, it is necessary to introduce indicators of patent impact into the assessment of patent quality.

[0005] In existing technologies, the calculation of patent quality mainly includes measurements based on patent citation data, economic effects, patent scope, and patent terminology. Measurements based on patent citation data may not reflect the true flow of knowledge, as they are essentially retrospective and historical studies. Furthermore, patent citation data is easily affected by institutional adjustments and changes in writing standards, making it difficult to accurately reflect changes in patent quality itself. Measurements based on patent economic effects only apply to listed companies and cannot cover all patents from private enterprises, research institutions, and government departments. Measurements based on patent scope may be problematic due to patent applicants using broad claims and vague patent terminology for rent-seeking, failing to accurately reflect the novelty of innovative knowledge. Measurements based on patent terminology may not fully reflect the actual technical content of a patent; focusing solely on the terminology of the patent name cannot reflect the true technical characteristics of the patent. Therefore, the above techniques are insufficient for accurately measuring patent quality levels. Summary of the Invention

[0006] To overcome the shortcomings of the prior art, this invention provides a patent quality assessment method based on patent text similarity. This method is based on natural language processing technology and is applicable to the information features of patent text, resulting in high accuracy.

[0007] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: Firstly, a patent quality assessment method based on patent text similarity is disclosed, including: Calculate the term frequency-inverse inverse document frequency for each word in the patent to be tested; A high-dimensional matrix of patent text vocabulary is constructed based on word frequency-inverse inverse document frequency. The vector in the p-th row of the matrix represents the word vector of the patent, and its length is the length of the union of all patent text vocabulary. For words that exist in the patent, the corresponding vector element takes the value of word frequency-inverse inverse document frequency, while for words that do not exist in the patent, the value is zero. By using the model pairing of word vectors for word vector standardization, the similarity between any two patent pairs is the cosine similarity between them. The larger the value, the higher the similarity between the two patents. Based on similarity, the reverse similarity index and the forward similarity index of patents are defined. The ratio of a patent's forward similarity index to its backward similarity index represents the patent's quality.

[0008] As a further technical solution, the word frequency-inverse inverse document frequency of each word is defined as the product of the frequency of the word appearing in the text of the patent and the inverse inverse document frequency.

[0009] As a further technical solution, the reverse inverse file frequency The definition of is: .

[0010] As a further technical solution, it also includes: standardizing the similarity of patents to obtain the average forward similarity and backward similarity of patents, and defining patent quality as the ratio of the average forward similarity and the average backward similarity of patents.

[0011] As a further technical solution, patent applications from the same applicant were excluded when calculating the forward similarity, backward similarity, or average forward similarity and backward similarity of patents.

[0012] As a further technical solution, when calculating patent quality, different combinations of forward similarity across IPC domains and backward similarity within the same IPC domain are used.

[0013] As a further technical solution, the calculation of patent quality includes: Define the average reverse similarity and average forward similarity of patents; Based on the defined average reverse similarity and average forward similarity of patents, the relative text similarity coefficients of different technical fields relative to all technical fields are defined; Patent quality is calculated based on the average reverse similarity and average forward similarity of patents, as well as the relative text similarity coefficients of different technical fields relative to all technical fields.

[0014] Secondly, a patent quality assessment system based on patent text similarity is disclosed, including: The term frequency-inverse inverse document frequency calculation module is configured to calculate the term frequency-inverse inverse document frequency of each word in the patent to be tested; The high-dimensional matrix construction module is configured to construct a high-dimensional matrix of patent text vocabulary based on word frequency-inverse inverse document frequency. The p-th row vector of the matrix represents the word vector of the patent, and its length is the length of the union of all patent text vocabulary. For words that exist in the patent, the corresponding vector element takes the value of word frequency-inverse inverse document frequency, while for words that do not exist in the patent, the value is zero. The similarity calculation module is configured to: use the module of word vectors to standardize word vectors, and the similarity between any two patent pairs is the cosine similarity between the two, with a larger value indicating a higher similarity between the two patents; The patent quality calculation module is configured to: define the reverse similarity index and the forward similarity index of the patent based on similarity. The ratio of a patent's forward similarity index to its backward similarity index represents the patent's quality.

[0015] The above one or more technical solutions have the following beneficial effects: Based on a hybrid similarity index of "forward similarity across all categories + backward similarity within the same category," and incorporating weighted text similarity based on part + two-digit IPC grouping for correction, this scheme both considers cross-domain technology spillover and accurately captures the depth of innovation within its own domain. It effectively overcomes common challenges in Chinese patents, such as terminology ambiguity, word segmentation difficulties, and unclear technology transfer paths, significantly improving the applicability and interpretability of Chinese patent quality metrics. It can provide a fundamental identification tool for effectively assessing the true market value and independent innovation capabilities of patents held by various micro-entities.

[0016] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0017] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0018] Figure 1 This is a flowchart of the method according to an embodiment of the present invention; Figure 2A schematic diagram showing the distribution of the full sample and the sampled labeled sample in the application year, and the distribution of the full sample and the sampled labeled sample in the IPC classification. Figure 3 A schematic diagram showing the changes in the loss function of the training and validation sets during the training period, as well as the trends of the macro F1 score and Pearson correlation coefficient; Figure 4 This is a diagram illustrating the similarity levels (classes 1 to 5) of the best model on the test set. Figure 5 This is a schematic diagram showing the distribution of average forward similarity and average backward similarity. Figure 6 This is a schematic diagram showing the distribution of average forward similarity and average backward similarity for each IPC category; Figure 7 This is a schematic diagram illustrating the correlation between average forward similarity and average backward similarity. Figure 8 A schematic diagram illustrating the correlation between the average forward similarity and the average backward similarity of IPC categories; Figure 9 This is a schematic diagram of the Chinese-RoBERTa-wwm-ext structure. Detailed Implementation

[0019] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0020] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0021] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0022] Patents are widely considered important carriers of new knowledge and technologies. The knowledge and technical features contained in a patent need to be clearly identified and effectively defined in the description and definition within the patent text. A basic judgment can be drawn from this: if a patent shows a high degree of similarity to the innovative knowledge and technical content of existing patents, the innovative content contained in that patent is likely closer to incremental innovation. Conversely, if the relevant knowledge and technology in a patent text show a significant difference compared to existing patents, demonstrating a high degree of novelty, the technological innovation contained in that patent is likely more groundbreaking. Such patents are likely to bring greater market value and economic benefits to enterprises in the long run.

[0023] Analogous to the novelty index of a patent, the impact index of a patent describes the degree to which a patent influences subsequent patents. Influential patents are often those that provide a foundation for incremental innovation. Following similar logic, if subsequent patents are more similar to a published patent, the greater the patent's impact on subsequent innovation, and thus, given other conditions, the higher the patent's quality. In summary, patent novelty describes the innovativeness of the knowledge and technology contained in the patent, while patent impact reflects the spillover effect of that patent knowledge on subsequent innovation. By considering both "novelty" and "impact" as indicators, we can comprehensively characterize the technical level of the innovative knowledge contained in a patent, thus providing a more complete picture of the patent's true quality.

[0024] One of the core aspects of patent text analysis is calculating the similarity of innovative knowledge and technical processes among different patent texts. To achieve this goal, for each patent text, we will calculate its "novelty" by comparing it with historical patent texts, and its "influence" by comparing it with future patent texts. A higher similarity between a patent text and historical patent texts indicates lower novelty, while a higher similarity between a patent text and subsequent patent texts indicates higher influence.

[0025] Due to the inherent semantic diversity of text, textual data naturally possesses the characteristics of high-dimensional data. To reflect the uniqueness of specialized vocabulary in patent texts, the classic "one-hot encoding" method from natural language processing is employed to construct word vectors. The basic idea of ​​one-hot encoding is to assign a unique code to each word, with each word occupying a single dimension of the word vector. The biggest challenge in analyzing massive amounts of patent text using one-hot encoding stems from the high-dimensionality of the word vectors, which places extremely high demands on computational power. Although different nouns in patent texts each have a unique position in the word vectors, it must be pointed out that different words reflect different materials, technologies, and processes, and their importance and relevance to the core innovative technologies contained in the patent are not entirely the same. For example, nouns closely related to the invention patent technology, such as "5G" and "lithium battery," more accurately reflect the technical characteristics and degree of innovation of the patent than general terms such as "material" and "structure." Therefore, they should be given higher weight in the calculation of similarity between patents to reflect the importance of groundbreaking vocabulary and related concepts.

[0026] Example 1 See appendix Figure 1 As shown, this embodiment discloses a patent quality assessment method based on patent text similarity, including: Step 1: Calculate patent p Term frequency - inverse document frequency Its definition is:

[0027] in, Indicator word " "The frequency of occurrence in the text of patent p is the term frequency commonly used in text analysis."

[0028] Defined as:

[0029] In the above formula, Indicates patent p Number of times it appears In patent p

[0030] The backward inverse document frequency (BIDF) is defined as follows: (3) Unlike inverse document frequency (IDF) used in traditional text analysis, BIDF assigns higher weight to groundbreaking technologies in inventions. For example, consider the significant technological innovation of 5G communication technology. Before 5G, very few patents contained the term "5G," but after the publication of 5G patents, subsequent patents began using this term very frequently. Note that according to the traditional IDF definition, highly innovative terms like "5G" would be assigned lower weights, potentially leading to an incorrect assessment of the patent's novelty.

[0031] Step Two: Constructing a high-dimensional matrix of patent text vocabulary .matrix p Let p represent the word vector, whose length is the length of the union of all patent text words. For the words present in this patent... The corresponding vector element takes the value For words that do not exist in the patent, the value is zero.

[0032] It should be noted that the high-dimensional patent text vocabulary matrix T essentially transforms each patent text into a vocabulary frequency vector, primarily serving to unify the analysis dimensions and preserve semantic features. Regardless of the technical field of the patents or the differences in text length, the high-dimensional patent text vocabulary matrix T can be used to transform them into numerical vectors of the same dimension, achieving comparability of patents across periods and fields. Furthermore, by filtering highly diagnostic terms, such as industry-specific terms and technical keywords, this matrix can capture the core technical content of the patents, providing a quantitative basis for subsequently measuring the similarity between the patents and their predecessors and successors.

[0033] Step 3: Utilize the model of vocabulary vectors Standardize the word vectors. The similarity between any two patent pairs (p, q) is their cosine similarity: Its value range is The larger the value, the more patent p The higher the q similarity, the better.

[0034] After calculating the similarity between patents, two dimensions of indicators can be defined: novelty and influence. Specifically, the novelty of patent p filed in a certain year can be defined as the backward similarity indicator:

[0035] in Indicates the time before the invention patent application p. A collection of patent applications filed within the year. If The smaller the value, the lower the similarity between the patent and existing patents, indicating that the patent is more novel.

[0036] The influence of a patent can be characterized by its forward similarity metric.

[0037] in, Indicates the future when patent application p is filed. A collection of patent applications filed within the year. If... The larger the value, the more likely the patent is to influence subsequent patents, and thus the greater its influence.

[0038] The KPST method defines patent quality as:

[0039] This calculation formula shows that the higher the relative impact and novelty of a patent, the higher its quality.

[0040] However, for the purposes of this system's calculations, the KPST method may still have the following non-negligible problems: First, the rapid increase in the number of invention patents. Against the backdrop of this rapid change in the number of patents, simply using formula (6) can lead to biases in patent quality assessment. Note that both backward and forward similarity are sums of the similarities between patents. Given that the cosine similarity between patent texts is non-negative, the rapid increase in the number of patents over time tends to increase the forward similarity of patents from more recent years, thus causing an upward bias in the patent quality assessment results. Second, the widespread existence of incremental innovation. The widespread incremental innovation model plays a crucial role in technological development. This innovation model is not achieved overnight but gradually through continuous R&D investment and technological accumulation. This process is usually accompanied by the accumulation of a large number of related patents, which may be technologically interconnected, forming a gradual technological development path. This patent accumulation reflects the continuous R&D investment and technological accumulation of the innovation entity in a specific field and is an important part of the innovation process. Therefore, when assessing patent quality, the characteristics of this incremental innovation must be considered, especially when calculating forward and backward similarity. If patent applications from the same applicant are not excluded, it may lead to biases in the estimation of the quality of such cumulative technology patents. Third, comparisons within and across technology categories. Different patents contain different technical features, making direct comparisons between patents in different technical fields difficult. Theoretically, forward similarity between a patent and patents in different technical fields reflects the comprehensiveness of patent knowledge, while backward similarity better illustrates the universality of that patent knowledge across different fields. Conversely, forward similarity between a patent and other patents in the same technical field better reflects the patent's similarity within that field, while backward similarity indicates the patent's quality in a specific field. Therefore, researchers need to differentiate between cross-technology comparisons and comparisons within the same technical field, depending on the research objective. Fourth, the inherent differences in textual features of patents across different technical fields. Innovations in different technical fields will differ in the combination of professional terms, especially in some specialized fields where new and significant innovations manifest as breakthroughs through combinations of traditional concepts. Particularly in the field of "chemistry and materials," for example, "a process that can achieve superconductivity at room temperature"—room temperature and superconductivity are old concepts, but combining "room temperature" and "superconductivity" represents a significant technological breakthrough. This suggests that directly using text similarity to calculate patent quality without considering the differences in textual features across different technological fields may lead to a relative underestimation of patent quality in technological fields where conceptual combination innovation is prevalent, such as "chemistry" and "metallurgy." To address the potential problems of the KPST method, the patent quality measurement method has been adjusted and improved as follows: the specific steps are the same as the KPST method, but the calculation methods for average reverse similarity, average forward similarity, text similarity coefficient, and patent quality have been improved. Steps one through three are identical, and formulas 7-10 creatively replace formulas 4-6 of the traditional KPST method.First, to avoid the impact of the rapid increase in the number of patents on the similarity of patent texts themselves, we standardize the similarity of patents using the number of patents, obtaining the average forward similarity and backward similarity of patents. Patent quality is then defined as the ratio of the average forward similarity to the average backward similarity. Because this step eliminates the spurious inflation of patent quality caused by the rapid increase in the number of patents, it can more accurately depict the changes in patent quality over time. Second, to avoid the impact of continuous accumulation of innovation by the same applicant on the measurement of patent quality, we exclude patent applications from the same applicant when calculating the forward and backward similarity. To solve the problem of applicants with the same name, we combine the strings of "applicant + applicant's address (accurate to the street level)" to identify the same applicant. Third, to consider the different meanings of cross-technology comparisons and comparisons within the same technology field, this paper considers calculating forward and backward similarity of patents under both limited cross-patent technology classifications and same technology classifications. Because the span at the IPCSection level is too large, it is difficult to capture the technical differences between different sub-fields. Referring to the approach in the literature (e.g., Kogle et al., 2017), this paper adopts Section + Class / Subclass + MainGroup (three to four levels) as the 'same field' criterion. "Section + Main Group" can effectively reflect technical characteristics, avoid noise, and simultaneously consider statistical stability and field distinguishability. Specifically, to reflect the natural dependence of technological innovation on knowledge within its own field, and the cross-technology field influence that technological innovation possesses after completion, this paper attempts to combine different combinations of forward similarity across IPC fields and backward similarity within the same IPC field to calculate patent quality. Fourth, considering the differences in patent technology fields, especially the textual features of some technology fields with high similarity in vocabulary and concepts, the relative backward similarity of patent texts is standardized to account for the differences in the inherent characteristics of patent texts in different technology fields.

[0041] Define the average reverse similarity of patents and average forward similarity They are respectively:

[0042]

[0043] In the above expression, Indicates invention patent p Indicates invention patent p A collection of patent applications within the year, Indicates the future when patent application p is filed. A collection of patent applications filed within the year. and These represent the sets of patents removed based on the same applicant or patent IPC technology classification. Indicating that the patent sample has been granted The number of invention patents applied for each year.

[0044] To account for the inherent heterogeneity of patent text similarity across different technological fields, Indicates in The application filed that year falls under the patent technology category. The set of [texts] is defined as follows: the relative text similarity coefficient between different technical fields and all technical fields is:

[0045] in Indicates technology classification Down The total number of patents applied for in a year express Total number of patent applications in a year This represents the set of all patent technology fields.

[0046] Similarity coefficient Depicting the technical field The average backward similarity of patent texts is relative to the average similarity of all other technical fields. Under this adjustment factor, technical fields with high inherent similarity in their technical concept texts (such as chemical engineering and metallurgy) will naturally have a higher relative text similarity. Based on the average forward similarity, backward similarity, and technical field differences in patent texts, the final assessment of the technical field... patent p :

[0047] Based on the typical characteristic of "short life cycle" of Chinese patents, in the benchmark analysis, we selected... This involves calculating the similarity of patent texts using a set of invention patents from five years before and after the patent application date.

[0048] Experimental examples Before using the analysis model, it needs to be trained. The training process is explained below: Label Dataset Construction: To train models on a larger scale, more corpus data is needed. To ensure the representativeness of the training set samples, the following steps are taken to construct the training set: The first step involves using stratified random sampling to randomly select 10% of the samples from each major IPC category in each year as the core sample from the complete patent data. Based on this, another 1% of the patents from all patents in each year are randomly selected as the candidate pool, ensuring that these patents do not overlap with the core sample.

[0049] The second step involves generating patent text pairs based on the combination of core samples and candidate pool patents. For each patent in the core samples, it is paired with all patents in the candidate pool from the previous and subsequent five years (excluding the current year).

[0050] The third step, to ensure computational efficiency and maintain randomness, employs reservoir sampling to randomly select 1 million pairs from all possible patent pairings to construct the final dataset.

[0051] The final 1 million patent pairings included application numbers, application years, and IPC main classifications, and were used for the training and analysis of the pre-trained Chinese language model Chinese-RoBERTa-wwm-ex. Figure 2 The image shows a comparison of the distribution of the extracted core samples in terms of year and IPC classification with the complete patent data. It can be seen that the full sample and the labeled samples have highly similar distributions in both year and IPC classification.

[0052] Effective and accurate annotation plays a crucial role in model fine-tuning. The following strategy was employed to annotate and score a training dataset of 1 million patents. Considering the high cost and significant error associated with manual annotation, a state-of-the-art AI large language model was used to score the similarity of the patent texts. The prompt was: "Read and understand the following two patent texts and give a similarity score. Note that you should not analyze; simply answer with a score between 0 and 5." Furthermore, to ensure accuracy during the annotation phase and reduce annotation errors caused by the inherent illusions of large language models, high-scoring patent pairs underwent secondary verification and score adjustments.

[0053] After experimentation, we selected to use only the Kimi Large Language Model (moonshot-v1-8k) with the same prompt words to sample all high-scoring patent pairs (patent pairs with a score of 1 or higher) and then validate the low-scoring patent pairs (patent pairs with a score of 0). The results showed that, overall, the correlation between the annotation results based on the Qianwen Large Language Model and the Kimi Large Language Model was 79.25%.

[0054] Furthermore, comparing the annotations of the large language model with those of the manually annotated training set, the results show that the correlation between the scores of the Qianwen large language model, the Kimi large language model, and the manually annotated training set is 59.24% and 62.05%, respectively. This fully demonstrates the accuracy of the annotations. Since there are relatively few scores of 5 in the annotated samples, we merged the annotations with scores of 4 and 5. The final similarity score set is {0, 1, 2, 3, 4}, which has a clear hierarchical order, that is, there is a natural increasing meaning between different categories (e.g., categories 0 to 4 represent "dissimilar", "slightly similar", "moderately similar", "relatively similar", "highly similar", etc.). Therefore, the problem is modeled as an ordinal classification task, that is, the category labels themselves reflect a clear ranking relationship of strength and weakness.

[0055] Formal Model Training: For the pre-trained model, the Chinese-RoBERTa-wwm-ext pre-trained language model based on the RoBERTa architecture was adopted to implement the patent text similarity scoring task. This model, by introducing a Whole Word Masking (WWM) strategy, completely masks all the constituent characters of Chinese words during pre-training, enabling it to better capture the semantic information of Chinese text and effectively solve common problems of polysemy and context dependence in the Chinese domain. Simultaneously, the model's pre-training corpus covers rich Chinese text resources, including Wikipedia, news corpora, and professional vocabulary, demonstrating significant generalization ability. Furthermore, Chinese-RoBERTa-wwm-ext has shown leading performance in multiple Chinese natural language processing tasks, particularly excelling in semantic similarity recognition, text classification, and information extraction. Therefore, this study directly applies it to the patent text similarity scoring task, fully utilizing the language representation capabilities of the pre-trained model, reducing the fine-tuning costs required for model convergence, and effectively improving the accuracy and robustness of patent text similarity evaluation. In terms of fine-tuning, a dual-encoder architecture is employed for fine-tuning based on the Chinese-RoBERTa-wwm-ext pre-trained language model, which has demonstrated outstanding performance in the Chinese domain. This model independently encodes each of the two text segments in a patent text pair and predicts the similarity score between them by fusing the semantic vector representations of the two segments. The advantage of this architecture is that it significantly improves inference speed, reduces computational cost, and is more suitable for applications handling large-scale patent data.

[0056] In steps one to three of this implementation example, formulas 4-6 are not included in the model. After training, this model can output the similarity between any two text segments. Therefore, by inputting the patent text into the model, the similarity between the patent texts can be obtained, and then a patent quality index (i.e., formulas 7-10) can be constructed based on this similarity. That is, formula 7-10 replaces formulas 4-6 of the traditional KPST. To ensure that the model fully learns the complex semantic relationships of patent texts, this study designs an optimized training scheme: First, the training dataset is processed through a class balancing strategy to avoid bias caused by class imbalance. Specifically, from the large-scale original patent similarity dataset, the number of samples for each class is limited and balanced to ensure that the amount of data for each class is basically consistent, thus alleviating the class skew problem during model training. Second, stratified sampling is used to divide the dataset into training, validation, and test sets to rigorously evaluate the model's generalization ability.

[0057] During the training optimization process, this study fine-tuned the model's parameter settings (deliberate training schedule) and adopted a differential learning rate: the encoder body was updated with a lower learning rate (4e-6) to maintain the pre-trained model's strong semantic representation ability; while the classification head used a relatively higher learning rate (2e-5) for rapid convergence, thus efficiently adapting to the target task. Mixed precision training (FP16) was enabled during model training, significantly reducing memory usage and improving computational efficiency while maintaining accuracy. Furthermore, to effectively avoid overfitting, this study used Dropout regularization and early stopping based on the macro-F1 score of the validation set, maximizing the model's generalization performance.

[0058] Model Training Process and Convergence Results: The input text of the model is processed by an efficient tokenizer, truncated and padded to the maximum sequence length (256 tokens) before being fed into the model. The model encodes two representation vectors for each patent text pair, and then captures the semantic differences and similarity features between the texts through concatenation and absolute difference fusion. The similarity category of the patent pair is then output through a classification head. Since this study treats the patent similarity scoring problem as a multi-class classification problem with an order relationship, the traditional cross-entropy loss can effectively capture the probability differences between categories. However, when there is a clear order between categories, the simple cross-entropy loss function does not consider the severity differences of misclassification. For example, the error of predicting highly similar (category 4) as dissimilar (category 0) is significantly more serious than the error of predicting as fairly similar (category 3). To address this, this study introduces a cross-entropy loss with adjusted class weights during training. By assigning lower weight differences to adjacent categories and higher weight differences to distant categories, the loss function explicitly considers the severity differences of misclassification between different categories. This design not only helps guide the model to focus more on penalizing serious misclassifications, but also effectively improves the ability to distinguish between predicted similarity levels.

[0059] For details, please refer to the appendix. Figure 3As suggested, the trends of the loss function and key performance indicators (macro-F1 score and Pearson correlation coefficient) during model training are first shown. The loss curves show that both training loss and validation loss decrease rapidly and then gradually stabilize. Particularly in the early training phase (epochs 0 to 4), the loss decreases rapidly, indicating that the model quickly learns the basic semantic relationships between patent texts. From the 6th epoch onwards, both loss curves gradually converge to a lower level and stabilize, indicating that the model has reached a good fit and there are no obvious overfitting or underfitting issues. The performance indicator curves show that both the macro-F1 score and the Pearson correlation coefficient (validationPearson) increase rapidly in the early training phase and stabilize after approximately 4 to 6 epochs. The Pearson correlation coefficient exceeded 0.95 early in the training process (around the second epoch), indicating a rapid improvement in the model's accuracy in ranking patent text similarity levels and a strong ability to distinguish between different levels. The macro F1 score stabilized above 0.93 around the sixth epoch, further demonstrating the model's balanced and reliable prediction accuracy and recall across all categories. Overall, the trend of this training curve clearly shows that the fine-tuning strategy proposed in this study effectively improved the model's prediction performance for patent text similarity, quickly achieving good generalization performance without significant overfitting.

[0060] Appendix Figure 4 The results show the precision, recall, and F1 score of the best model on the test set for each similarity level (classes 1 to 5). Overall, the model achieves extremely high classification performance across all classes, with all metrics approaching or reaching full marks, demonstrating excellent and stable recognition capabilities across different similarity levels. This indicates that the model not only effectively distinguishes between different degrees of patent similarity but also exhibits higher classification accuracy and stability for highly similar text pairs.

[0061] The Chinese-RoBERTa model was used to calculate patent similarity and quality: After obtaining a fine-tuned Chinese-RoBERTa model, it was applied to more than 5 million patent texts to calculate patent importance. To save computation time and improve efficiency, the fine-tuned model was first read all at once to generate embeddings for the patent texts. Then, for each patent text, text vectorization was performed in parallel to calculate the cosine similarity between patents to measure the similarity between patent content.

[0062] In this implementation example, the Chinese-RoBERTa model is the previously trained Chinese-RoBERTa-wwm-ext model. The structure of Chinese-RoBERTa-wwm-ext is as follows: Figure 9 As shown.

[0063] The data processing procedure for this model is as follows: First, the text is segmented using the Word Piece Tokenizer combined with the traditional Chinese word segmentation tool Jieba, employing whole-word masking (WWM) technology. A new mask pattern is generated dynamically for each input sequence. Second, during the pre-training phase, the model learns rich semantic knowledge and linguistic rules through tasks such as Masked Language Modeling (MLM) and MLMs correction (Mac Masking), providing a foundation for subsequent text similarity tasks. Then, in the model fine-tuning phase, the 1 million patent pairing data obtained above are used for fine-tuning using a dual-encoder architecture. Finally, the patent text is input into the model, and the model's encoder obtains the text's embedded representation. The cosine similarity between the two embedded vectors is then calculated (steps one to three) to obtain the text similarity score. The first step is data preprocessing, the second and third steps are model training, and the final step is model application.

[0064] The specific process is as follows: First, the Chinese-RoBERTa model is used to generate embedded representations of the patent text. Considering the effectiveness of text feature extraction, this study employs the CLS (classification) special tag as the semantic representation of the entire text to capture its overall information. Next, the generated CLS embedding vectors are whitened, i.e., by subtracting the mean and removing the first two principal components, to reduce the exaggerated effect of cosine similarity between BERT model embedding representations. The motivation for this method is that BERT model embedding representations often tend to cluster close together in the vector space, and directly calculating cosine similarity can easily lead to inflated similarity scores. Whitening effectively eliminates this bias, making the similarity calculation more accurate.

[0065] Secondly, cosine similarity between patent pairs is calculated based on the processed embedded representation. This study only calculates similarity between patents with the same main IPC classification and application year difference within ±5 years. After reading the vectorized patent text, patent text similarity is calculated based on the Chinese-RoBERTa model. However, since the original TF-IDF calculation method results in no negative values, while the cosine similarity of text vectors obtained through the Chinese-RoBERTa model may have negative values, we scale the data by adding 1 and dividing by 2 to obtain a numerical value of text importance based on this text similarity result. Then, we use the patent quality calculation formula to obtain the patent quality index.

[0066] Patent quality index features are applied using the Chinese-RoBERTa model.

[0067] Example 2 The purpose of this embodiment is to provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the above-described method.

[0068] Example 3 The purpose of this embodiment is to provide a computer-readable storage medium.

[0069] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps of the above method.

[0070] Example 4 The purpose of this embodiment is to provide a patent quality assessment system based on patent text similarity, including: The term frequency-inverse inverse document frequency calculation module is configured to calculate the term frequency-inverse inverse document frequency of each word in the patent to be tested; The high-dimensional matrix construction module is configured to construct a high-dimensional matrix of patent text vocabulary based on word frequency-inverse inverse document frequency. The p-th row vector of the matrix represents the word vector of the patent, and its length is the length of the union of all patent text vocabulary. For words that exist in the patent, the corresponding vector element takes the value of word frequency-inverse inverse document frequency, while for words that do not exist in the patent, the value is zero. The similarity calculation module is configured to: use the module of word vectors to standardize word vectors, and the similarity between any two patent pairs is the cosine similarity between the two, with a larger value indicating a higher similarity between the two patents; The patent quality calculation module is configured to: define the reverse similarity index and the forward similarity index of the patent based on similarity. The ratio of a patent's forward similarity index to its backward similarity index represents the patent's quality.

[0071] Example 5 The purpose of this embodiment is to provide a computer program product containing instructions that, when run on a computer, cause the computer to perform the methods and functions involved in any of the above embodiments. The steps and methods involved in the apparatus of the above embodiments correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0072] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0073] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A patent quality assessment method based on patent text similarity, characterized in that, include: Calculate the term frequency-inverse inverse document frequency for each word in the patent to be tested; A high-dimensional matrix of patent text vocabulary is constructed based on word frequency-inverse inverse document frequency. The vector in the p-th row of the matrix represents the word vector of the patent, and its length is the length of the union of all patent text vocabulary. For words that exist in the patent, the corresponding vector element takes the value of word frequency-inverse inverse document frequency, while for words that do not exist in the patent, the value is zero. By using the model pairing of word vectors for word vector standardization, the similarity between any two patent pairs is the cosine similarity between them. The larger the value, the higher the similarity between the two patents. Based on similarity, the reverse similarity index and the forward similarity index of patents are defined. The ratio of a patent's forward similarity index to its backward similarity index represents the patent's quality.

2. The patent quality assessment method based on patent text similarity as described in claim 1, characterized in that, The term frequency-inverse inverse document frequency of each word is calculated as the product of the frequency of the word in the text of the patent and the inverse inverse document frequency.

3. The patent quality assessment method based on patent text similarity as described in claim 1, characterized in that, The reverse file frequency The definition of is: 。 4. The patent quality assessment method based on patent text similarity as described in claim 1, characterized in that, Also includes: The similarity of patents is standardized to obtain the average forward similarity and backward similarity of patents. Patent quality is defined as the ratio of the average forward similarity to the average backward similarity of patents.

5. The patent quality assessment method based on patent text similarity as described in claim 1, characterized in that, Patent applications from the same applicant were excluded when calculating forward similarity, backward similarity, or average forward similarity and backward similarity of patents.

6. The patent quality assessment method based on patent text similarity as described in claim 1, characterized in that, When calculating patent quality, different combinations of forward similarity across IPC domains and backward similarity within the same IPC domain are used. The calculation of patent quality includes: Define the average reverse similarity and average forward similarity of patents; Based on the defined average reverse similarity and average forward similarity of patents, the relative text similarity coefficients of different technical fields relative to all technical fields are defined; Patent quality is calculated based on the average reverse similarity and average forward similarity of patents, as well as the relative text similarity coefficients of different technical fields relative to all technical fields.

7. A patent quality assessment system based on patent text similarity, characterized in that, include: The term frequency-inverse inverse document frequency calculation module is configured to calculate the term frequency-inverse inverse document frequency of each word in the patent to be tested; The high-dimensional matrix construction module is configured to construct a high-dimensional matrix of patent text vocabulary based on word frequency-inverse inverse document frequency. The p-th row vector of the matrix represents the word vector of the patent, and its length is the length of the union of all patent text vocabulary. For words that exist in the patent, the corresponding vector element takes the value of word frequency-inverse inverse document frequency, while for words that do not exist in the patent, the value is zero. The similarity calculation module is configured to: use the module of word vectors to standardize word vectors, and the similarity between any two patent pairs is the cosine similarity between the two, with a larger value indicating a higher similarity between the two patents; The patent quality calculation module is configured to: define the reverse similarity index and the forward similarity index of the patent based on similarity. The ratio of a patent's forward similarity index to its backward similarity index represents the patent's quality.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 6.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method described in any one of claims 1-6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it performs the steps of the method described in any one of claims 1-6.

Images