Patent text similarity analysis method, device, storage medium and equipment

Abstract

The application discloses a patent text similarity analysis method and device, a storage medium and equipment, and belongs to the technical field of machine learning. The method comprises the following steps: extracting n fragment groups from a first patent text and a second patent text, wherein two fragments in each fragment group are from the same paragraph in the two patent texts; obtaining an analysis model comprising n K-BERT models with the same model parameters; injecting a domain named entity triple selected from a specific domain into the n fragment groups by using each K-BERT model to obtain n sentence trees; processing each sentence tree by using each K-BERT model respectively, and outputting the similarity of the first patent text and the second patent text. The application can analyze in combination with specific domain knowledge, the quality of the triple is high, knowledge noise is avoided, and n fragment groups can be processed to extract more core features, thereby improving the accuracy of the similarity analysis.

Description

Technical Field

[0001] This application relates to the field of machine learning technology, and in particular to a method, apparatus, storage medium and device for similarity analysis of patent texts. Background Technology

[0002] When it is necessary to determine the inventiveness of a patent text, the computer device can first calculate the similarity between the patent text and other patent texts, then filter out some prior art documents based on the similarity, and finally provide these prior art documents to the user for reference.

[0003] When calculating the similarity between patent texts, a deep learning model such as BERT or XLNET is usually created first. Then, training data is used to train the deep learning model so that it can obtain the text vectors of common patent texts from a large-scale corpus. Finally, the similarity between the text vectors of each pair of patent texts is calculated.

[0004] Although deep learning models can extract text vectors from patent texts, they lack domain-specific knowledge and perform poorly in tasks that require it. Summary of the Invention

[0005] This application provides a method, apparatus, storage medium, and device for similarity analysis of patented texts, to address the problem that existing deep learning models lack specific domain knowledge and perform poorly in tasks requiring such knowledge. The technical solution is as follows:

[0006] On the one hand, a method for similarity analysis of patent texts is provided, the method comprising:

[0007] Extract n fragment groups from the first patent text and the second patent text. The two fragments in each fragment group are respectively from the same paragraph in the first patent text and the second patent text, and n≥2;

[0008] Obtain the analysis model, which includes n K-BERT models based on knowledge graphs for deep bidirectional translational language, and the model parameters of the n K-BERT models are the same;

[0009] Using each K-BERT model in the analysis model, triples of domain-specific named entities selected from a specific domain are injected into the n fragment groups respectively, resulting in n sentence trees;

[0010] Each sentence tree is processed using each K-BERT model in the analysis model, and the similarity between the first patent text and the second patent text is output.

[0011] In one possible implementation, when n is 3, the paragraphs are a summary, the first paragraph of the specification, and the last paragraph of the specification.

[0012] In one possible implementation, when the analysis model further includes a pooling layer, a concatenation layer, a fully connected layer, and a linear classifier, and the n K-BERT models are respectively connected to the pooling layer, the concatenation layer is respectively connected to the pooling layer and the fully connected layer, and the fully connected layer is connected to the linear classifier; the step of processing each sentence tree using each K-BERT model in the analysis model and outputting the similarity between the first patent text and the second patent text includes:

[0013] Each K-BERT model is used to process an input sentence tree to obtain the first vector of each sentence tree;

[0014] The pooling layer is used to pool the n first vectors to obtain the classification vector of each sentence tree;

[0015] The concatenation layer is used to concatenate the n classification vectors to obtain the second vector;

[0016] The second vector is processed by the fully connected layer to obtain the third vector;

[0017] The linear classifier is used to perform Softmax processing on the third vector to obtain the similarity between the first patent text and the second patent text.

[0018] In one possible implementation, the method further includes:

[0019] After concatenation, a text vector is output, which represents the vector of the first patent text and the second patent text;

[0020] The text vector is analyzed.

[0021] In one possible implementation, each K-BERT model in the analysis model is used to inject triples of domain-specific named entities selected from a specific domain into the n fragment groups, resulting in n sentence trees, including:

[0022] Using each K-BERT model in the analysis model, obtain triples of domain-specific named entities selected by experts and machines in a specific domain;

[0023] For each fragment group, when there is a triple that matches the domain named entity in the fragment group, the triple is injected into the position corresponding to the domain named entity in the fragment group using the K-BERT model corresponding to the triple, thus obtaining a sentence tree.

[0024] In one possible implementation, the method further includes:

[0025] Obtain a training dataset. Each training dataset includes n segments and a label. Two segments in each segment group are from the same paragraph in the third patent text and the fourth patent text, respectively. The label indicates whether the third patent text and the fourth patent text are similar.

[0026] Create the aforementioned analysis model;

[0027] Using each K-BERT model in the analysis model, triples of domain-specific named entities selected from a specific domain are injected into n segments of each training data group to obtain a sentence tree;

[0028] The analysis model is trained using the sentence tree and the labels to obtain the final analysis model.

[0029] In one possible implementation, training the analysis model using the sentence tree and the labels to obtain the final analysis model includes:

[0030] For the i-th set of training data, each sentence tree corresponding to the i-th set of training data is processed by each K-BERT model in the analysis model, and the analysis model is trained according to the labels in the i-th set of training data to obtain a set of model parameters, where i is a positive integer;

[0031] The model parameters are shared with each K-BERT model in the analysis model;

[0032] Update i to i+1, and continue to execute the step of processing each sentence tree using each K-BERT model in the analysis model until the analysis model meets the preset conditions and then stop training to obtain the final analysis model.

[0033] On the one hand, a similarity analysis device for patent texts is provided, the device comprising:

[0034] An extraction module is used to extract n fragment groups from the first patent text and the second patent text, wherein the two fragments in each fragment group are respectively from the same paragraph in the first patent text and the second patent text, and n≥2;

[0035] The first acquisition module is used to acquire the analysis model, which includes n K-BERT models based on knowledge graphs for deep bidirectional transformation of language, and the model parameters of the n K-BERT models are the same.

[0036] The first injection module is used to inject triples of domain-specific named entities selected from a specific domain into the n fragment groups using each K-BERT model in the analysis model, thereby obtaining n sentence trees;

[0037] The processing module is used to process each sentence tree using each K-BERT model in the analysis model and output the similarity between the first patent text and the second patent text.

[0038] On the one hand, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, the at least one instruction being loaded and executed by a processor to implement the patent text similarity analysis method as described above.

[0039] On one hand, a computer device is provided, the computer device including a processor and a memory, the memory storing at least one instruction, the instruction being loaded and executed by the processor to implement the patent text similarity analysis method as described above.

[0040] The beneficial effects of the technical solution provided in this application include at least the following:

[0041] Since the triples injected into the n fragment groups are domain-specific named entities selected from a specific field, the analysis of patent text similarity can be performed by incorporating domain-specific knowledge of the patent text's domain, thereby improving the accuracy of the similarity analysis. Furthermore, the triples are selected by domain-specific experts and machine learning, ensuring high-quality injected triples and preventing the K-BERT model's attention from shifting due to low-quality triples, distorting the original meaning, and generating knowledge noise, thus further improving the accuracy of the similarity analysis.

[0042] Since the analysis model includes n K-BERT models with identical model parameters, each K-BERT model processes one fragment group, allowing the n K-BERT models to process n fragment groups to extract more core features from the patent text without increasing model parameters. This results in a linear increase in time and space complexity compared to the original single K-BERT model, solving the text length limitation of a single K-BERT model and improving the accuracy of similarity analysis.

[0043] For a patent document, its abstract, the first paragraph of the specification, and the last paragraph of the specification can basically summarize the core information of the entire patent document. Therefore, the fragments extracted from the abstract, the first paragraph of the specification, and the last paragraph of the specification can express the core features of the patent document, thereby improving the accuracy of similarity analysis. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a flowchart of a patent text similarity analysis method provided in one embodiment of this application;

[0046] Figure 2 This is a flowchart of a method for training an analysis model according to an embodiment of this application;

[0047] Figure 3 This is a schematic diagram of the structure of an analysis model provided in one embodiment of this application;

[0048] Figure 4 This is a flowchart of a patent text similarity analysis method provided in another embodiment of this application;

[0049] Figure 5 This is a structural block diagram of a patent text similarity analysis device provided in another embodiment of this application;

[0050] Figure 6 This is a structural block diagram of a patent text similarity analysis device provided in another embodiment of this application. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0052] Please refer to Figure 1 This document illustrates a flowchart of a patent text similarity analysis method according to an embodiment of this application. This patent text similarity analysis method can be applied to computer devices. The patent text similarity analysis method may include:

[0053] Step 101: Extract n fragment groups from the first patent text and the second patent text. The two fragments in each fragment group are from the same paragraph in the first patent text and the second patent text, respectively, and n≥2.

[0054] The computer device needs to calculate the similarity between two patent texts. In this embodiment, one of the patent texts is referred to as the first patent text, and the other patent text is referred to as the second patent text.

[0055] Patent texts typically include an abstract, claims, and a description, all sharing the same text format. In this embodiment, the computer device can extract segments of identical paragraphs from both the first and second patent texts, and then combine the two segments into a single segment group. For example, since the abstract consists of only one paragraph, the computer device can combine the abstracts of the first and second patent texts into a single segment group. Similarly, since the description contains multiple paragraphs, the computer device needs to select one paragraph. If the paragraph is the first paragraph of the description, then the first paragraph of the first and second patent texts' descriptions will be combined into a single segment group; if the paragraph is the last paragraph of the description, then the last paragraph of both the first and second patent texts' descriptions will be combined into a single segment group.

[0056] Step 102: Obtain the analysis model, which includes n K-BERT models based on knowledge graphs for deep bidirectional language transformation, and the model parameters of the n K-BERT models are the same.

[0057] The analysis model is used to analyze two patent texts and output their similarity scores. In this embodiment, the analysis model can be trained by a computer device or obtained by the computer device from other devices; this embodiment does not limit the source of the analysis model.

[0058] The analysis model includes three K-BERT (Knowledge Graph-Bidirectional Encoder Representations from Transformers) models, and each K-BERT model has the same structure and model parameters.

[0059] Step 103: Using each K-BERT model in the analysis model, the triples of domain-specific named entities selected from the specific domain are injected into the n fragment groups respectively to obtain n sentence trees.

[0060] Since a domain named entity can correspond to multiple triples, in this embodiment, the triples of the domain named entity can be screened first to remove the triples of poor quality and retain the triples of high quality. Then, each K-BERT model in the analysis model is used to inject the screened triples into the corresponding n fragment groups to obtain n sentence trees.

[0061] Step 104: Process each sentence tree using each K-BERT model in the analysis model, and output the similarity between the first patent text and the second patent text.

[0062] In this embodiment, each K-BERT model can be used to process a corresponding sentence tree, and then the intermediate results output by n K-BERT models can be processed to obtain the similarity between the first patent text and the second patent text.

[0063] In summary, the patent text similarity analysis method provided in this application improves the accuracy of similarity analysis by incorporating domain-specific knowledge into the triples injected into the n segment groups, since these triples are domain-specific named entities selected from a specific domain. Furthermore, the triples are selected by experts and machines in a specific domain, ensuring high-quality injected triples and preventing the K-BERT model from shifting its attention, distorting the original meaning, and generating knowledge noise due to the injection of low-quality triples. This further enhances the accuracy of similarity analysis.

[0064] Since the analysis model includes n K-BERT models with identical model parameters, each K-BERT model processes one fragment group, allowing the n K-BERT models to process n fragment groups to extract more core features from the patent text without increasing model parameters. This results in a linear increase in time and space complexity compared to the original single K-BERT model, solving the text length limitation of a single K-BERT model and improving the accuracy of similarity analysis.

[0065] Before using the analysis model, it needs to be trained. The training process is explained below; please refer to the instructions. Figure 2 .

[0066] Step 201: Obtain the training dataset. Each training dataset includes n segments and a label. The two segments in the segment group are from the same paragraphs in the third patent text and the fourth patent text, respectively. The label indicates whether the third patent text and the fourth patent text are similar.

[0067] Patent texts typically include an abstract, claims, and a description, and have the same text format. In this embodiment, the computer device can extract segments of the same paragraphs from the third and fourth patent texts, and then combine the two segments into a segment group.

[0068] When n is 3, the paragraphs are the abstract, the first paragraph of the specification, and the last paragraph of the specification. Then, a set of training data includes three fragment groups and one label. The first fragment group contains the abstract of the third patent text and the abstract of the fourth patent text. The second fragment group contains the first paragraph of the specification of the third patent text and the first paragraph of the specification of the fourth patent text. The third fragment group contains the last paragraph of the specification of the third patent text and the last paragraph of the specification of the fourth patent text.

[0069] Since the abstract, the first paragraph, and the last paragraph of the specification can basically summarize the core information of the entire patent text, the fragments extracted from the abstract, the first paragraph, and the last paragraph of the specification can express the core features of the patent text, thereby improving the accuracy of similarity analysis.

[0070] The label can be two different numbers to indicate whether the third and fourth patent texts are similar or dissimilar. For example, 0 can represent that the third and fourth patent texts are dissimilar, and 1 can represent that they are similar.

[0071] Step 202: Create the analysis model.

[0072] Please refer to Figure 3 The analysis model in this embodiment includes three K-BERT models, a pooling layer, a splicing layer, a fully connected layer, and a linear classifier. The n K-BERT models are connected to the pooling layer, the splicing layer is connected to both the pooling layer and the fully connected layer, and the fully connected layer is connected to the linear classifier.

[0073] While the BERT model performs exceptionally well in natural language processing, it underperforms in specific domains. The K-BERT model, which combines knowledge graphs with BERT, demonstrates superior performance. However, like BERT, K-BERT is limited by the number of characters it can process, only 512 characters. Since each training dataset consists of two patent texts, each text can only process 256 characters, which is far from sufficient for extracting patent features from a single 512-character text. This embodiment uses three K-BERT models, allowing for the processing of three 512-character text segments. If a segment exceeds 512 characters, the excess characters are automatically discarded.

[0074] Step 203: Using each K-BERT model in the analysis model, the triples of domain-specific named entities selected from the specific domain are injected into the n segments of each training data group to obtain the sentence tree.

[0075] Experiments have shown that if the quality of the injected triples is low, the K-BERT model will inject some poor-quality triples that cannot express specific domain knowledge into the training data, causing the K-BERT model's attention to shift, distorting the meaning of the original text, and creating knowledge noise. Therefore, the quality of the injected triples must be high. The knowledge graph in this embodiment consists of high-quality triples that have been manually collected by domain experts and automatically filtered using syntactic patterns. Experiments have verified that using the filtered knowledge graph can improve the accuracy of the analysis model. Taking the specific domain of Traditional Chinese Medicine (TCM) as an example, manual collection combined with automatic machine collection can be used to extract domain-named entities (such as entities related to technical fields, components, processing methods, and the efficacy of objects) to construct a TCM knowledge graph, thereby ensuring the injection of effective knowledge.

[0076] Specifically, for each fragment group, when a triple matches a domain named entity in the fragment group, the computer device can use the K-BERT model corresponding to that triple to inject the triple into the position corresponding to the domain named entity in the fragment group. The fragment with the injected triple is then flattened, and the relative positions between words are represented by position indices, thus obtaining a sentence tree. In this way, three sentence trees can be obtained for each set of training data.

[0077] When a domain-specific named entity in a knowledge graph corresponds to multiple triples, the computer device can directly inject the first two triples, avoiding ambiguity and knowledge noise caused by injecting too many triples.

[0078] Step 204: Train the analysis model using sentence trees and labels to obtain the final analysis model.

[0079] Specifically, training the analysis model using sentence trees and labels to obtain the final analysis model can include: for the i-th set of training data, processing each sentence tree corresponding to the i-th set of training data using each K-BERT model in the analysis model, and training the analysis model based on the labels in the i-th set of training data to obtain a set of model parameters, where i is a positive integer; sharing the model parameters with each K-BERT model in the analysis model; updating i to i+1, and continuing to execute the step of processing each sentence tree using each K-BERT model in the analysis model until the analysis model meets the preset conditions and then stopping the training to obtain the final analysis model.

[0080] Using all the constructed sentence trees as input and defining a set of parameters (which are shared by each K-BERT model), we can obtain the model corresponding to this set of parameters, along with metrics such as precision and recall. Different models can be obtained by changing the combination of model parameters. We can select the parameter combination that yields the highest precision, recall, or F-score to obtain the optimal analysis model.

[0081] After obtaining the analysis model, we can use it to analyze the similarity between the two patent texts. Please refer to [link / reference]. Figure 4 This document illustrates a flowchart of a patent text similarity analysis method according to an embodiment of this application. This patent text similarity analysis method can be applied to computer devices. The patent text similarity analysis method may include:

[0082] Step 401: Extract n fragment groups from the first patent text and the second patent text. The two fragments in each fragment group are from the same paragraph in the first patent text and the second patent text, respectively, and n≥2.

[0083] Patent texts typically include an abstract, claims, and a description, and have the same text format. In this embodiment, the computer device can extract segments of the same paragraphs from the first patent text and the second patent text, and then combine the two segments into a segment group.

[0084] When n is 3, the paragraphs are the abstract, the first paragraph of the specification, and the last paragraph of the specification. The computer device can extract three fragment groups. The first fragment group contains the abstract of the first patent text and the abstract of the second patent text. The second fragment group contains the first paragraph of the first patent text and the first paragraph of the second patent text. The third fragment group contains the last paragraph of the first patent text and the last paragraph of the second patent text.

[0085] Since the abstract, the first paragraph, and the last paragraph of the specification can basically summarize the core information of the entire patent text, the fragments extracted from the abstract, the first paragraph, and the last paragraph of the specification can express the core features of the patent text, thereby improving the accuracy of similarity analysis.

[0086] Step 402: Obtain the analysis model, which includes n K-BERT models, and the model parameters of the n K-BERT models are the same.

[0087] The analytical model acquired by computer equipment can be obtained through... Figure 2 The model trained using the steps shown.

[0088] Step 403: Using each K-BERT model in the analysis model, the triples of domain-specific named entities selected from the specific domain are injected into the n fragment groups respectively to obtain n sentence trees.

[0089] Specifically, using each K-BERT model in the analysis model, triples of domain-specific named entities selected from a specific domain are injected into n fragment groups to obtain n sentence trees. This can include: using each K-BERT model in the analysis model to obtain triples of domain-specific named entities selected by experts and machines in a specific domain; for each fragment group, when there is a triple that matches a domain-specific named entity in the fragment group, the K-BERT model corresponding to that triple is used to inject the triple into the position in the fragment group corresponding to the domain-specific named entity to obtain a sentence tree.

[0090] The knowledge graph in this embodiment is a high-quality triplet that has been manually collected by experts in a specific domain and automatically filtered using syntactic patterns. Experiments have verified that using the filtered knowledge graph can improve the accuracy of the analysis model. Taking the field of traditional Chinese medicine as an example, manual collection combined with automatic machine collection can be used to extract domain-named entities (such as entities related to technical fields, components, processing methods, and efficacy of objects) to construct a knowledge graph for the field of traditional Chinese medicine, thereby ensuring the injection of effective knowledge.

[0091] The computer device can inject triples into the fragment group at the positions corresponding to the domain named entities, flatten the fragments with injected triples, and use position indices to represent the relative positions between words, thus obtaining a sentence tree. In this way, the computer device can obtain a total of three sentence trees.

[0092] When a domain-specific named entity in a knowledge graph corresponds to multiple triples, the computer device can directly inject the first two triples, avoiding ambiguity and knowledge noise caused by injecting too many triples.

[0093] Step 404: When the analysis model also includes a pooling layer, a concatenation layer, a fully connected layer, and a linear classifier, and n K-BERT models are connected to the pooling layer, the concatenation layer is connected to both the pooling layer and the fully connected layer, and the fully connected layer is connected to the linear classifier, each K-BERT model is used to process an input sentence tree to obtain the first vector of each sentence tree.

[0094] like Figure 3 As shown, the computer device can input each group of fragments into a corresponding K-BERT model, and each K-BERT model can obtain a first vector.

[0095] Since the analysis model includes n K-BERT models with identical model parameters, each K-BERT model processes one fragment group, allowing the n K-BERT models to process n fragment groups to extract more core features from the patent text without increasing model parameters. This results in a linear increase in time and space complexity compared to the original single K-BERT model, solving the text length limitation of a single K-BERT model and improving the accuracy of similarity analysis.

[0096] Step 405: Use a pooling layer to perform pooling processing on the n first vectors to obtain the classification vector for each sentence tree.

[0097] The pooling layer can pool n first vectors to obtain n classification CLS vectors.

[0098] Step 406: Use the concatenation layer to concatenate the n classification vectors to obtain the second vector.

[0099] Step 407: The second vector is processed by a fully connected layer to obtain the third vector.

[0100] Step 408: Use a linear classifier to perform Softmax processing on the third vector to obtain the similarity between the first patent text and the second patent text.

[0101] The similarity mentioned here can be a specific percentage, such as 85%; or it can be a numerical value set for the label during training, such as 1 or 0.

[0102] In this embodiment, a text vector is output after splicing. This text vector represents the vector of the first patent text and the second patent text. The computer device can also analyze the text vector. In this embodiment, no specific analysis method or application scenario is limited.

[0103] In summary, the patent text similarity analysis method provided in this application improves the accuracy of similarity analysis by incorporating domain-specific knowledge into the triples injected into the n segment groups, since these triples are domain-specific named entities selected from a specific domain. Furthermore, the triples are selected by experts and machines in a specific domain, ensuring high-quality injected triples and preventing the K-BERT model from shifting its attention, distorting the original meaning, and generating knowledge noise due to the injection of low-quality triples. This further enhances the accuracy of similarity analysis.

[0104] Since the analysis model includes n K-BERT models with identical model parameters, each K-BERT model processes one fragment group, allowing the n K-BERT models to process n fragment groups to extract more core features from the patent text without increasing model parameters. This results in a linear increase in time and space complexity compared to the original single K-BERT model, solving the text length limitation of a single K-BERT model and improving the accuracy of similarity analysis.

[0105] For a patent document, its abstract, the first paragraph of the specification, and the last paragraph of the specification can basically summarize the core information of the entire patent document. Therefore, the fragments extracted from the abstract, the first paragraph of the specification, and the last paragraph of the specification can express the core features of the patent document, thereby improving the accuracy of similarity analysis.

[0106] Please refer to Figure 5 This diagram illustrates a structural block diagram of a patent text similarity analysis apparatus according to an embodiment of this application. This patent text similarity analysis apparatus can be applied to a computer device. The patent text similarity analysis apparatus may include:

[0107] Extraction module 510 is used to extract n fragment groups from the first patent text and the second patent text, wherein the two fragments in each fragment group are respectively from the same paragraph in the first patent text and the second patent text, and n≥2;

[0108] The first acquisition module 520 is used to acquire the analysis model, which includes n deep bidirectional transformation language K-BERT models based on knowledge graphs, and the model parameters of the n K-BERT models are the same.

[0109] The first injection module 530 is used to inject the triples of domain-specific named entities selected from the domain into n fragment groups by utilizing each K-BERT model in the analysis model, thereby obtaining n sentence trees;

[0110] The processing module 540 is used to process each sentence tree using each K-BERT model in the analysis model and output the similarity between the first patent text and the second patent text.

[0111] In an optional embodiment, when n is 3, the paragraphs are a summary, a first paragraph of the specification, and a last paragraph of the specification.

[0112] In an optional embodiment, when the analysis model further includes a pooling layer, a concatenation layer, a fully connected layer, and a linear classifier, and n K-BERT models are respectively connected to the pooling layer, the concatenation layer is respectively connected to the pooling layer and the fully connected layer, and the fully connected layer is connected to the linear classifier; the processing module 540 is further configured to:

[0113] Each K-BERT model is used to process an input sentence tree to obtain the first vector of each sentence tree;

[0114] The pooling layer is used to pool the n first vectors to obtain the classification vector for each sentence tree;

[0115] The n classification vectors are concatenated using a concatenation layer to obtain the second vector;

[0116] The second vector is processed using a fully connected layer to obtain the third vector;

[0117] The similarity between the first and second patent texts is obtained by applying Softmax processing to the third vector using a linear classifier.

[0118] Please refer to Figure 6 In an optional embodiment, the device further includes:

[0119] The output module 550 is used to output a text vector after the splicing process. The text vector represents the vector of the first patent text and the second patent text.

[0120] Analysis module 560 is used to analyze text vectors.

[0121] In an optional embodiment, the first injection module 530 is further configured to:

[0122] Using each K-BERT model in the analysis model, obtain triples of domain-specific named entities that have been screened by experts and machines in a specific domain;

[0123] For each fragment group, when there is a triple that matches the domain named entity in the fragment group, the K-BERT model corresponding to the triple is used to inject the triple into the position in the fragment group corresponding to the domain named entity, resulting in a sentence tree.

[0124] In an optional embodiment, the device further includes:

[0125] The second acquisition module 570 is used to acquire the training dataset. Each training dataset includes n fragment groups and a label. The two fragments in the fragment group are from the same paragraphs in the third patent text and the fourth patent text, respectively. The label indicates whether the third patent text and the fourth patent text are similar.

[0126] Create module 580 to create the analysis model;

[0127] The second injection module 590 is also used to inject the triples of domain-specific named entities selected from the domain into the n fragment groups in each training data group using each K-BERT model in the analysis model, so as to obtain a sentence tree;

[0128] Training module 600 is used to train the analysis model using sentence trees and labels to obtain the final analysis model.

[0129] In an optional embodiment, the training module 600 is further configured to:

[0130] For the i-th set of training data, each sentence tree corresponding to the i-th set of training data is processed by each K-BERT model in the analysis model, and the analysis model is trained according to the labels in the i-th set of training data to obtain a set of model parameters, where i is a positive integer;

[0131] Share the model parameters with each K-BERT model in the analysis model;

[0132] Update i to i+1 and continue to process each sentence tree using each K-BERT model in the analysis model until the analysis model meets the preset conditions and then stop training to obtain the final analysis model.

[0133] In summary, the patent text similarity analysis device provided in this application, because the triples injected into the n segment groups are domain-specific named entities selected from a specific domain, can combine domain-specific knowledge of the patent text to perform similarity analysis, thereby improving the accuracy of similarity analysis. Furthermore, the triples are selected by experts and machines in a specific domain, ensuring high-quality injected triples and avoiding the problem of K-BERT model attention shifting, distorting the original meaning, and generating knowledge noise due to the injection of low-quality triples, thus further improving the accuracy of similarity analysis.

[0134] Since the analysis model includes n K-BERT models with identical model parameters, each K-BERT model processes one fragment group, allowing the n K-BERT models to process n fragment groups to extract more core features from the patent text without increasing model parameters. This results in a linear increase in time and space complexity compared to the original single K-BERT model, solving the text length limitation of a single K-BERT model and improving the accuracy of similarity analysis.

[0135] For a patent document, its abstract, the first paragraph of the specification, and the last paragraph of the specification can basically summarize the core information of the entire patent document. Therefore, the fragments extracted from the abstract, the first paragraph of the specification, and the last paragraph of the specification can express the core features of the patent document, thereby improving the accuracy of similarity analysis.

[0136] One embodiment of this application provides a computer-readable storage medium storing at least one instruction, which is loaded and executed by a processor to implement the patent text similarity analysis method described above.

[0137] One embodiment of this application provides a computer device including a processor and a memory, wherein the memory stores at least one instruction, which is loaded and executed by the processor to implement the patent text similarity analysis method as described above.

[0138] It should be noted that the patent text similarity analysis device provided in the above embodiments is only illustrated by the division of the above functional modules when performing patent text similarity analysis. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the patent text similarity analysis device can be divided into different functional modules to complete all or part of the functions described above. In addition, the patent text similarity analysis device and the patent text similarity analysis method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0139] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0140] The above description is not intended to limit the embodiments of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this application should be included within the protection scope of the embodiments of this application.

Claims

1. A similarity analysis method of patent texts, characterized by, The method comprises: extracting n fragment groups from the first patent text and the second patent text, two fragments in each fragment group being from the same paragraph in the first patent text and the second patent text respectively, n≥2; obtaining an analysis model, the analysis model comprising n knowledge graph-based deep bidirectional conversion language K-BERT models, and the model parameters of the n K-BERT models being the same; using each K-BERT model in the analysis model to inject a triple of domain named entities selected from a specific domain into the n fragment groups respectively, to obtain n sentence trees; using each K-BERT model in the analysis model to process each sentence tree respectively, and outputting the similarity of the first patent text and the second patent text; The method further comprises: obtaining a training data set, each group of training data in the training data set comprising n fragment groups and a label, two fragments in the fragment groups being from the same paragraph in a third patent text and a fourth patent text respectively, and the label indicating whether the third patent text and the fourth patent text are similar; creating the analysis model; using each K-BERT model in the analysis model to inject a triple of domain named entities selected from a specific domain into the n fragment groups in each group of training data, to obtain a sentence tree; training the analysis model using the sentence tree and the label to obtain a final analysis model; The training of the analysis model using the sentence tree and the label to obtain a final analysis model comprises: for the i-th group of training data, using each K-BERT model in the analysis model to process each sentence tree corresponding to the i-th group of training data respectively, and training the analysis model according to the label in the i-th group of training data to obtain a group of model parameters, i being a positive integer; sharing the model parameters to each K-BERT model in the analysis model; updating i to i+1, and continuing to perform the step of processing each sentence tree using each K-BERT model in the analysis model until the analysis model meets a preset condition to stop training, to obtain a final analysis model.

2. The method of claim 1, wherein, When n is 3, the paragraph is an abstract, a first paragraph of the specification, and a last paragraph of the specification.

3. The method of claim 1, wherein, When the analysis model further comprises a pooling layer, a concatenation layer, a fully connected layer, and a linear classifier, and the n K-BERT models are connected to the pooling layer respectively, the concatenation layer is connected to the pooling layer and the fully connected layer respectively, and the fully connected layer is connected to the linear classifier; the processing of each sentence tree using each K-BERT model in the analysis model and the output of the similarity of the first patent text and the second patent text comprise: processing an input sentence tree using each K-BERT model to obtain a first vector of each sentence tree; performing pooling processing on n first vectors using the pooling layer to obtain a classification vector of each sentence tree; The n classification vectors are spliced by using the splicing layer to obtain a second vector; The second vector is fully connected by using the full connection layer to obtain a third vector; The third vector is processed by using the linear classifier to obtain the similarity of the first patent text and the second patent text.

4. The method of claim 3, wherein, The method further comprises: Outputting a text vector after splicing processing, the text vector representing the vector of the first patent text and the second patent text; Analyzing the text vector.

5. The method of claim 1, wherein, Each K-BERT model in the analysis model is used to inject a triple of domain named entities screened from a specific field into the n fragment groups respectively to obtain n sentence trees, including: Each K-BERT model in the analysis model is used to obtain a triple of domain named entities screened by experts and machines in a specific field; For each fragment group, when there is a triple of domain named entities matching the domain named entities in the fragment group, the K-BERT model corresponding to the triple is used to inject the triple into the position corresponding to the domain named entities in the fragment group to obtain a sentence tree.

6. A similarity analysis device for patent texts, characterized by, The device comprises: An extraction module is configured to extract n fragment groups from a first patent text and a second patent text, two fragments in each fragment group being from the same paragraph in the first patent text and the second patent text respectively, and n≥2; A first acquisition module is configured to acquire an analysis model, the analysis model comprising n K-BERT models based on a knowledge graph and a deep bidirectional conversion language, and the model parameters of the n K-BERT models being the same; A first injection module is configured to use each K-BERT model in the analysis model to inject a triple of domain named entities screened from a specific field into the n fragment groups respectively to obtain n sentence trees; A processing module is configured to use each K-BERT model in the analysis model to process each sentence tree respectively and output the similarity of the first patent text and the second patent text; The device further comprises: a second acquisition module configured to acquire a training data set, each set of training data in the training data set comprising n fragment groups and a label, two fragments in the fragment groups being from the same paragraph in a third patent text and a fourth patent text respectively, and the label representing whether the third patent text and the fourth patent text are similar; a creation module configured to create the analysis model; a second injection module configured to use each K-BERT model in the analysis model to inject a triple of domain named entities screened from a specific field into the n fragment groups in each set of training data to obtain a sentence tree; and a training module configured to train the analysis model by using the sentence tree and the label to obtain a final analysis model. The training module is further configured to: for the ith set of training data, process each sentence tree corresponding to the ith set of training data by using each K-BERT model in the analysis model respectively, and train the analysis model according to labels in the ith set of training data to obtain a set of model parameters, i being a positive integer; share the model parameters to each K-BERT model in the analysis model; update i to i+1, and continue to perform the step of processing each sentence tree by using each K-BERT model in the analysis model respectively until the analysis model meets a preset condition to stop training, and obtain a final analysis model.

7. A computer readable storage medium characterized in that, The storage medium has at least one instruction stored therein, and the at least one instruction is loaded and executed by the processor to implement the patent text similarity analysis method according to any one of claims 1 to 5.

8. A computer device, comprising: The computer device includes a processor and a memory, and the memory has at least one instruction stored therein, and the instruction is loaded and executed by the processor to implement the patent text similarity analysis method according to any one of claims 1 to 5.

Images