Patent investigation system and performance evaluation method for the same

Abstract

To solve a problem of preparing a large amount of learning data by utilizing knowledge of a skilled patent engineer when learning a specification text of a patent publication and creating a language model dedicated to patent publications for evaluating a degree of coincidence and a degree of similarity between patent publications.SOLUTION: A patent investigation system 10 uses a sentence encoder 1 based on a language model for calculating a similarity between a text for each claim of a patent claim range of a patent publication 80 of interest and a text for each claim of patent claims of an investigation target patent publication 90. The sentence encoder 1 deletes stop words from texts of patent claims and specifications of a large number of patent publications having the same section symbol of IPC classification described at the head of the patent publication 80 of interest by a division unit 20, divides the texts into segments divided at each punctuation mark equal to or less than one fourth of the number of tokens of an input limitation of the language model, and performs relearning using the segments.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] In product design in the technology departments of manufacturing companies and research and development in laboratories, in order to understand domestic and foreign patent publications of other companies that are highly relevant to newly developed products and elemental technologies, in addition to retrospective research that looks back on the past (also known as thematic research or technological trend research), regular research such as weekly and monthly SDI (Selective Dissemination of Information) searches is essential. This invention relates to a "patent research system and performance evaluation method for a patent research system" that uses a sentence encoder based on a language model to streamline the practice of researching new third-party patents in a patent research system that searches domestic and foreign patent publications of third parties that are relevant to a specific technological theme. [Background technology]

[0002] In recent years, with the rapid development of language models, various attempts have been made to utilize them in patent research. For example, BERT (Bidirectional Encoder Representations from Transformers) is known as one such language model. BERT is a general-purpose large-scale language model incorporating an architecture called a Transformer. It is a natural language processing technology for performing various tasks involving text. Language models pre-trained using general-purpose data consisting of text not biased toward a specific field, such as Wikipedia (registered trademark), have been made public. In practice, to improve suitability for a specific field, they are retrained (also known as fine-tuning) using a relatively small amount of data related to that field.

[0003] Patent Document 1 introduces a research support method that uses the language model BERT to import a patent claim (hereinafter referred to as the "target sentence") that is the subject of literature research, breaks it down into token units (text units such as words and phrases), detects matches on these token units, compares the target sentence with reference sentences in patent documents that are candidates for cited documents, and determines that the parts of the tokens in the target sentence that are described in the reference sentences are matches.

[0004] Patent decision documents are used to fine-tune this language model. Patent decision documents clearly describe the similarities and differences between the present application and the cited references in patent examinations, which has the advantage of making it easy to extract target sections to input into the BERT model. However, the number of patent decision documents is not large compared to the number of published patent applications, so it is difficult to say that the volume is sufficient for fine-tuning.

[0005] Furthermore, using patent examination documents as training data requires a lot of manual effort and cost because there are no headings for (similarities) and (differences) like in patent decision documents, making it difficult to secure training data that can be used to fine-tune the BERT model and achieve sufficient performance.

[0006] Patent Document 2 describes a method for analyzing patent claims and evaluating the similarity between patent specifications. It involves morphological analysis of the patent specification claim text, dividing it into words, vectorizing the divided words using distributed representations, and then converting the claim text into a tensor (multidimensional array) using LSTM (Long Short-Term Memory) to evaluate the similarity between patent publications. However, patent publications cover a wide range of technical content, as shown by the eight broad IPC classifications. The same word may have different meanings depending on the IPC classification. Therefore, it is difficult to say that word vectorization alone is sufficient to evaluate the similarity between patent publications. Here, "patent publication" refers to the patent publication (or, if there is no patent publication, the patent publication). [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Japanese Patent Application Publication No. 2020-181529 [Patent Document 2] Japanese Patent Publication No. 2022-052212 Summary of the Invention [Problem to be solved by the invention]

[0008] As mentioned above, several methods have been proposed for learning the specification text of patent publications and understanding their meaning and content. However, when creating a language model specifically for patent publications to evaluate the similarity between patent publications, in order to understand the meaning of words and phrases taking into account the highly specialized nature of patent publications, training these language models requires the preparation of large amounts of training data, including the knowledge of experienced patent engineers, which is a drawback in that it requires high manpower costs.

[0009] Therefore, in this disclosure, we focus on the International Patent Classification (hereinafter referred to as the IPC classification) and propose a patent research system that mechanically collects a large number of patent publications without requiring the knowledge or effort of experienced patent engineers, and uses eight sentence encoders for each of the eight IPC classifications, with language models fine-tuned using a large number of patent publications. [Means for solving the problem]

[0010] One form of the present disclosure made to solve the above problem is a patent research system using a sentence encoder based on a language model that calculates the similarity between the text of each claim in the claims of a patent publication of interest and the text of each claim in the claims of a patent publication being searched, characterized in that the sentence encoder is a patent research system that uses a segmentation unit to remove stop words from the claims and text of specifications of a large number of patent publications that have the same IPC classification section symbol listed at the beginning of the patent publication of interest, and divides them into segments divided by punctuation marks equal to or less than 1 / 4 of the number of tokens in the input limit of the language model, and re-trains using the segments.

[0011] According to this aspect, it is desirable that the sentence encoder used in the patent research system is adapted to remove stop words from the claims and specifications of a number of patent publications that have the same IPC classification section symbol at the beginning of the patent publication of interest, divide them into segments separated by punctuation marks, each of which has a number equal to or less than 1 / 4 of the number of tokens that is the input limit of the language model, and then re-train using the segments.

[0012] This allows the patent research system to not only understand words through pre-trained language models, but also acquire knowledge about words described in patent publications corresponding to IPC classifications through eight types of sentence encoders created by fine-tuning different patent publications for each IPC classification section symbol, enabling searches that are compatible with the IPC classification of the patent publication.

[0013] In the above response, it is desirable to have a division unit that creates segments and corresponding segment symbols for each claim in the text of the claims of the patent publication of interest and the patent publication being investigated, a vector generation unit that generates a set of sentence vectors from the segments using a sentence encoder, a similarity calculation unit that calculates the cosine similarity for each combination of the set of sentence vectors for each segment of the patent publication of interest and the set of sentence vectors for each segment of the patent publication being investigated, and a display unit that displays the similarity based on the cosine similarity corresponding to the combination of segments for each combination of segment symbols.

[0014] According to this correspondence, the patent research system includes a division unit that creates segments and corresponding segment symbols for each claim in the text of the claims of the patent publication of interest and the patent publication being researched, a vector generation unit that generates a set of sentence vectors from the segments using a sentence encoder, a similarity calculation unit that calculates the cosine similarity for each combination of the set of sentence vectors for each segment of the patent publication of interest and the set of sentence vectors for each segment of the patent publication being researched, and a display unit that displays the similarity based on the cosine similarity corresponding to the combination of segments for each combination of segment symbols.

[0015] As a result, the patent research system divides the text of the claims of the patent publication of interest and the patent publications being searched into segments for each claim, and generates a set of sentence vectors for each segment using a sentence encoder. By calculating the cosine similarity for all combinations of the sentence vectors of the patent publication of interest and the patent publication being investigated, the similarity can be calculated based on this and displayed for each combination of segment symbols. In other words, the similarity between the claims of two patent publications can be confirmed based on the segments.

[0016] In the above-mentioned measures, if there are multiple patent publications being searched, it is desirable to provide a display unit that calculates a score based on the similarity between the patent publication of interest and each of the patent publications being searched, and displays the patent publications being searched based on the score.

[0017] According to this approach, when there are multiple patent publications being searched, the patent search system is equipped with a display unit that calculates a score based on the similarity between the patent publication of interest and each of the patent publications being searched, and displays the patent publications being searched based on the score.

[0018] As a result, when there are multiple patent publications to be searched, for example 100, the patent search system can calculate the similarity between the patent publication of interest and each of the patent publications to be searched, calculate a score based on an aggregate value of all or part of the similarities or an average value thereof, and display the patent publications to be searched in order based on the score. In other words, it is possible to check the 100 patent publications to be searched, starting with the important patent publications to be searched that have the highest similarity based on the score.

[0019] In the above response, in the performance evaluation method of the patent search system, it is desirable to select a patent publication of interest, and in the patent information platform such as a notice of reasons for refusal or a search report, select multiple patent publications that are related to the patent publication of interest through citation or cited relationships, and use these as the correct data for the patent publication to be searched.Furthermore, as the test data for the patent publication to be searched, select multiple patent publications that have the same IPC classification section, class, subclass, and main group listed at the top of the patent publication of interest, and use a test set consisting of the patent publication of interest, the correct data, and the test data.

[0020] According to this approach, the performance evaluation method for a patent search system involves selecting a patent publication of interest, selecting multiple patent publications that are related to the patent publication of interest through citation or cited relationships in patent information platforms such as rejection notices or search reports, and using these as the correct data for the patent publication to be searched, and further selecting multiple patent publications that have the same IPC classification section, class, subclass, and main group listed at the top of the patent publication of interest as the test data for the patent publication to be searched, and using a test set consisting of the patent publication of interest, the correct data, and the test data.

[0021] This allows a test set to be input into the patent search system, and if the correct data is at the top of the searched patent publications displayed in order of score on the display, the patent search system can be determined to be useful. [Effects of the Invention]

[0022] As described above, the patent search system disclosed herein for evaluating the similarity between patent publications focuses on IPC classification section symbols, and mechanically collects a large number of patent publications for each section symbol without requiring the knowledge or effort of an experienced patent engineer. It divides the text of the patent publications into segments, and fine-tunes a language model using the large number of segments to create eight sentence encoders dedicated to patent publications for each IPC classification section symbol. The patent search system disclosed herein investigates the similarity between the claims of a patent publication of interest and a patent publication being searched, and when there are multiple patent publications being searched, it makes it possible to search the patent publications being searched in order of score based on similarity. [Brief explanation of the drawings]

[0023] [Figure 1] FIG. 2 is a block diagram showing the configuration of a sentence encoder according to the first embodiment. [Figure 2] 10 is a flowchart of retraining a language model. [Figure 3] FIG. 10 is a diagram showing segment division in an actual claim. [Figure 4] FIG. 10 is a block diagram showing the configuration of a patent research system according to a second embodiment. [Figure 5] 10 is a flowchart of a calculation of the similarity of claims. [Figure 6] FIG. 1 is a block diagram showing the configuration of a patent research system using an H-section text encoder. [Figure 7] FIG. 10 is a conceptual diagram showing combinations of segment symbols and similarities. [Figure 8] FIG. 10 is a block diagram showing the configuration of a patent research system according to a third embodiment. [Figure 9] FIG. 8 is a conceptual diagram showing calculation of a score based on the similarity in FIG. 7. [Figure 10] FIG. 10 is a diagram showing the results of the performance evaluation method of the patent search system of the fourth embodiment as points. DETAILED DESCRIPTION OF THE INVENTION

[0024] Below, embodiments of the patent search system 10 and the performance evaluation method of the patent search system 10 of the present disclosure are shown using figures. Note that the embodiments are merely disclosures and do not limit the present disclosure in any way, and it goes without saying that various improvements and modifications are possible within the scope of the gist thereof.

[0025] Patent publications are classified according to the International Patent Classification (IPC). The IPC classification has a hierarchical structure with five levels: Section, Class, Subclass, Main Group, and Subgroup. Each patent publication is typically assigned multiple IPC classifications. The first IPC classification listed represents the invention. However, patent publications are broadly divided into eight IPC classifications, and the technical content is so diverse that the same word may have different meanings in patent publications with different IPC classifications. For example, to accurately grasp the different meanings of words in different fields, such as "coverall" (Section A: clothing coverall; Section B: bridging beam), "lead" (Section B: pencil; Section C: lead oxide), and "membrane" (Section A: mucous membrane, retina; Sections G and H: thin film), it is necessary to learn the appropriate context for each specialized field of the patent publication.

[0026] (First embodiment) The first embodiment made to solve the above problem is a patent research system 10 using a sentence encoder 1 based on a language model that calculates the similarity between the text of each claim in the claims of a patent publication 80 of interest and the text of each claim in the claims of a patent publication 90 being investigated, in which the sentence encoder 1 is characterized in that the patent research system 10 is re-trained using the segments 72, which are separated by stop words in a division unit 20 from the text of the claims and specifications of a large number of patent publications 70 that have the same IPC classification section symbol listed at the beginning of the patent publication 80 of interest, and are divided into segments 72 separated by punctuation marks, each of which is approximately 1 / 4 of the number of tokens limited by the input limit of the language model.

[0027] According to the first embodiment, the sentence encoder 1 used in the patent research system 10 removes stop words from the claims and specification text of a large number of patent publications 70 (by IPC classification section) that have the same IPC classification section symbol listed at the top of the patent publication 80 of interest (by IPC classification section) and divides them into segments 72 divided by punctuation marks, each of which is approximately 1 / 4 of the number of tokens that is the input limit of the language model, in a division unit 20, and then re-trains using the segments 72.

[0028] As a result, the sentence encoder 1 used in the patent search system 10 can be prepared with eight types of sentence encoders 1 (1A, 1B, 1C, 1D, 1E, 1F, 1G, and 1H) that have been retrained to correspond to the section symbols of the IPC classification. These eight types of sentence encoders 1 are created by not only understanding words through pre-training the language model, but also by fine-tuning with different patent publications 70 for each section symbol of the IPC classification. Therefore, they have acquired knowledge about the words described in the patent publications corresponding to the IPC classification, enabling searches that are tailored to the IPC classification of the patent publication. In other words, the meanings of words and word combinations that may differ depending on the IPC classification can be accurately captured in context.

[0029] FIG. 1 is a block diagram showing the configuration of a sentence encoder 1 according to a first embodiment. Patent publications 70 are patent publications with the same IPC section symbol listed at the top. A large number of patent publications 70 were mechanically acquired using a commercial patent search system, focusing on patent publications with the same section symbol listed at the top IPC classification. Approximately 200,000 patent publications 70 were acquired by section. The text of each claim and the text of a description of a preferred embodiment of the invention are divided into segments 72, each consisting of one to 200 punctuation characters, by removing stop words in a segmentation unit 20, which is half or less of the token input limit of the language model. A language model 12 is retrained (fine-tuned) using the segments 72 to create sentence encoders 1 for each IPC section symbol. Eight types of sentence encoders 1 corresponding to the IPC section symbols are designated 1A, 1B, 1C, 1D, 1E, 1F, 1G, and 1H.

[0030] When obtaining Patent Publication 70, we used the IPC classification listed first as a clue because, in a patent, when the content of an invention is composed of a combination of multiple technologies, an IPC classification symbol corresponding to each technology is assigned, and the IPC classification symbol of the most central technical content is listed first.Therefore, it can be said that the first IPC classification symbol often explains the content of the first claim.

[0031] The language model 12 is, for example, BERT (Bidirectional Encoder Representations from Transformers), a publicly available, general-purpose large-scale language model. This language model is pre-trained using general-purpose data composed of text unbiased to any particular field, such as Wikipedia (registered trademark). However, because it does not learn appropriate contexts for specific specialized fields, it is necessary to retrain (fine-tune) each of the eight technical fields for each IPC classification of the patent publication, especially for patent publications containing many technical terms written with highly specialized words and word combinations. In practice, fine-tuning was performed by preparing 50,000 cases for each class one level below the eight sections of the IPC classification. For example, in the G section, patent publications with the highest IPC corresponding to classes G01, G02, G03, and G06 account for more than 75% of the patent publications with the highest IPC in the entire G section. Therefore, 50,000 cases were prepared for each class and fine-tuned, creating sentence encoders for each of the classes G01, G02, G03, and G06. Furthermore, we fine-tuned using all patent publications from these four classes to create a sentence encoder for the entire G section. Similarly, for the other sections, we selected classes that accounted for more than 75% of the patent publications of the top IPC in the entire section, prepared 50,000 entries for each class, and fine-tuned them to create a sentence encoder for each class. We then fine-tuned using all patent publications from these classes to create a sentence encoder for the entire section.

[0032] Figure 2 is a flowchart of the language model retraining process. In step S202, patent publications 70 are those with the same section symbol in the first IPC classification. Approximately 50,000 patent publications with the same section symbol in the first IPC classification were mechanically retrieved using a commercial patent search system. Approximately 200,000 results retrieved by section are prepared. In step S204, the claims and specification text of the patent publications are extracted. In step S205, the items [Claim*] are removed from the claims of the patent publications, and the items [Brief Description of Drawings] and [Explanation of Symbols] are removed from the specification text. In step S206, stop words are removed from the remaining text, including [Technical Field], [Background Art], [Prior Art Documents], [Summary of the Invention], [Disclosure of the Invention], [Problem to be Solved by the Invention], [Means for Solving the Problem], [Advantages of the Invention], and [Form for Carrying Out the Invention], and the text is decomposed into segments 72. In step S208, the BERT of the language model 12 is fine-tuned on the segment 72.

[0033] In the first embodiment, fine tuning was performed using approximately 50,000 patent publications 70 by IPC classification (200,000 in the G section and 150,000 in the H section). A certain number of 10,000 can be expected. Fine tuning was performed up to approximately 200,000, and it was confirmed that approximately 50,000 is sufficient for practical use. A larger number may also be used. In the case of new technology, a smaller number may also be acceptable. Furthermore, if the patent publication 80 of interest can be limited, the IPC classification may be limited from section to class and subclass. In this case, fine tuning can be performed with 10,000. In this application, multiple refers to approximately 2 to 100, and large number refers to approximately 10,000 to 200,000.

[0034] Stop words are words that are not considered important for the purpose of processing input text in natural language processing. By removing stop words, it is possible to exclude words that are not directly related to the specialized field for which the language model is being fine-tuned, and only target relevant words and phrases.

[0035] Stop words in patent publication claims and specifications are words that have been removed as terms that appear frequently in the text, and in the case of Japanese patents, examples include, but are not limited to, "claim*", "said", "said", "feature", "or", "including", "describe", "any of", "described in", "said one or more", "in", "in", "comprise", "comprises", "and", "or", "or", "or", "and", "and", etc.

[0036] A segment 72 is a unit of text that is input to the sentence encoder 1, which is a fine-tuned version of the language model 12. The sentence encoder 1 vectorizes each token (a processing unit such as a word) included in the segment 72, and converts each segment 72 into a set of vectors. BERT for language model 12 generally has a limit of 512 tokens for the number of input tokens (equivalent to approximately 700 to 800 characters in terms of segment length, which is the number of characters). However, if input up to this limit were allowed, the vector representation obtained from Sentence Encoder 1 would be used to calculate the similarity between texts using BERTScore, which requires exponential calculation time; for example, doubling the segment length would increase the calculation time by four times, and the processing time required for calculations on a general-purpose computer such as a personal computer would become unacceptable in practical terms. Therefore, the segment length was set to 128 tokens or less, which is one-quarter of the upper limit of the input token limit for BERT for language model 12. This corresponds to a segment length of 50 to 150 characters in terms of the number of characters, but it is long enough to express the meaning of a relatively short sentence, and experiments have demonstrated that even if patent claims are divided into segments of this character length as input targets for BERT, each segment 72 is long enough to express meaning according to the degree of similarity with other segments 72. For segment 72, 150 characters is 1 / 4 of the upper limit of the number of tokens, and 50 characters is equivalent to 1 / 12 of the number of tokens.

[0037] Here, segments 72 of 50 to 150 characters were generally divided by punctuation marks. The punctuation marks were "," and "." Furthermore, text with 150 or more characters between punctuation marks was divided into segments of 50 to 150 characters. If the text had fewer than 50 characters, it was combined with the surrounding text and divided into segments of 50 to 150 characters. The reason for dividing the segments into 50 to 200 characters is that this is a sufficient length to express the meaning of the sentence. Even if a patent claim is divided into one or more segments 72, each segment 72 is long enough to express a certain meaning (as confirmed by experiment). Furthermore, if a sentence is too long, even if some parts of the sentence are similar, the overall similarity of the sentences tends to be reduced due to the presence of other dissimilar parts. Conversely, if the sentences are too short, even if the content itself is not similar, the similarity may be significantly high if the sentence contains just one identical word, and there is also a high possibility that similar parts will be interrupted. The characteristics of segment 72, such as the number of characters, are the same as those of segment 82 and segment 92, and both are created by the division unit 20. The segment length is set to 50 to 200 characters in the description of the first embodiment, and 50 to 150 characters in the claims of the second embodiment.

[0038] Figure 3 shows the division of segments 72 in an actual claim. The patent publication is JP 2020-167728 A. Figure 3(1) shows a portion of actual claims 1 and 2. Figure 3(2) shows stop words in black. Figure 3(3) shows the first segment 72 (60 characters), the second segment 72 (95 characters), the third segment 72 (50 characters) of claim 1, and a portion of the fourth segment 72 of claim 2.

[0039] As described above, the eight types of sentence encoders 1 according to the first embodiment fine-tune the language model 12 by IPC classification using segments 72 from approximately 200,000 patent publications 70. Therefore, in addition to the basic understanding of common words that the language model has been pre-trained with, the eight types of sentence encoders created by fine-tuning with different patent publications for each section symbol of the IPC classification are expected to acquire knowledge about words contained in patent publications corresponding to the IPC classification.

[0040] (Second embodiment) The second embodiment is preferably the first embodiment, further comprising a division unit 20 that creates segments (82, 92) and corresponding segment symbols (α, β) for each claim in the text of the claims of the patent publication 80 of interest and the patent publication 90 under investigation, a vector generation unit 30 that generates sentence vectors from the segments (82, 92) using a sentence encoder 1, a similarity calculation unit 40 that calculates the cosine similarity for each combination of the set of sentence vectors for each segment 82 of the patent publication 80 of interest and the set of sentence vectors for each segment 92 of the patent publication 90 under investigation, and a display unit 50 that displays the similarity based on the cosine similarity corresponding to the combination of segments for each combination of segment symbols (α, β).

[0041] According to the second embodiment, as shown in Figure 4, the patent search system 10 includes a segmentation unit 20 that creates segments (82, 92) and corresponding segment symbols (α, β) for each claim in the text of the claims of the patent publication 80 of interest and the patent publication 90 under investigation, a vector generation unit 30 that generates a set of sentence vectors from the segments (82, 92) using a sentence encoder 1, a similarity calculation unit 40 that calculates the cosine similarity for each combination of the set of sentence vectors for each segment 82 of the patent publication 80 of interest and the set of sentence vectors for each segment 92 of the patent publication 90 under investigation, and a display unit 50 that displays the similarity based on the cosine similarity corresponding to the combination of segments for each combination of segment symbols (α, β). Note that the patent search system 10 in Figure 4 includes the patent search system 10 in Figure 1, not shown.

[0042] As a result, the patent research system 10 divides the text of the claims of the patent publication 80 of interest and the patent publication 90 under investigation into segments (82, 92) for each claim, and generates a set of sentence vectors for each segment (82, 92) using the sentence encoder 1. The cosine similarity of all possible combinations of each sentence vector belonging to the sets of sentence vectors of the patent publication 80 of interest and the patent publication 90 under investigation is calculated, and the similarity based on this is displayed for each combination of segment symbols (α, β). In other words, the similarity between the claims of the two patent publications can be evaluated based on the segments (82, 92) that are the source of the sentence vectors for calculating the cosine similarity.

[0043] FIG. 5 is a flowchart for calculating the similarity of claims. In step S302, the claim texts of the patent publication 80 of interest and the patent publication 90 under investigation are input. In step S304, the division unit 20 divides the text of each claim into segments (82, 92). Segment symbols are also assigned at the same time. In step S306, the vector generation unit 30 (using BERT) inputs the text and calculates a sentence vector (vector set) for each segment (82, 92). In step S308, the similarity calculation unit 40 (using BERTScore) calculates the similarity for each combination of segments of the patent publication 80 of interest and the patent publication 90 under investigation. In step S310, the similarity for each combination of segments is displayed. The segment combinations are also displayed in order of similarity. In step S312, a score is calculated based on the similarity. Note that step S312 is mainly applied when there are multiple patent publications 90 under investigation in the third embodiment.

[0044] 6 is a block diagram showing the configuration of a patent search system 10 using an H-section sentence encoder 1H. As an example, if the patent publication 80 of interest is Japanese Patent Application Laid-Open No. 2020-167728, the IPC classification listed at the top is H04N5 / 365, and therefore the sentence encoder 1 for calculating the similarity between the newly published patent publication 90 and the subject matter described in the claims of patent publication 80 uses the H-section sentence encoder 1H.

[0045] The text of the claims described in the specification of the patent publication 80 of interest is divided into segments 82 of 50 to 150 characters each for each punctuation mark by the division unit 20, after which stop words are removed for each claim. Next, the vector generation unit 30 creates a vector set 84 from the segments 82 using a sentence encoder 1H fine-tuned for the patent publication 70 that has the same IPC classification section symbol as the first one described in the patent publication 80 of interest.

[0046] The similarity calculation unit 40 calculates the cos similarity between each sentence vector belonging to the vector set 84 corresponding to the segment 82 of the focused patent publication 80 and each sentence vector belonging to the vector set 94 corresponding to the segment 92 of the searched patent publication 90 through a brute force calculation, thereby making it possible to identify, from the searched patent publication 90, the searched patent publication 90 that has a claim highly relevant to the focused patent publication 80 based on the similarity with the focused patent publication 80. The processing performed by the similarity calculation unit 40 uses BERTScore, an evaluation method that calculates the similarity between texts using vector representations obtained from BERT of the fine-tuned language model 12.

[0047] BERTScore has scores for Recall, Precision, and F-value, and the similarity calculated by the similarity calculation unit 40 of the present disclosure uses the F-value of the following formula (3), but it may also be either formula (1) or (2). These formulas (1) to (3) derive the similarity from Ai, which indicates a segment derived from the text of the claim of the patent publication 80 of interest, and Bj, which indicates a segment derived from the text of the claim of the patent publication 90 being investigated.

[0048]

number

[0049] Figure 7 is a conceptual diagram showing the combinations of segment symbols and their similarities. Figure 7(a) is a matrix display with segment symbols α1 to α7 for each of claims 1 to 3 in the patent publication 80 of interest on the vertical axis and segment symbols β1 to β6 for each of claims 1 to 3 in the patent publication 90 under investigation on the horizontal axis. The similarity calculated for each combination of segment symbols is displayed. The similarity is calculated using cosine similarity, where the closer to 1 the similarity is, the greater the similarity, and the closer to 0 the similarity is, the less similarity there is. Figure 7(b) displays the combinations of segment symbols in descending order of similarity. From the above, it is possible to confirm the similarity between the claims and segment symbol β of the patent publication 90 under investigation for each claim and segment symbol α in the patent publication 80 of interest.

[0050] As described above, according to the second embodiment, the patent search system 10 divides the text of the claims of the focused patent publication 80 and the searched patent publication 90 into segments (82, 92) for each claim, and generates a set of sentence vectors for each segment (82, 92) using the sentence encoder 1. The similarity can be calculated for all combinations of sentence vectors of the focused patent publication 80 and the searched patent publication 90. The similarity can be displayed in a matrix for each combination of claim and segment symbol (α, β), allowing an overview of the similarity between the claims of the focused patent publication 80 and the searched patent publication 90. The segment symbols (α, β) can also be displayed in order of similarity. In other words, the similarity between the claims of the two patent publications can be confirmed based on the segments (82, 92) that form the source of the sentence vectors generated by the sentence encoder 1.

[0051] (Third embodiment) In the third embodiment, when there are multiple patent publications 90 to be investigated in the second embodiment, it is desirable to have a display unit 50 that calculates a score based on the similarity between the patent publication 80 of interest and each of the patent publications 90 to be investigated, and displays the patent publications 90 to be investigated based on the score.

[0052] According to the third embodiment, when there are multiple patent publications 90 to be searched, the patent search system 10 is provided with a display unit 50 that calculates a score based on the similarity between the patent publication 80 of interest and each of the patent publications 90 to be searched, and displays the patent publications 90 to be searched based on the score.

[0053] As a result, when there are a plurality of patent publications 90 to be searched, for example 100, the patent search system 10 can calculate the similarity between the patent publication 80 of interest and each of the patent publications 90 to be searched, calculate a score based on an aggregate value of all or part of the similarities or an average value thereof, and display the patent publications 90 to be searched in order based on the score on the display unit 50. In other words, the 100 patent publications 90 to be searched can be checked starting from the important patent publications 90 to be searched that have the highest similarity based on the score.

[0054] As a usage example of the third embodiment, in patent research work in, for example, a technical department or research institute of a manufacturing company, a patent research system 10 can be considered which has a sentence encoder 1 created by fine-tuning a language model, in which newly published patent publications 90 are searched for patent publications which have similar claims to a patent publication 80 which has been judged to be important for reasons such as a high relevance to an upcoming product or unique technology being developed in the technical department or research institute to which the researcher belongs, thereby viewing and evaluating patent publications 90 in order of their degree of similarity to patent publication 80, with the aim of streamlining patent research work.

[0055] FIG. 8 is a block diagram showing the configuration of a patent search system 10 according to a third embodiment. The patent search system 10 includes a segmentation unit 20 that creates segments (82, 92) and corresponding segment symbols (α, β) for each claim in the text of the claims of a patent publication 80 of interest and multiple patent publications 90 under investigation; a vector generation unit 30 that generates a set of sentence vectors from the segments (82, 92) using a sentence encoder 1; a similarity calculation unit 40 that calculates the cosine similarity for each combination of the set of sentence vectors for each segment 82 of the patent publication 80 of interest and the set of sentence vectors for each segment 92 of the patent publications 90 under investigation; and a display unit 50 that displays the similarity corresponding to each combination of segment symbols (α, β). The patent search system 10 in FIG. 8 includes the patent search system 10 in FIG. 1 (not shown).

[0056] The display unit 50 calculates a score based on the similarity calculated for each patent publication 90 under investigation. The score is a value obtained by adding up all or part of the similarities, or an average value thereof. FIG. 9 is a conceptual diagram showing the calculation of a score based on the similarities in FIG. 7. The patent publication 90 under investigation in FIG. 7 is the first patent publication 90 under investigation, and its similarity is displayed here. The same applies to the second and subsequent patent publications 90 under investigation.

[0057] Figure 9 shows the formulas for calculating scores using three methods. Method 1 uses all combinations of segment symbols (combinations of α1 to α7 and β1 to β6, 7 x 6 = 42). As shown in Method 1 in Figure 9, the score is calculated as follows: number of segments n = 42, sum = 14.1, and average value 0.336. However, since there are multiple patent publications 90 surveyed, the number of segments varies. Therefore, it is desirable to use the average value as the score. Method 2 is based on the vertical axis in Figure 7(a). That is, the highest similarity between each segment symbol (α1 to α7) for each claim in the patent publication 80 under investigation and the segment symbol (β1 to β6) for each claim in the patent publication 90 under investigation is used. Therefore, as shown in Method 2 in Figure 9, number of segments n = 7, sum = 5.1, and average value 0.73. Although there are multiple patent publications 90 to be investigated, the number n of segment symbols for each claim in the patent publication 80 of interest is constant, so both the sum and the average can be used as the score. Here, if claim 1 in the patent publication 80 of interest is important, the score can be calculated by using the segment symbols (β1 to β6) for each claim in the patent publication 90 with the highest similarity to the segment symbols (α1 to α4) in claim 1. Method 3 selects five combinations of segment symbols in descending order of similarity. Segment symbols already selected are excluded from the score calculation. Method 3 is shown in Figure 9. The top five most similar pairs are considered similar segments, and the average of their scores is calculated as the similarity between the patent publication 80 and one of the patent publications constituting the patent publication 90 to be investigated. The number n = 5, the sum = 3.6, and the average = 0.72. In the fourth embodiment described below, the average score obtained by Method 3 is used.

[0058] According to the third embodiment, when there are a plurality of, for example, 100, patent publications 90 to be searched, the patent search system 10 calculates the similarity between the patent publication 80 of interest and each of the patent publications 90 to be searched, calculates a score based on an aggregate value of all or part of the similarities or an average value thereof, and displays the patent publications 90 to be searched in order based on the score on the display unit 50. In other words, the 100 patent publications 90 to be searched can be checked starting from the important patent publications 90 to be searched that have the highest similarity based on the score.

[0059] (Fourth embodiment) In the fourth embodiment, in the performance evaluation method of the patent research system 10 of the third embodiment, a patent publication 80 of interest is selected, and in the patent information platform such as a notice of reasons for refusal or a search report, multiple patent publications that are related to the patent publication 80 of interest through citation or cited relationships are selected to be used as the correct data for the patent publication 90 to be investigated, and further, as the test data for the patent publication 90 to be investigated, multiple patent publications that have the same IPC classification section, class, subclass, and main group listed at the top of the patent publication 80 of interest are selected, and a test set consisting of the patent publication 80 of interest, the correct data, and the test data is preferably used.

[0060] According to the fourth embodiment, the performance evaluation method of the patent research system 10 selects a patent publication 80 of interest, and selects multiple patent publications that are related to the patent publication 80 of interest through citation or cited relationships in the patent information platform, such as a notice of reasons for refusal or a search report, to use them as the correct data for the patent publication 90 to be investigated, and further selects multiple patent publications that have the same IPC classification section, class, subclass, and main group listed at the top of the patent publication 80 of interest as the test data for the patent publication 90 to be investigated, and a test set consisting of the patent publication 80 of interest, the correct data, and the test data can be used.

[0061] This allows the patent research system 10 to input a test set consisting of the patent publication 80 of interest, correct answer data, and test data, and if the correct answer data is ranked high in the patent publications 90 to be researched that are displayed in score order on the display unit 50, it can be determined that the patent research system 10 is useful.

[0062] In the fourth embodiment, the following three sets are used as sets of patent publications as a means for objectively verifying the performance of the sentence encoder 1. The first set is a set Ai (i=1,...,N:N≧1000) of specific patent publications to be evaluated in the following formula (4), and the second set is a set Bij (j=1,...,Mi) of cited documents, where each Ai in the following formula (5) is a patent publication cited in a notice of rejection or a search report during the examination by the Patent Office, and each Ai is a set given a semantic interpretation that there are similar or identical parts in the examination by the Patent Office of the second set. The third set is a set Cik(k=1,···,Li) consisting of patent publications that have the same section symbol, main class symbol, subclass symbol, or main group symbol in the head IPC of each Ai in the following formula (6), and is a set that has been given a semantic interpretation by a patent office examiner as belonging to the same category as the individual technical problems possessed by each of the set Ai(i=1,···,N:N≧1000) of N patent publications to be evaluated. In this way, the second set Bij(j=1,···,Mi) and the third set Cik(k=1,···,Li) can be said to be sets that have been associated with the first set Ai(i=1,···,N:N≧1000) by the examiner and given meaning.

[0063]

number

[0064] FIG. 10 is a graph showing the results of the performance evaluation method of the patent research system 10 of the fourth embodiment, expressed as points. FIG. 10(a) shows the ranking of the correct data for each test set. The vertical axis represents the ranking from 1st to 100th, and the horizontal axis represents the test set number. 1,200 test sets were evaluated. From an overall perspective, the correct data generally ranks within the top 10. FIG. 10(b) shows the results of the correct data for test set i=255. The patent publication 80 of interest is Japanese Patent Application Laid-Open No. 2014-199247. Its top IPC classification is G01N27 / 409. The test data are 99 cases whose IPC classification section, class, subclass, and main group listed at the top of Japanese Patent Application Laid-Open No. 2014-199247, one of the patent publications 80 of interest, are the same as G01N27. As a result, five pieces of data were selected as correct answer data, with JP 2015-132492 in first place, JP 2015-132471 in second place, JP 2016-156824 in sixth place, JP 2019-012010 in tenth place, and JP 2018-128369 in thirteenth place. That is, out of a total of 104 pieces of data, the correct answer data ranked first, second, sixth, tenth, and thirteenth. The correct answer data was almost always within the top ten, and in determining the similarity with the target patent publication 80, the correct answer data, which is a second set given a semantic interpretation that the patent office's examination contained similar or identical parts to the target patent publication 80, was able to be detected roughly within the top ten out of a total of 104 pieces of data, so it can be said that the patent search system 10 demonstrates high performance.

[0065] The patent information platform used was that of the National Center for Industrial Property and Training, an independent administrative institution. Citation or cited relationships refer to cited documents such as rejection notices or search reports prepared by the Patent Office, and include patent publications of cited documents in the cited document information. However, this does not include prior art documents cited by applicants in patent application specifications.

[0066] According to the fourth embodiment, test data is input into the patent research system 10, and if the correct data is ranked high in the patent publications 90 to be researched, which are displayed in order of score on the display unit 50, the patent research system 10 can be determined to be useful.

[0067] As described above, the patent search system 10 for evaluating the similarity between patent publications shown in the first embodiment focuses on IPC classification section symbols. It mechanically collects numerous patent publications by section symbol, without requiring the knowledge or effort of an experienced patent engineer. It divides the text of the patent publications into segments and uses eight sentence encoders 1 for each IPC classification section symbol, each of which is specifically designed for patent publications, with fine-tuned language models. Furthermore, the patent search system 10 using the sentence encoder 1 of the second embodiment investigates the similarity between the claims of a focused patent publication 80 and a search target patent publication 90. The patent search system 10 of the third embodiment, when there are multiple search target patent publications 90, can search the search target patent publications 90 in order of score based on similarity. In the fourth embodiment, the effectiveness of the patent search system 10 of the third embodiment was demonstrated using a performance evaluation method using a test set of 1,200 cases. [Explanation of symbols]

[0068] 1···Sentence encoder, 1H···Sentence encoder for H section, 10···Patent search system, 20···Segmentation unit, 30···Vector generation unit, 40···Similarity calculation unit, 50···Display unit, 70···Patent publication for each IPC classification section symbol, 72···Segment, 80···Patent publication of interest, 82···Segment, α1···Segment symbol, 90···Patent publication to be searched, 92···Segment β1···Segment symbol

Claims

1. A patent research system using a sentence encoder based on a language model that calculates the similarity between the text of each claim in the claims of a patent publication of interest and the text of each claim in the claims of patent publications being searched, characterized in that the sentence encoder removes stop words in a segmentation section from the text of the claims and specifications of a large number of patent publications that have the same IPC classification section symbol listed at the beginning of the patent publication of interest, and divides them into segments divided by punctuation marks, the number of which is less than 1 / 4 of the number of tokens that is the input limit of the language model, and then re-trains using the segments.

2. The patent research system described in claim 1 is characterized in that it comprises: a division unit that creates segments and corresponding segment symbols for each claim in the text of the claims of the patent publication of interest and the patent publication being searched; a vector generation unit that generates a set of sentence vectors from the segments using the sentence encoder; a similarity calculation unit that calculates the cosine similarity for each combination of the set of sentence vectors for each segment of the patent publication of interest and the set of sentence vectors for each segment of the patent publication being searched; and a display unit that displays the similarity based on the cosine similarity corresponding to the combination of the segments for each combination of the segment symbols.

3. The patent research system of claim 1 or claim 2 is characterized in that, when there are multiple patent publications being searched, a display unit is provided which calculates a score based on the similarity for each of the patent publications targeted by the search and the patent publications being searched, and displays the patent publications being searched based on the score.

4. A performance evaluation method for a patent research system as described in claim 3, characterized in that a patent publication of interest is selected, and in a patent information platform such as a notice of reasons for rejection or a search report, multiple patent publications that are related to the patent publication of interest through citation or cited relationships are selected as the correct data for the patent publication to be searched, and further, as test data for the patent publication to be searched, multiple patent publications that have the same IPC classification section, class, subclass, and main group listed at the top of the patent publication of interest are selected, and a test set consisting of the patent publication of interest, the correct data, and the test data is used.

Images