Intellectual property business support system, method and program using large-scale language model

Abstract

To support intellectual property business of users.SOLUTION: An input unit (103) of a business support system (1) accepts input of at least a part of a document for inference, which is a document for a patent application or application for utility model registration, to a large-scale language model including a first model in which first training data including a first input part related to a part of a document for training at least and a first output part related to whether or not the part thereof at least should be claimed up has been re-learnt, and another model used supplementarily for inference related to whether or not the part thereof at least should be claimed up. An output acquisition unit (104) acquires an output related to whether or not at least a part of the document for inference should be claimed up from the large-scale language model. A business support unit (105) supports intellectual property business by determining, based on the output, a text subject to claim-up among texts included in the document for inference, and recommending the text subject to the claim-up to a user.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present disclosure relates to an intellectual property business support system, method, and program that utilizes a large-scale language model. [Background technology]

[0002] Conventionally, technologies for supporting tasks related to documents for patent applications or utility model registration applications have been considered. For example, Patent Document 1 describes a technology that extracts from a database a number of documents similar to an invention text entered by a user, and automatically generates a patent application specification including a means for solving a problem and a form for carrying out the invention, based on the invention text and a document selected by the user from the multiple documents extracted from the database. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2023-007268 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technology of Patent Document 1 can support users, such as the applicant's staff or agents, in preparing specifications, but cannot support other tasks. For example, a user may search for unclaimed parts in a specification in order to consider whether or not to file a divisional application. The technology of Patent Document 1 cannot support intellectual property tasks, such as this task.

[0005] One of the purposes of this disclosure is to assist users in their intellectual property management. [Means for solving the problem]

[0006] The intellectual property business support system using the large-scale language model of the present disclosure includes an input unit that inputs at least a portion of the document for estimation into a pre-trained large-scale language model including a first model in which first training data, the first training data including a first input portion related to at least a portion of the document for training and a first output portion related to whether or not the at least a portion should be claimed up, and another model that is used supplementarily in estimating whether or not the at least a portion should be claimed up, the first model being a first model in which first training data including a first input portion related to at least a portion of the document for training and a first output portion related to whether or not the at least a portion should be claimed up, an output acquisition unit that acquires from the large-scale language model an output related to whether or not at least a portion of the document for estimation should be claimed up, and a business support unit that determines text that is subject to claim up among the texts included in the document for estimation based on the output, and supports intellectual property business by recommending the text that is subject to claim up to a user. Effect of the Invention

[0007] According to the present disclosure, it is possible to support users in their intellectual property management. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of a hardware configuration of an intellectual property business support system. [Diagram 2] FIG. 1 is a diagram showing an example of a patent application document. [Diagram 3] 1 is an example of a natural language processing task (NLP task) for the purpose of business support in this embodiment. [Figure 4] FIG. 1 illustrates an example of a large-scale language model. [Diagram 5] FIG. 2 is a diagram showing an example of a language model such as a first model. [Figure 6] FIG. 13 is a diagram showing an example of output data displayed on a business support screen. [Figure 7] FIG. 2 is a diagram showing an example of functions realized in the intellectual property business support system. [Figure 8] FIG. 1 is a diagram showing an example of an application document database. [Figure 9] FIG. 2 is a diagram showing an example of a first training database. [Figure 10] FIG. 13 is a diagram showing an example of a second training database. [Figure 11] FIG. 13 is a diagram showing an example of a third training database. [Figure 12] FIG. 2 is a diagram showing detailed examples of language models such as a first model. [Figure 13] FIG. 2 is a diagram showing an example of processing executed in the intellectual property business support system. [Figure 14] FIG. 2 is a diagram showing an example of processing executed in the intellectual property business support system. [Figure 15] FIG. 13 is a diagram showing an example of functions realized in a modified example of an intellectual property business support system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] [1. Hardware configuration of intellectual property management support system] An example of an embodiment of an intellectual property business support system (hereinafter, "intellectual property business" may be simply referred to as business) according to the present disclosure will be described. FIG. 1 is a diagram showing an example of a hardware configuration of the business support system. For example, the business support system 1 includes a server 10 and a user terminal 20. Each of the server 10 and the user terminal 20 can be connected to a network N such as the Internet or a LAN.

[0010] The server 10 is a server computer. For example, the server 10 includes a control unit 11, a storage unit 12, and a communication unit 13. The control unit 11 includes at least one processor. The storage unit 12 includes at least one of a volatile memory such as a RAM and a non-volatile memory such as a flash memory. The communication unit 13 includes at least one of a communication interface for wired communication and a communication interface for wireless communication.

[0011] The user terminal 20 is a user's computer. For example, the user terminal 20 is a personal computer, a smartphone, or a tablet. For example, the user terminal 20 includes a control unit 21, a storage unit 22, a communication unit 23, an operation unit 24, and a display unit 25. The hardware configurations of the control unit 21, the storage unit 22, and the communication unit 23 may be similar to those of the control unit 11, the storage unit 12, and the communication unit 13, respectively. The operation unit 24 is an input device such as a touch panel or a mouse. The display unit 25 is a display such as a liquid crystal or organic EL.

[0012] The programs stored in the storage units 12 and 22 may be supplied to each computer via the network N. Each computer may also include at least one of a reading unit (e.g., a memory card slot) that reads a computer-readable information storage medium and an input / output unit (e.g., a USB port) for inputting and outputting data to and from an external device. For example, a program stored in an information storage medium may be supplied to each computer via at least one of the reading unit and the input / output unit.

[0013] Furthermore, the business support system 1 only needs to include at least one computer, and is not limited to the example of FIG. 1. For example, the business support system 1 may include only the server 10 without including the user terminal 20. In this case, the user terminal 20 exists outside the business support system 1. The business support system 1 may only include the user terminal 20 without including the server 10. In this case, the server 10 exists outside the business support system 1. The business support system 1 may include a computer not shown in FIG. 1.

[0014] [2. Overview of the business support system] In this embodiment, a case where a person who has a job related to intellectual property corresponds to the user is taken as an example. The user may be any person who uses the business support system 1, and is not limited to the example of this embodiment. For example, a user may be an employee belonging to the intellectual property department, an employee belonging to a department other than the intellectual property department, an agent for a patent application or utility model registration application, a person working at the agent's office, a person whose job is to research and analyze intellectual property, an inventor, or any other person.

[0015] Furthermore, in this embodiment, an example is given of a case where the business support system 1 supports the business of a user who considers whether or not to file a divisional application. For example, a user considers whether or not to file a divisional application using a patent application or utility model registration application filed by his or her company as an original application. In this embodiment, a patent application or utility model registration application in Japan is taken as an example, but the business support system 1 may also support the user's business regarding patent applications or utility model registration applications in countries or regions other than Japan.

[0016] 2 is a diagram showing an example of documents for a patent application. For example, a patent application includes a plurality of documents such as a patent application (application), a specification, claims, an abstract, and drawings. Documents for utility model registration applications are generally similar in structure. In this embodiment, the business support system 1 supports the user's business by using a large-scale language model M to detect, from at least one of the specification and drawings, a portion that indicates the characteristics of an invention or device and that has not yet been claimed.

[0017] Claim-up is an act in which a user or other person (including a computer) creates all or part of a patent claim or utility model registration claim based on at least one of the description in the specification and drawings. Although the entire scope of a patent claim or utility model registration claim is sometimes referred to as a claim, in this embodiment, the scope of a patent claim or utility model registration claim means the name of a document, and the claim means the content described in the document. Note that in some countries or regions, the specification may include the scope of a claim. In some countries or regions, claim-up may be possible from documents other than the specification and drawings.

[0018] 3 is a diagram showing an example of a natural language processing task (NLP task) for the purpose of business support in this embodiment. In this embodiment, a task (UEE task: Unclaimed Embodiment Extraction task) for estimating whether or not at least a part of a document such as a patent application should be claimed up is defined as an example of the natural language processing task. The UEE task is formulated, for example, in the following form.

[0019]

number

[0020] The above p_i,j_i correspond to text such as the jth paragraph in the i-th document (e.g., patent application document). Also, the above C_i corresponds to a claim in the i-th document. These functions f correspond to functions that estimate whether or not a claim should be made about the target text, and output y_i,j_i that correspond to labels indicating whether or not a claim should be made. The target of parameter optimization by re-learning (fine-tuning) in the large-scale language model M described later corresponds to this function.

[0021] In this embodiment, "it would be better to make a complaint" means that there is more room left for complaint consideration for the target text compared to other texts, and that it is recommended that complaint consideration be given priority to the target text over other texts, which corresponds to a suggestion to assist the user in their work.

[0022] The large-scale language model M includes a language model created based on a machine learning technique. For example, the large-scale language model M includes a language model learned by a supervised learning, semi-supervised learning, or unsupervised learning technique. Machine learning in this embodiment is a concept that includes deep learning. The language model included in the large-scale language model M may be, for example, any language model based on a Transformer-Encoder such as BERT (Bidirectional Encoder Representations from Transformers), RoBERTa (Robustly-optimized BERT approach), DeBERTa (Decoding-enhanced BERT with disentangled attention), Longformer, ALBERT (A Lite BERT), or ELECTRA, and may further include a network such as FFNN (Feed-Forward-Neural-Network) as a classification model that treats the vector representation output by the language model as an input.

[0023] FIG. 4 is an example of a large-scale language model M. In this embodiment, the large-scale language model M includes language models of a first model M1, a second model M2, a third model M3,..., an m-th model Mm (in FIG. 4, m is an integer equal to or greater than 4), and classification models such as the m+1-th model Mm+1. In this embodiment, hereinafter, the large-scale language model M includes the first model M1, the second model M2, and the third model M3 as language models, and the fourth model M4 as a classification model. For example, the first model M1 is RoBERTa that has been retrained (fine-tuned).

[0024] In this embodiment, re-learning refers to re-learning of a pre-trained language model. The large-scale language model M in this embodiment includes a pre-trained language model that has been re-trained so as to be able to output a score such as the first score. Here, the pre-trained language model refers to a language model that has been previously subjected to unsupervised learning related to embedded expressions of phrases (tokens) that constitute character strings included in a predetermined language corpus (dataset) such as CC-100. In this embodiment, re-learning of the large-scale language model M refers to re-learning of a pre-trained language model included in the large-scale language model M.

[0025] When the first model M1 is input with text indicating the content of the nth (n is a natural number) paragraph or sentence that is at least part of a document such as a specification in a patent application or utility model registration application, and a claim set in the patent application or utility model registration application, it outputs a first score that indicates whether the text should be claimed up.

[0026] In this embodiment, the text corresponding to the input portion to the language model may be, for example, a character string divided by paragraph numbers (corresponding to a paragraph), a character string divided by punctuation marks (corresponding to a sentence), a character string divided on the condition that the total number of occurrences of punctuation marks reaches a predetermined number, or a unit of a character string divided so that the number of characters is below a predetermined number. Moreover, the text may refer to the entire character string in which paragraphs or sentences corresponding to the character string are combined so that the number of characters does not exceed a predetermined number. Hereinafter, "at least a part of a document" and paragraphs or sentences in a document will simply be read as text.

[0027] A claim set is a collection of at least one claim. Hereinafter, a claim set is simply read as a claim. The first score can be said to indicate whether or not the text has yet been claimed up. This also applies to the second model M2 and the third model M3 described below.

[0028] For example, the second model M2 is a retrained RoBERTa. When the content of a text is input, the second model M2 outputs a second score indicating whether the text shows features of an invention or invention that should be claimed up. The features are essential contents of the invention or invention. For example, the content of the hardware configuration that is the premise of the invention or invention, or the explanation of the prior art, is often not essential contents of the invention or invention, so the second score may be low. The second model M2 is an example of another model.

[0029] For example, the third model M3 is a retrained RoBERTa. When the third model M3 receives the content of a text and a claim set that is the same as the claim set input to the first model M1, it outputs a third score indicating whether the content of the text has already been claimed up. The third score is somewhat similar to the first score, but is a score specialized in whether the content of the text has already been claimed up. The first score is a score that comprehensively represents the second score and the third score. The third model M3 is an example of another model.

[0030] FIG. 5 is a diagram showing an example of a language model such as the first model M1. In this embodiment, each of the first model M1, the second model M2, and the third model M3 is assumed to be retrained RoBERTa, but each of the first model M1, the second model M2, and the third model M3 may adopt any of the above-mentioned types of language models. The example of FIG. 5 shows a case where the text of the nth paragraph of the specification, "This invention is...", is input to the language model. "CLS" in FIG. 5 is a token indicating the beginning of the input. The text of the nth paragraph of the specification is divided into tokens and placed after the token of "CLS". The language model calculates the embedding representation of these tokens, and after the processing of the Transformer-Encoder is performed, calculates the output vector. The output of the language model is input to the classification model. The classification model outputs at least one of a score and a label indicating whether or not it is better to make a claim up.

[0031] As described above, the large-scale language model M in this embodiment includes the first model M1 capable of outputting the first score, which is an integrated score corresponding to whether or not a claim should be made, and the second model M2 and the third model M3 capable of outputting the second score and the third score, the amount of which is reflected in the first score. That is, in addition to the first model M1 that performs a strong estimation according to the integrated criterion of whether or not a claim should be made, the large-scale language model M further includes other models from the second model M2 onwards that perform a weak estimation according to a limited criterion, such as whether or not a claim has been made, which is included in the integrated criterion of whether or not a claim should be made. The other models from the second model M2 onwards are used supplementarily in the estimation of whether or not a claim should be made. As a result, the large-scale language model M can support the weak estimation of the unknown text (input target) with the strong estimation of the unknown text, thereby improving the final estimation accuracy.

[0032] For example, the fourth model M4 (one example of the classification model in FIG. 5) is a support vector machine. The fourth model M4 uses an ensemble learning technique to output an estimation result that comprehensively considers the outputs of each of the language models such as the first model M1, the second model M2, and the third model M3. When a group of scores such as the first score, the second score, and the third score are input, the fourth model M4 outputs a label indicating whether the content of the text should be claimed up or not. The label indicates either a value (e.g., 1) indicating that the text should be claimed up, or a value (e.g., 0) indicating that the content of the nth paragraph should not be claimed up.

[0033] In this embodiment, the contents of the text of a specification of a patent application or utility model registration application and a claim set of the patent application or utility model registration application are input to the large-scale language model M one after another. For each text input to the large-scale language model M, the large-scale language model M outputs a label indicating whether the contents of the text should be claimed up or not. The user terminal 20 displays on the display unit 25 a business support screen that supports the user's business based on the label output by the large-scale language model M. For example, the business support screen displays output data indicating the output from the large-scale language model M.

[0034] FIG. 6 is a diagram showing an example of output data displayed on the business support screen. For example, the output data displayed on the business support screen shows the application number of the application processed by the large-scale language model M, the label output from the large-scale language model M, the score output from the large-scale language model M, the paragraph number of the specification processed by the large-scale language model M, and the text described in the paragraph number. In the example of FIG. 6, if the score is 0.5 or more, the label becomes 1. The score threshold is not limited to 0.5 and may be any threshold. The user checks the contents of the output data and considers whether or not to file a divisional application.

[0035] As described above, the large-scale language model M mainly learns two viewpoints: whether or not features of an invention or device that should be claimed are indicated in each text of the specification, and whether or not the contents of the text have already been claimed. The business support system 1 supports the user's business of considering the necessity of filing a divisional application or other business based on the large-scale language model M. The details of this embodiment will be described below.

[0036] [3. Functions realized by the business support system] FIG. 7 is a diagram showing an example of functions realized by the task support system 1. As shown in FIG.

[0037] [3-1. Functions realized by the server] For example, the server 10 includes a data storage unit 100, a generation unit 101, a learning unit 102, an input unit 103, an output acquisition unit 104, and a business support unit 105. The data storage unit 100 is realized by the memory unit 12. The generation unit 101, the learning unit 102, the input unit 103, the output acquisition unit 104, and the business support unit 105 are realized by the control unit 11.

[0038] [Data storage section] The data storage unit 100 stores data related to business support. For example, the data storage unit 100 stores an application document database DB0.

[0039] FIG. 8 is a diagram showing an example of the application document database DB0. The application document database DB0 is a database in which data related to documents of patent applications or utility model registration applications is stored. For example, the application document database DB0 stores application numbers of patent applications or utility model registration applications, data on claims (patent claims or utility model registration claims), and data on specifications. The data stored in the application document database DB0 is not limited to the example in FIG. 8. For example, the application document database DB0 may store data on text extracted from drawings, data on abstracts, and data on amendments to procedures.

[0040] The data of the specification and the data of the claims may each be in any format, for example, a document file format, a text file format, an XML format, a JSON format, or an HTML format. The application document database DB0 may store not only filed data but also pre-filed data. That is, the processing target in the business support system 1 is not limited to filed data, but may also be pre-filed data. The data of the specification does not need to show the entire text of the specification, and may only show the text following a specific item such as a mode for carrying out the invention.

[0041] For example, the data storage unit 100 stores a first training database DB1, a second training database DB2, and a third training database DB3. The data storage unit 100 may store a fourth training database DB4 in which fourth training data to be learned by the fourth model M4 is stored. Hereinafter, when the first training database DB1, the second training database DB2, the third training database DB3, and the fourth training database DB4 are not differentiated from each other, they are simply referred to as training databases. Similarly, when the first training data, the second training data, the third training data, and the fourth training data described below are not differentiated from each other, they are simply referred to as training data.

[0042] For example, the training data includes an input portion having the same format as the data input to the large-scale language model M at the time of estimation, and an output portion having the same format as the data output from the large-scale language model M at the time of estimation. The training data includes an input portion related to text, and an output portion related to a claim-up of the text. The input portion of the training data is input to the large-scale language model M at the time of learning (training). The output portion of the training data corresponds to a correct answer at the time of learning of the large-scale language model M. The output portion of the training data may include annotations by an administrator.

[0043] In this embodiment, since the specification is the processing target (the specification is the document), the training data includes an input portion related to the training text and an output portion related to the claim-up of the text. Hereinafter, the part described as the document basically means the specification. Note that in this embodiment, since the large-scale language model M processes text, when a drawing corresponds to a document, the text in the drawing is the processing target.

[0044] FIG. 9 is a diagram showing an example of the first training database DB1. In the example of FIG. 9, the first training database DB1 is in JSON format. The first training database DB1 is a database in which the first training data to be learned by the first model M1 or information required to identify the first training data is stored. The first training data includes a first input portion having the same format as the data input to the first model M1 at the time of estimation, and a first output portion having the same format as the data output from the first model M1 at the time of estimation. The first input portion of the first training data is input to the first model M1 at the time of learning. The first output portion of the first training data corresponds to the correct answer at the time of learning of the first model M1.

[0045] For example, the first input portion of the first training data may also include at least a portion of a claim corresponding to the training document (hereinafter, "at least a portion of a claim" treated as an input portion will be read simply as a claim). In this embodiment, an example is given in which the first input portion of the first training data is a claim set including all claims, but the first input portion of the first training data may be a claim set including some of the claims, or may include only a single claim. The first input portion of the first training data may be only a portion of a single claim.

[0046] For example, the first output part of the first training data indicates whether the training text should be claimed up or not. In this embodiment, the first output part of the first training data is taken as an example to be a first label indicating whether the training text should be claimed up or not. The first label indicates either a value indicating that the training text should be claimed up or a value indicating that the training text should not be claimed up.

[0047] In addition, the first output portion of the first training data may be a first score having an intermediate value, instead of a binary value like the first label. The first score indicates the degree (probability) that the training text should be claimed up. The first score can also be said to be a score indicating the likelihood of the first label. For example, when the first score is expressed as a numerical value, the higher the first score, the higher the degree. Conversely, the lower the first score, the higher the degree. The first score may be expressed in other forms, such as characters or other symbols.

[0048] FIG. 10 is a diagram showing an example of the second training database DB2. In the example of FIG. 10, the second training database DB2 is in JSON format. The second training database DB2 is a database in which second training data to be learned by the second model M2 or information required to identify the second training data is stored. The second training data includes a second input portion having the same format as the data input to the second model M2 at the time of estimation, and a second output portion having the same format as the data output from the second model M2 at the time of estimation. The second input portion of the second training data is input to the second model M2 at the time of learning. The second output portion of the second training data corresponds to the correct answer at the time of learning of the second model M2.

[0049] For example, the second input portion of the second training data is a training text. The second output portion of the second training data indicates whether or not the training text describes a feature related to an invention or idea that should be claimed up. In this embodiment, an example is given in which the second output portion of the second training data is a second label indicating whether or not the feature is described in the training text. The second label indicates either a value (e.g., 1) indicating that the feature is described in the training text, or a value (e.g., 0) indicating that the feature is not described in the training text.

[0050] The second output portion of the second training data may be a second score having an intermediate value, instead of a binary value like the second label. The second score indicates the degree (probability) that the training text describes a feature related to an invention or idea that should be claimed up. The second score can also be said to be a score indicating the likelihood of the second label. For example, when the second score is expressed as a numerical value, the higher the second score, the higher the degree. Conversely, the lower the second score, the higher the degree. The second score may be expressed in other forms, such as letters or other symbols.

[0051] FIG. 11 is a diagram showing an example of the third training database DB3. In the example of FIG. 11, the third training database DB3 is in JSON format. The third training database DB3 is a database in which the third training data to be learned by the third model M3 or information required to identify the third training data is stored. The third training data includes a third input portion having the same format as the data input to the third model M3 at the time of estimation, and a third output portion having the same format as the data output from the third model M3 at the time of estimation. The third input portion of the third training data is input to the third model M3 at the time of learning. The third output portion of the third training data corresponds to the correct answer at the time of learning of the third model M3.

[0052] For example, the third input portion of the third training data also includes a claim corresponding to the training document. The third input portion of the third training data is similar to the first input portion of the first training data. The third output portion of the third training data indicates whether the training text has already been claimed up. In this embodiment, the third output portion of the third training data is taken as an example as a third label indicating whether the training text has already been claimed up. The third label indicates either a value (e.g., 1) indicating that the training text has not yet been claimed up, or a value (e.g., 0) indicating that the training text has already been claimed up.

[0053] In addition, the third output portion of the third training data may be a third score having an intermediate value, instead of a binary value like the third label. The third score indicates the degree (probability) that the training text has already been claimed up. The third score can also be said to be a score indicating the likelihood of the third label. For example, when the third score is expressed as a numerical value, the higher the third score, the higher the degree. Conversely, the lower the third score, the higher the degree. The third score may be expressed in other forms, such as letters or other symbols.

[0054] For example, the fourth training database DB4 is a database in which fourth training data to be learned by the fourth model M4 is stored. The fourth training data includes a fourth input portion having the same format as the data input to the fourth model M4 at the time of estimation, and a fourth output portion having the same format as the data output from the fourth model M4 at the time of estimation. The fourth input portion of the fourth training data is input to the fourth model M4 at the time of learning. The fourth output portion of the fourth training data corresponds to the correct answer at the time of learning of the fourth model M4.

[0055] For example, the fourth input portion of the fourth training data includes an output from the first model M1, an output from the second model M2, and an output from the third model M3. The fourth input portion of the fourth training data may include a first score, a second score, and a third score, or the fourth input portion of the fourth training data may include a first label, a second label, and a third label. For example, the first score may be output from the first model M1 together with the first label. The second score may be output from the second model M2 together with the second label. The third score may be output from the third model M3 together with the third label. For example, the first score, the second score, and the third score may be any value between 0.0 and 1.0.

[0056] For example, the fourth output part of the fourth training data indicates whether the training text should be claimed up. In this embodiment, the fourth output part of the fourth training data is taken as an example to be a fourth label indicating whether the training text should be claimed up. The fourth label indicates either a value (e.g., 1) indicating that the training text should be claimed up, or a value (e.g., 0) indicating that the training text should not be claimed up.

[0057] In addition, the fourth model M4 may output a fourth score. The fourth score indicates the degree (probability) that the training text should be claimed up. The fourth score can also be said to be a score indicating the likelihood of the fourth label. For example, when the fourth score is expressed as a numerical value, the higher the fourth score, the higher the degree. Conversely, the lower the fourth score, the higher the degree. The fourth score may be expressed in other forms, such as letters or other symbols.

[0058] For example, the data storage unit 100 stores a large-scale language model M. The large-scale language model M includes a program indicating a process such as obtaining a vector sequence of an input character string, and parameters adjusted by learning. In this embodiment, the large-scale language model M includes a first model M1, a second model M2, a third model M3, and a fourth model M4. Each of the first model M1, the second model M2, and the third model M3 includes a program for a process such as obtaining a vector sequence of an input character string, and parameters adjusted by learning. Hereinafter, when the first model M1, the second model M2, and the third model M3 are not distinguished from each other, they are simply referred to as language models. A group of models including the first model M1, the second model M2, the third model M3, and the fourth model M4 is referred to as a large-scale language model M.

[0059] In this embodiment, since the server 10 includes the learning unit 102, before the processing of the learning unit 102 is executed, the data storage unit 100 stores the first model M1, the second model M2, the third model M3, and the fourth model M4, whose parameters are the initial values. After the processing of the learning unit 102 is executed, the data storage unit 100 stores a model group such as the first model M1, the second model M2, the third model M3, and the fourth model M4 that have been learned. When the learning is executed on a computer other than the server 10, the data storage unit 100 stores the first model M1, the second model M2, the third model M3, and the fourth model M4 that have been re-learned and re-learned on the other computer. In this case, the server 10 does not include the generation unit 101 and the learning unit 102. In this embodiment, an external server other than the server 10 may have a part of the model group (parameters, etc.), and the user terminal 20 may have the part.

[0060] It should be noted that the data storage unit 100 may store other data. For example, the data storage unit 100 may store a program showing a learning algorithm and data of a business support screen.

[0061] [Generation part] The generating unit 101 generates training data. In this embodiment, it is assumed that annotation is performed manually by a user or other person. Annotation is a task in which a user or other person gives a correct answer (output part) to the training data.

[0062] For example, a user performs annotation through the operation unit 24 of the user terminal 20. The user terminal 20 transmits first training data, second training data, third training data, and fourth training data to the server 10. The generation unit 101 receives the first training data, second training data, third training data, and fourth training data from the user terminal 20, and stores them in a first training database DB1, a second training database DB2, a third training database DB3, and a fourth training database DB4, respectively.

[0063] [Learning Department] The learning unit 102 performs re-learning of the large-scale language model M as supervised learning based on the training data. The learning unit 102 performs re-learning of the large-scale language model M such that an output portion of the training data is output when an input portion of the training data is input. The learning algorithm itself may be an algorithm used in a known machine learning method, and parameters such as the weights of the model are updated so that the loss between the true value indicated by the training data and the predicted value output by the model is reduced. For example, the learning unit 102 may perform re-learning of the large-scale language model M based on a gradient descent method, an error backpropagation method, or other methods.

[0064] Here, re-learning refers to updating parameters such as weights in a classification model such as an FFNN connected to a language model. The re-learning may refer to updating parameters such as weights in the language model in addition to parameters such as weights in a classification model such as an FFNN connected to a language model. Note that the learning unit 102 may generalize the language model by updating hyperparameters further using validation data that has been annotated in the same way as the training data.

[0065] For example, the first model M1 is re-learned with first training data including a training text, a first input portion related to a claim corresponding to the text, and a first output portion related to whether the text should be claimed up. The learning unit 102 executes re-learning of the first model M1 so that the output portion of the first training data is output when the input portion of the first training data is input. In other words, the re-learning is executed so that the loss between the output result of the first model M1 and the output portion of the first training data is reduced. The learning algorithm of the first model M1 may also be the known algorithm described above.

[0066] For example, the second model M2 is re-trained with second training data including a second input portion related to a training text and a second output portion related to whether or not the text describes features related to an invention or idea that should be claimed. The learning unit 102 executes re-training of the second model M2 so that the output portion of the second training data is output when the input portion of the second training data is input. In other words, the re-training is executed so that the loss between the output result of the second model M2 and the output portion of the second training data is reduced. The learning algorithm of the second model M2 may also be the publicly known algorithm described above.

[0067] For example, the third model M3 is re-learned with third training data including a training text, a third input portion related to a claim corresponding to the text, and a third output portion related to whether the text has already been claimed up. The learning unit 102 executes re-learning of the third model M3 so that the output portion of the third training data is output when the input portion of the third training data is input. In other words, the re-learning is executed so that the loss between the output result of the third model M3 and the output portion of the third training data is reduced. The learning algorithm of the third model M3 may also be the known algorithm described above.

[0068] For example, the fourth model M4 has been trained with fourth training data including a fourth input portion related to the outputs of the first model M1, the second model M2, and the third model M3, and a fourth output portion related to the final estimation result. The training unit 102 executes training of the fourth model M4 such that the output portion of the fourth training data is output when the input portion of the fourth training data is input. In other words, the re-training is executed so that the loss between the output result of the fourth model M4 and the output portion of the fourth training data is reduced. The training algorithm of the fourth model M4 may also be the known algorithm described above.

[0069] [Input section] The input unit 103 inputs a text for estimation (at least a part of the document for estimation) to a large-scale language model M including a first model trained with first training data including a first input portion related to training text (at least a part of the document for training) and a first output portion related to whether the text should be claimed up, which is a document for a patent application or utility model registration application, and another model used supplementarily in estimating whether the at least a portion of the text should be claimed up. The document related to the text for estimation is a document input to the trained large-scale language model M. In other words, the document for estimation is a document that is the subject of estimation as to whether or not it should be claimed up.

[0070] The estimated text is all or a part of the estimated document. As described above, in this embodiment, the case where each paragraph or sentence of the specification stored in the application document database DB0 corresponds to the estimated text will be described, but multiple paragraphs or sentences or the entire text may correspond to the estimated text. The estimated text may be a character string extracted from the drawing by a method such as optical character recognition or machine learning.

[0071] FIG. 12 is a diagram showing a detailed example of a language model such as the first model M1. As shown in FIG. 12, in this embodiment, the input unit 103 performs some preprocessing on the training or estimation text, and then inputs the preprocessed text to the language model. The input unit 103 acquires, via an embedding layer, an embedded representation of a token group including subtokens such as a token and a classification token ([CLS] token) corresponding to each of the morphemes (phrases) constituting the training or estimation text, and a vector sequence (E_[CLS] to E_N) of an input vector based on an embedded representation of a token group obtained by position encoding of each phrase. In this embodiment, the embedded representation is interpreted as a feature. The input unit 103 may acquire a vector sequence of an input vector by reflecting an embedded representation of a segment indicating a classification of a document (for example, a patent classification) in each token.

[0072] The language model outputs a vector sequence of output vectors (C to T_N) corresponding to a vector sequence of input vectors related to training or estimation text based on a group of language model parameters adjusted by pre-learning. The language model outputs an estimation result corresponding to the training or estimation text by inputting the output vector sequence to a classification model such as FFNN. An output acquisition unit 104 described later acquires an estimation result of the training or estimation text, which is output by inputting scores such as first to third scores indicating the likelihood of the first to third labels to a classification model connected to the language model.

[0073] In addition to the morphological analysis and feature acquisition, the input unit 103 may perform syntactic analysis, context analysis, text proofreading, or other processing as preprocessing. The embedded representation obtained by the preprocessing may be further used to acquire a vector sequence.

[0074] As shown in FIG. 12, the input unit 103 may also input a claim corresponding to the training or estimation document to the language model. The claim is a claim of the same patent application or utility model registration application as the estimation document, or a claim of another application related to the patent application or utility model registration application of the estimation document. The other application is an original application of a divisional application, an application other than the original application of a divisional application (for example, an application for which the Japan Patent Office requests an explanation in the petition), an application on which priority is claimed, or another application with the same embodiment. When the input unit 103 further inputs a claim corresponding to the training or estimation document to the language model, the input unit 103 may combine a group of tokens of the training or estimation text (other than the claim) and a group of tokens of the claim with a subtoken ([SEP] token) between them, and obtain a vector sequence of the input vector through an embedding layer. At this time, the vector sequence of the input vector may be obtained by reflecting an embedded expression of a segment indicating whether it is a claim or not in each token.

[0075] In this embodiment, a claim set consisting of all claims corresponding to the document used for estimation is described as equivalent to the claims corresponding to the document, but a claim set consisting of only some of the claims corresponding to the document, only one claim corresponding to the document, or only a portion of one claim may also be equivalent to the claim corresponding to the document.

[0076] For example, the input unit 103 inputs the text for estimation and the claim corresponding to the document for estimation to the first model M1. When the text for estimation and the claim are input, the first model M1 acquires the vector sequence of these as features. The first model M1 outputs an estimation result according to the features. In this embodiment, a case where the estimation result of the first model M1 is a first label and a first score is taken as an example, but the estimation result may be only the first label or only the first score. When the first model M1 outputs only the first score, it is assumed that it is predetermined whether the first score indicates the degree to which a claim should be made or the degree to which a claim should not be made.

[0077] The large-scale language model of this embodiment includes, as another model, a second model M2 obtained by relearning second training data including a second input portion related to at least a part of the text of a specification or drawing corresponding to a training document, and a second output portion related to whether or not the at least a part of the text describes a feature related to an invention or idea that should be claimed. The input unit 103 inputs a text for estimation (at least a part of the text of a specification or drawing corresponding to a document for estimation) to the second model M2.

[0078] For example, when a text for estimation is input, the second model M2 acquires a vector sequence of the text as a feature. The second model M2 outputs an estimation result according to the feature. In this embodiment, an example is given in which the estimation result of the second model M2 is a second label and a second score, but the estimation result may be only the second label or only the second score. When the second model M2 outputs only the second score, it is assumed that it is predetermined whether the second score indicates the degree to which the features related to the invention or device to be claimed are described, or indicates the degree to which the content is not the feature.

[0079] The large-scale language model of this embodiment includes, as another model, a third model M3 obtained by relearning third training data including a third input portion related to at least a part of the text of a specification or drawing corresponding to a training document, at least a part of the claim corresponding to the training document, and a third output portion related to whether the content of the at least a part of the text has already been claimed up. The input unit 103 inputs the text for estimation (at least a part of the text of a specification or drawing corresponding to the estimation document) and the claim corresponding to the estimation document (at least a part of the claim corresponding to the estimation document) to the third model M3.

[0080] For example, when the third model M3 receives the text for estimation and the claim corresponding to the document for estimation, the third model M3 acquires the vector sequence of these as features. The third model M3 outputs an estimation result according to the features. In this embodiment, the estimation result of the third model M3 is the third label and the third score, but the estimation result may be only the third label or only the third score. When the third model M3 outputs only the third score, it is assumed that it is predetermined whether the third score indicates the degree to which a claim has not already been made or the degree to which a claim has already been made.

[0081] The input to the fourth model M4 is automatically performed inside the large-scale language model M, but the input unit 103 may perform the input to the fourth model M4. When the first score, the second score, and the third score are input, the fourth model M4 outputs an estimation result according to the scores. In this embodiment, a case where the estimation result of the fourth model M4 is the fourth label is taken as an example, but the estimation result may be only the fourth score, or the fourth label and the fourth score. When the fourth model M4 outputs only the fourth score, it is assumed that it is predetermined whether the fourth score indicates the degree to which a claim should be made or the degree to which a claim should not be made.

[0082] [Output acquisition section] The output acquisition unit 104 acquires an output regarding whether the text for estimation (at least a part of the document for estimation) should be claimed up from the large-scale language model M. The output corresponding to the text for estimation is an estimation result of whether the text for estimation should be claimed up. In this embodiment, since the fourth model M4 outputs the final estimation result, the output acquisition unit 104 acquires an output from the fourth model M4.

[0083] For example, the output acquisition unit 104 may acquire from the large-scale language model M an output indicating whether or not the text for estimation should be claimed up. The output acquisition unit 104 may acquire the output from the first model M1. For example, the output acquisition unit 104 may acquire from the large-scale language model M an output indicating whether or not a feature is described in the text for estimation. The output acquisition unit 104 may acquire the output from the second model M2. For example, the output acquisition unit 104 may acquire from the large-scale language model M an output indicating whether or not the text for estimation has already been claimed up. The output acquisition unit 104 may acquire the output from the third model M3.

[0084] [Business Support Department] The business support unit 105 supports business related to the document for estimation (intellectual property business) based on the output acquired by the output acquisition unit 104. The business is business for granting rights to the patent application or utility model registration application of the document for estimation, business for granting rights to a divisional application of the patent application or utility model registration application, or business for evaluating these. The business for granting rights includes not only business at the time of filing, but also other business such as business at the time of voluntary amendment or business at the time of intermediate correspondence.

[0085] For example, the business support unit 105 determines text that is the subject of a claim-up from among texts included in a document for estimation, and supports intellectual property business by recommending the text that is the subject of a claim-up to a user. The recommendation here means displaying the text on the user terminal 20, or transmitting data of the text to the user terminal 20. For example, the business support unit 105 recommends the text that is the subject of a claim-up to a user by transmitting data such as that shown in FIG. 6 to the user terminal 20.

[0086] The business support unit 105 may support a business by executing some processing related to the user's business. In this embodiment, the business support unit 105 supports a business by displaying output data on a business support screen. For example, the business support unit 105 generates output data based on the output acquired by the output acquisition unit 104 and transmits the output data to the user terminal 20, thereby displaying the business support screen on the user terminal 20. The business support unit 105 displays on the user terminal 20 a business support screen including information indicating an estimation text that is estimated to be better filed.

[0087] Furthermore, the business support unit 105 may execute other processes based on the output acquired by the output acquisition unit 104. For example, the business support unit 105 may display on the user terminal 20 a screen that allows a distinction between the estimation text that is estimated to be better filed and other parts based on the output acquired by the output acquisition unit 104.

[0088] In addition, the business support unit 105 may limit the scroll speed while the estimated text that is estimated to be subject to a claim based on the output acquired by the output acquisition unit 104 is displayed, or limit the scroll speed while the text is not displayed, thereby making the display time of the text longer than other parts.

[0089] [3-2. Functions realized on user terminals] For example, the user terminal 20 includes a data storage unit 200, an operation reception unit 201, and a display control unit 202. The data storage unit 200 is realized by the storage unit 22. The operation reception unit 201 and the display control unit 202 are realized by the control unit 21.

[0090] [Data storage section] The data storage unit 200 stores data necessary for supporting a business operation. For example, the data storage unit 200 stores data necessary for displaying a business operation support screen.

[0091] [Operation reception section] The operation reception unit 201 receives various operations by a user. For example, the operation reception unit 201 receives operations on a business support screen. The operation reception unit 201 may receive a designation of an annotation result by a user.

[0092] [Display control section] The display control unit 202 causes various screens to be displayed on the display unit 25. For example, the display control unit 202 causes a business support screen to be displayed on the display unit 25. The display control unit 202 causes the display unit 25 to display a screen for the user to make annotations.

[0093] [4. Processing performed by the business support system] 13 and 14 are diagrams showing an example of processing executed in the task support system 1. The control units 11 and 21 execute programs stored in the storage units 12 and 22, respectively, to execute the processing of Fig. 13 and Fig. 14.

[0094] As shown in Fig. 13, the server 10 executes re-learning of the pre-trained first model M1 based on the first training data stored in the first training database DB1 (S1). The server 10 executes re-learning of the pre-trained second model M2 based on the second training data stored in the second training database DB2 (S2). The server 10 executes re-learning of the pre-trained third model M3 based on the third training data stored in the third training database DB3 (S3). The server 10 executes learning of the fourth model M4 based on the fourth training data stored in the fourth training database DB4 (S4).

[0095] The user terminal 20 accepts the designation of any application number of a patent application or utility model registration application whose application number is stored in the application document database DB0 (S5). The user terminal 20 transmits the application number of the patent application or utility model registration application designated by the user to the server 10 (S6). Upon receiving the application number from the user terminal 20 (S7), the server 10 obtains data on the specification and claims of the patent application or utility model registration application indicated by the application number from the application document database DB0 (S8).

[0096] The server 10 acquires the contents of the text to be processed from the specification acquired in S8 (S9). Each piece of text may be identified by a symbol surrounding a paragraph number and / or a sentence number, and the numerical value of the paragraph number, etc., surrounded by the symbol. The server 10 inputs the contents of the text to be processed and a claim set indicated by the claims acquired in S8 to the first model M1 (S10). The server 10 acquires the output from the first model M1 (S11). The server 10 inputs the contents of the text to be processed to the second model M2 (S12). The server 10 acquires the output from the second model M2 (S13). The server 10 inputs the contents to be processed and the claim set to the third model M3 (S14). Moving to FIG. 14, the server 10 acquires the output from the third model M3 (S15).

[0097] The server 10 inputs the output from the first model M1, the output from the second model M2, and the output from the third model M3 to the fourth model M4 (S16). The server 10 acquires the output from the fourth model M4 (S17). The server 10 determines whether or not all text has been processed (S18). If there is text that has not yet been processed (S18: N), the process returns to S9, and the next text becomes the processing target.

[0098] If it is determined in S18 that all texts have been processed (S18: Y), the server 10 generates display data for a business support screen including the output data based on the output from the fourth model M4 and transmits the display data to the user terminal 20 (S19). The user terminal 20 receives the display data for the business support screen (S20) and displays the business support screen on the display unit 25 (S21), and this process ends.

[0099] [5. Summary of the embodiment] The business support system 1 of this embodiment inputs a text for estimation to a large-scale language model M in which training data has been learned. The business support system 1 obtains an output from the large-scale language model M regarding whether the text for estimation should be claimed up. The business support system 1 determines a text to be claimed up among the texts included in the document for estimation based on the output, and supports intellectual property business by recommending the text to be claimed up to the user. In this way, the business support system 1 can support the intellectual property business of the user. For example, when the user considers whether or not to file a divisional application, it is not necessary to visually search for content that has not yet been claimed up and that indicates the characteristics of the invention or device in at least one of the specification and drawings, so that the business support system 1 can reduce the workload of the user. Similarly, when the user performs intermediate processing work such as responding to a notice of rejection or a rejection decision, it is not necessary to visually search for effective correction materials, so that the business support system 1 can reduce the workload of the user. For example, even if the description in the specification or drawings does not completely match the description contained in the claim, the large-scale language model M can interpret the meaning of words to determine similarity rather than simply determining whether the strings match, so it can accurately determine whether a claim has already been made.

[0100] The business support system 1 also inputs the text for estimation to the second model M2. The business support system 1 obtains an output indicating whether or not the text for estimation describes features of an invention or device that should be claimed up, from the large-scale language model M including the second model M2. This enables the business support system 1 to support the user's task of searching for features of an invention or device from documents of a patent application or utility model registration application.

[0101] Furthermore, the business support system 1 inputs the text to be estimated and a claim corresponding to the document to be estimated to the third model M3. The business support system 1 obtains an output indicating whether the text to be estimated has already been claimed from the large-scale language model M including the third model M3. This enables the business support system 1 to support the business of a user who searches for a description that has not yet been claimed in documents of a patent application or a utility model registration application, for example.

[0102] Moreover, the large-scale language model M includes a first model M1, a second model M2, a third model M3, and a fourth model M4. The business support system 1 inputs the text for estimation and a claim corresponding to the document for estimation to the first model M1. The business support system 1 inputs the text for estimation to the second model M2. The business support system 1 inputs the text for estimation and a claim corresponding to the document for estimation to the third model M3. The business support system 1 acquires an output from the fourth model M4. As a result, the business support system 1 can improve the accuracy of the large-scale language model M by integrating judgment results from different practical perspectives (estimation results by each model that has been subjected to different re-learning) using an ensemble learning technique, and can more effectively support the user's business.

[0103] [6. Modifications] The present disclosure is not limited to the above-described embodiments. The present disclosure can be modified as appropriate without departing from the spirit and scope of the present disclosure.

[0104] 15 is a diagram showing an example of functions realized by the modified task support system 1. The modified task support system 1 includes a reception unit 106. The reception unit 106 is realized by the control unit 11.

[0105] [6-1. Variation 1] For example, the large-scale language model M may not include the first model M1. The large-scale language model M of the first modification includes a second model M2, a third model M3, and a fourth model M4. The second model M2 and the third model M3 of the first modification are similar to the second model M2 and the third model M3 of the embodiment. The fourth model M4 of the first modification is generally similar to the fourth model M4 of the embodiment, but differs from the embodiment in that the output of the first model M1 is not input.

[0106] The fourth model M4 of the first modification has learned fourth training data including a fourth input portion related to the output of each of the second model M2 and the third model M3, and a fourth output portion related to the final estimation result. The fourth training data of the first modification differs from the embodiment in that it does not include the output of the first model M1, but is otherwise similar to the embodiment. The fourth model M4 of the first modification outputs an estimation result based on the output of the second model M2 and the output of the third model M3. The output acquisition unit 104 of the first modification acquires the output from the fourth model M4.

[0107] The large-scale language model M of the first modification does not include the first model M1, but includes a second model M2, a third model M3, and a fourth model M4. Even with such a large-scale language model M, the business support system 1 can support the user's business for the same reasons as those described in the embodiment.

[0108] [6-2. Variation 2] For example, the large-scale language model M may not include the third model M3. The large-scale language model M of the modified example 2 includes a first model M1, a second model M2, and a fourth model M4. The first model M1 and the second model M2 of the modified example 1 are similar to the first model M1 and the second model M2 of the embodiment. The fourth model M4 of the modified example 1 is generally similar to the fourth model M4 of the embodiment, but differs from the embodiment in that the output of the third model M3 is not input.

[0109] The fourth model M4 of the second modification has learned fourth training data including a fourth input portion related to the outputs of the first model M1 and the second model M2, and a fourth output portion related to the final estimation result. The fourth training data of the second modification differs from the embodiment in that it does not include the output of the third model M3, but is otherwise similar to the embodiment. The fourth model M4 of the second modification outputs an estimation result based on the output of the first model M1 and the output of the second model M2. The output acquisition unit 104 of the second modification acquires the output from the fourth model M4.

[0110] The large-scale language model M of the second modification does not include the third model M3, but includes the first model M1, the second model M2, and the fourth model M4. Even with such a large-scale language model M, the business support system 1 can support the user's business for the same reasons as those described in the embodiment.

[0111] [6-3. Variation 3] For example, the large-scale language model M may not include the second model M2. The large-scale language model M of the modified example 3 includes a first model M1, a third model M3, and a fourth model M4. The first model M1 and the third model M3 of the modified example 3 are similar to the first model M1 and the third model M3 of the embodiment. The fourth model M4 of the modified example 3 is generally similar to the fourth model M4 of the embodiment, but differs from the embodiment in that the output of the second model M2 is not input.

[0112] The fourth model M4 of the third modification has learned fourth training data including a fourth input portion related to the outputs of the first model M1 and the third model M3, and a fourth output portion related to the final estimation result. The fourth training data of the third modification differs from the embodiment in that it does not include the output of the second model M2, but is otherwise similar to the embodiment. The fourth model M4 of the third modification outputs an estimation result based on the output of the first model M1 and the output of the third model M3. The output acquisition unit 104 of the third modification acquires the output from the fourth model M4.

[0113] The large-scale language model M of the third modification does not include the second model M2, but includes the first model M1, the third model M3, and the fourth model M4. Even with such a large-scale language model M, the business support system 1 can support the user's business for the same reasons as those described in the embodiment.

[0114] [6-4. Variation 4] For example, the output from the large-scale language model M may be used for other purposes besides the purpose of supporting the user's work. In the fourth modification, a case where the output from the large-scale language model M is used to generate new training data will be described. As described above, the large-scale language model M includes a language model retrained with training data from an integrated perspective of whether a claim should be made, and a language model retrained with training data from a limited perspective of whether a claim has already been made. Therefore, each language model included in the large-scale language model M and the corresponding training data have a certain correlation. This makes it possible to more easily generate training data for other language models according to the output result of an arbitrary language model.

[0115] The generation unit 101 generates new training data based on the text for estimation and the output from the large-scale language model M. The input portion and the output portion of the new training data are similar to the input portion and the output portion of the training data already learned in the large-scale language model M. For example, the generation unit 101 sets the text for estimation input to the large-scale language model M as the input portion of the new training data. The generation unit 101 sets the output from the large-scale language model M as the output portion of the new training data. The generation unit 101 generates new training data including the input portion and the output portion. The generation unit 101 records the new training data in the data storage unit 100.

[0116] In the embodiment, when the large-scale language model M includes a first model M1, a second model M2, a third model M3, and a fourth model M4, the generation unit 101 generates at least one of the first training data, the second training data, the third training data, and the fourth training data. In the fourth modification, a case in which the generation unit 101 generates all of these will be described, but the generation unit 101 may generate only one, two, or three of these.

[0117] For example, the generation unit 101 sets the text for estimation input to the first model M1 and a claim corresponding to the text as an input portion of new first training data. The generation unit 101 sets the output from the first model M1 as an output portion of the new first training data. The generation unit 101 generates new first training data including the input portion and the output portion. The generation unit 101 stores the new first training data in the first training database DB1.

[0118] For example, the generation unit 101 sets the text for estimation input to the second model M2 as an input portion of new second training data. The generation unit 101 sets the output from the second model M2 as an output portion of the new second training data. The generation unit 101 generates new second training data including the input portion and the output portion. The generation unit 101 stores the new second training data in the second training database DB2.

[0119] For example, the generation unit 101 sets the text for estimation and the claim corresponding to at least the part of the text input to the third model M3 as an input portion of new third training data. The generation unit 101 sets the output from the third model M3 as an output portion of the new third training data. The generation unit 101 generates new third training data including the input portion and the output portion. The generation unit 101 stores the new first training data in the third training database DB3.

[0120] For example, the generation unit 101 sets the output from the first model M1, the output from the second model M2, and the output from the third model M3, which are input to the fourth model M4, as the input part of the new fourth training data. The generation unit 101 sets the output from the fourth model M4 as the output part of the new fourth training data. The generation unit 101 generates new fourth training data including the input part and the output part. The generation unit 101 stores the new fourth training data in the fourth training database DB4.

[0121] The generating unit 101 may generate new second and third training data corresponding to other models such as the second model M2 and the third model M3 according to the output result of a specific model such as the first model M1, and store the data in a corresponding database. The generating unit 101 may use a labeling function as appropriate in generating the training data. For example, when the first score, which is the output of the first model M1, exceeds a predetermined threshold value, the generating unit 101 generates the second training data by associating a label corresponding to a value indicating that it is better not to make a claim up with the input text according to the labeling function. The generating unit 101 may also generate training data corresponding to the remaining models (large-scale language models) according to the labeling function based on the output result of the first model M1 and the output results of one or more other models.

[0122] The learning unit 102 further performs re-learning of the large-scale language model M as semi-supervised learning based on new training data. This embodiment differs from the embodiment in that the learning unit 102 further performs re-learning of the large-scale language model M based on new training data, but the learning itself may be the same as the embodiment. The learning unit 102 further performs re-learning of the large-scale language model M such that an output portion of new training data is output when an input portion of new training data is input.

[0123] For example, the learning unit 102 further performs re-learning of the first model M1 based on new first training data. The learning unit 102 further performs re-learning of the first model M1 such that, when an input portion of new first training data is input, an output portion of new first training data is output. The learning unit 102 further performs re-learning of the second model M2 based on new second training data. The learning unit 102 further performs re-learning of the second model M2 such that, when an input portion of new second training data is input, an output portion of new second training data is output.

[0124] For example, the learning unit 102 further re-learns the third model M3 based on new third training data. The learning unit 102 further re-learns the third model M3 such that, when an input portion of new third training data is input, an output portion of new third training data is output. The learning unit 102 further re-learns the fourth model M4 based on new fourth training data. The learning unit 102 further re-learns the fourth model M4 such that, when an input portion of new fourth training data is input, an output portion of new fourth training data is output.

[0125] The business support system 1 of the fourth modification generates new training data based on the text for estimation and the output. The business support system 1 further executes re-learning of the large-scale language model M based on the new training data. This allows the business support system 1 to improve the accuracy of the large-scale language model M. The business support system 1 can save the user the trouble of preparing training data.

[0126] [6-5. Variation 5] For example, in the embodiment, a case has been exemplified in which a configuration not claimed in a certain patent application or utility model registration application is searched for from within the specification of the patent application or utility model registration application. If the patent application or utility model registration application is a divisional application, a configuration not claimed in the patent application or utility model registration application and other patent applications or utility model registration applications including the original application may be searched for from within the specification of the patent application or utility model registration application.

[0127] The training documents of variant 5 are documents of a training divisional application. The input portion of the third training data of variant 5 also includes at least a portion of the claims of the training original application corresponding to the training divisional application. Although the claims are those of the training divisional application or the original application, which differs from the embodiment, the other points of the third input portion of the third training data are similar to the embodiment. For example, the third training data also includes the claims of the training original application. The first input portion of the first training data may include the claims of the training original application.

[0128] The third output portion of the third training data of the fifth modified example indicates whether the training text (at least a part of the training document) has already been claimed in the parent training application. The third output portion of the third training data indicates whether the training text is from the divisional training application or the parent training application, which is different from the embodiment, but the other points of the output portion of the training data are the same as those of the embodiment.

[0129] The presumption document of the fifth modified example is a presumption divisional application document. The input unit 103 also inputs at least a part of the claims of the presumption original application corresponding to the presumption divisional application into the third model M3. The relationship between a certain patent application or utility model registration application and the original application is assumed to be shown in advance in the application document database DB0.

[0130] The present embodiment differs from the embodiment in that the claims input by the input unit 103 to the large-scale language model M are those of the presumed divisional application or the original application, but other points are the same as those of the embodiment. The input unit 103 may input a claim of the presumed divisional application and a claim of the presumed original application to the large-scale language model M. For example, the input to each of the first model M1 and the third model M3 also includes a claim of the presumed original application.

[0131] The output acquisition unit 104 of the fifth modified example acquires an output indicating whether or not the text for estimation has already been claimed in the parent application for estimation from the large-scale language model M. Although the output from the large-scale language model M indicates whether the text for estimation is for the divisional application for estimation or the parent application, the other points of the output from the large-scale language model M are the same as those of the embodiment.

[0132] The training documents in the fifth modified example are documents of a training divisional application. The estimation documents are documents of an estimation divisional application. The business support system 1 also inputs the claims of the estimation original application corresponding to the estimation divisional application into the third model M3. The business support system 1 obtains an output from the large-scale language model M indicating whether or not at least a part of the estimation document has already been claimed in the estimation original application. This allows the business support system 1 to more effectively support the user's business, since it can save the user the trouble of searching for content that has not been claimed not only in a certain patent application or utility model registration application, but also in the original application.

[0133] [6-6. Variation 6] For example, in the fifth modification, when divisional applications are repeatedly filed, the training divisional application may have a plurality of training other applications including the training parent application. In this case, text in the series of other applications may be trained into the large-scale language model M. The meaning of the other applications is as described in the embodiment.

[0134] The third output portion of the third training data of the sixth modification indicates whether the training text has already been claimed in each of the multiple other training applications. The sixth modification differs from the fifth modification in that the third output portion of the third training data targets each of the multiple other applications, but is otherwise similar to the fifth modification.

[0135] In the estimated divisional application of the sixth modification, there are a plurality of estimated other applications including the estimated original application. The input unit 103 also inputs the claims of each of the plurality of estimated other applications to the large-scale language model M. This modification differs from the fifth modification in that the input unit 103 inputs the claims in each of the plurality of other applications to the large-scale language model M, but is otherwise similar to the fifth modification.

[0136] The output acquisition unit 104 of the modified example 6 acquires an output indicating whether or not the text for estimation has already been claimed in each of the multiple other applications for estimation from the large-scale language model M. The output of the large-scale language model M differs from that of the modified example 5 in that it targets each of the multiple other applications, but other points of the output from the large-scale language model M are the same as those of the embodiment.

[0137] In the training divisional application of the sixth modified example, there are a plurality of other training applications including the original training application. In the estimation divisional application, there are a plurality of other estimation applications including the original estimation application. The business support system 1 obtains an output from the large-scale language model M indicating whether the estimation text has already been claimed in each of the plurality of other estimation applications. This allows the business support system 1 to save the user the trouble of searching for content that has not been claimed not only in a certain patent application or utility model registration application, but also in a series of other applications including the original application, thereby more effectively supporting the user's business.

[0138] [6-7. Variation 7] For example, a specification may contain multiple technical contents. In this case, a user may want to specify a technical field that is not claimed. For this reason, the technical contents specified by the user may be input to the large-scale language model M.

[0139] The first input portion of the first training data of the seventh modification also includes technical content for training on a claim-up. The first output portion of the first training data indicates that the content indicated by the input portion is the technical content and indicates whether or not a claim-up should be made. The technical content may be any information capable of classifying a technology from any viewpoint, and may be, for example, a text indicating the technical content, an IPC, an FI, an F-term, or a theme code. The technical content may be a classification adopted in a country other than Japan. The large-scale language model M also learns the technical content. The large-scale language model M of the seventh modification may be a question-answering model used in the field of natural language processing. In this case, the technical content corresponds to a question. The second training data and the third training data may also include technical content for training.

[0140] The business support system 1 of the seventh modified example includes a receiving unit 106. The receiving unit 106 further includes a receiving unit 106 that receives input of technical content for estimation related to a complaint. For example, a user inputs any technical content from the user terminal 20. The user terminal 20 transmits data indicating the technical content input by the user to the server 10. The receiving unit 106 receives the data from the user terminal 20, thereby receiving the input of the technical content.

[0141] The input unit 103 of the seventh modification also inputs the technical content for estimation to the large-scale language model M. The output acquisition unit 104 acquires an output corresponding to the text for estimation and the technical content for estimation from the large-scale language model M. For example, the output from the large-scale language model M indicates the technical content to which the text for estimation is input, and indicates whether or not a claim should be made. The large-scale language model M differs from the embodiment in that it executes estimation according to the technical content for estimation, but may be similar to the embodiment in other respects.

[0142] The first input portion of the first training data of the seventh modified example also includes technical content for training related to a claim-up. The business support system 1 accepts input of technical content for estimation related to a claim-up. The business support system 1 also inputs the technical content for estimation to the large-scale language model M. The business support system 1 obtains output corresponding to the text for estimation and the technical content for estimation from the large-scale language model M. This allows the business support system 1 to save the trouble of searching for content that should be claimed up among the technical content desired by the user, and therefore can more effectively support the user's business.

[0143] [6-8. Variation 8] For example, the large-scale learning model may be trained on progress information in a patent application or utility model registration application. For example, the progress information may be a procedural amendment, a written opinion, a written petition, or other information. In the eighth modification, a case where a claim in a procedural amendment corresponds to the progress information is taken as an example.

[0144] The first input portion of the first training data of the eighth modified example also includes training progress information corresponding to the training documents. For example, the input portion of the training data includes a claim set of a procedural amendment in a patent application or utility model registration application corresponding to the training documents. Although the claim set is a claim set of the patent application or utility model registration application or a claim set of a procedural amendment, the other points are the same as those of the embodiment. The second training data and the third training data may also include training progress information.

[0145] The input unit 103 of the eighth modification also inputs estimation progress information corresponding to the estimation document to the large-scale language model M. The output acquisition unit 104 acquires output corresponding to the estimation text and the estimation progress information from the large-scale language model M. The large-scale language model M differs from the embodiment in that it executes estimation according to the estimation progress information, but may be similar to the embodiment in other respects.

[0146] The first input portion of the first training data of the eighth modified example also includes training progress information corresponding to the training document. The business support system 1 also inputs estimation progress information corresponding to the estimation document to the large-scale language model M. The business support system 1 acquires output corresponding to the estimation text and the estimation progress information from the large-scale language model M. This allows the business support system 1 to take into account the progress information of the patent application or the registered utility model application, thereby more effectively supporting the user's business.

[0147] [6-9. Variation 9] For example, in the embodiment, the case has been described where the business support unit 105 supports business by displaying a business support screen on the user terminal 20. The processing executed by the business support unit 105 is not limited to the example of the embodiment. The business support unit 105 of the ninth modification supports business by evaluating a value (e.g., patent value) related to a document for estimation based on an output from the large-scale language model M. The value may be the degree of evaluation of the document, or the distribution or trend of evaluations of the document.

[0148] In the ninth variant, an example is given in which the value is expressed numerically, but the value may be expressed by letters or other symbols. In the ninth variant, the present invention provides an intellectual property business support system that functions as an intellectual property information analysis system. Also, in the ninth variant, the present invention provides an intellectual property business support system that functions as an intellectual property value assessment system. The intellectual property information analysis system may be a system for analyzing intellectual property information, rather than for the purpose of supporting intellectual property business as described in the embodiment. Therefore, the intellectual property information analysis system of the ninth variant does not need to include the intellectual property business functions described in the embodiment.

[0149] For example, the business support unit 105 evaluates the value so that the more texts that have an output indicating that a claim should be made up from the large-scale language model M, the higher the value of the document for estimation. The evaluation may be the number of texts that have an output indicating that a claim should be made up, or the total value of the fourth score. The evaluation may be adjusted according to the length of the document of the patent application or utility model registration application.

[0150] For example, the business support unit 105 may determine the evaluation by dividing the number of texts for which an output indicating that a claim should be filed is obtained from the large-scale language model M by the total number of texts processed by the large-scale language model M. The business support unit 105 displays the evaluation of the patent application or utility model registration application document on the business support screen. The business support unit 105 may store the evaluation of the patent application or utility model registration application document and its parameters in the application document database DB0, or may send it to the user's email address by email or the like.

[0151] For example, the business support unit 105 may obtain, as an evaluation of the document, a transition of scores for each text constituting the document output by the large-scale language model M consisting of at least the first model M1. Also, for example, the business support unit 105 may obtain, as an evaluation of the document, a distribution of scores for each text.

[0152] For example, the business support unit 105 may determine parameters obtained by applying a predetermined function to the aforementioned score transition (corresponding to time-series data) or distribution and performing parameter fitting as the evaluation parameters of the document.

[0153] For example, the business support unit 105 may search for other documents (e.g., documents of other patent applications) having similar evaluation parameters, for example, from the application document database DB0, and present the other documents found to the user. Here, the other documents found are documents that may have similar practical tendencies to the document for which the evaluation parameters have been determined (evaluated). This may provide a suggestion for business support to the user. The intellectual property value evaluation system of the ninth modification can accurately evaluate the value of documents such as patent applications by evaluating the value of the document for estimation based on the output from the large-scale language model M.

[0154] [6-10. Variation 10] As mentioned above, the large-scale language model M is not limited to the first model M1, the second model M2, the third model M3, and the fourth model M4, and may further include other models.

[0155] For example, the large-scale language model M may further include, as the m-th model Mm, a pre-trained large-scale language model obtained by re-training the m-th training data associated with the input text and a label indicating whether the text satisfies one or more requirements selected from various patent requirement groups including the invention applicability requirement, the industrial applicability requirement, the support requirement, the enablement requirement, and the clarity requirement. As another example, the large-scale language model M may further include a pre-trained large-scale language model obtained by re-training the m-th training data associated with a label indicating whether the text has already been claimed in a related application of the company that is not a divisional application.

[0156] Furthermore, for example, the large-scale language model M may further include, as the m-th model Mm, a pre-trained large-scale language model that has been re-trained with the m-th training data in which the text to be input and a label indicating whether the text is a design item are associated with each other.

[0157] In the m-th model Mm, the m-th training data including the m-th input part related to the training text and the m-th output part indicating whether the text is relevant in a predetermined viewpoint is re-learned. The learning unit 102 executes learning of the second model M2 so that the output part of the m-th training data is output when the input part of the m-th training data is input. That is, the learning algorithm of the m-th model Mm in which the re-learning is performed may be the above-mentioned known algorithm so that the loss between the output result of the m-th model Mm and the output part of the m-th training data is small. Here, the generating unit 101 generates the m-th training data and stores the m-th training data in the m-th training database.

[0158] [6-10. Other variations] For example, the above modifications may be combined.

[0159] For example, the large-scale language model M may not include the fourth model M4. In this case, the large-scale language model M may output each of the first model M1, the second model M2, and the third model M3, or may output a fourth label according to a combination of the outputs of the first model M1, the second model M2, and the third model M3. Whether or not each text of the specification should be claimed is determined based on the combination of the outputs of the first model M1, the second model M2, and the third model M3. The large-scale language model M may be composed of only one of the first model M1, the second model M2, or the third model M3. In this case, one of the first model M1, the second model M2, or the third model M3 corresponds to the large-scale language model M itself.

[0160] For example, in the embodiment, a case has been described in which the main processing is performed by the server 10, but the processing described as being performed by the server 10 may be performed by the user terminal 20 or another computer, or may be shared among multiple computers.

[0161] According to the present invention, it is possible to support users in their intellectual property work by utilizing a large-scale language model that includes a language model that performs strong inferences in line with comprehensive criteria in intellectual property work, as well as a model that performs weak inferences in line with limited criteria in the same work.

[0162] [7. Notes] For example, the business support system can be configured as follows. (1) an input unit for inputting at least a portion of a document for estimation, the document being a patent application or utility model registration application, into a pre-trained large-scale language model including a first model retrained with first training data including a first input portion related to at least a portion of the document for training and a first output portion related to whether or not the at least a portion should be claimed up, and another model used supplementarily in estimating whether or not the at least a portion should be claimed up; an output acquisition unit that acquires an output from the large-scale language model regarding whether or not at least a portion of the estimation document should be claimed; a business support unit that determines a text that is a target of a claim-up from among the texts included in the document for estimation based on the output, and supports intellectual property business by recommending the text that is a target of the claim-up to a user; An intellectual property management support system that uses a large-scale language model including (2) the large-scale language model includes, as the other model, a second model retrained with second training data including a second input portion related to at least a portion of text of a specification or drawing corresponding to the training document, and a second output portion related to whether or not the at least a portion of text describes a feature related to an invention or device that should be claimed; The input unit inputs at least a part of the text of a specification or a drawing corresponding to the document for estimation into the second model. An intellectual property management support system that uses the large-scale language model described in (1). (3) The large-scale language model includes, as the other model, a third model retrained with third training data including a third input portion related to at least a part of text of a specification or drawing corresponding to the training document and at least a part of a claim corresponding to the training document, and a third output portion related to whether the content of the at least a part of the text has already been claimed; The input unit inputs, into the third model, at least a part of the text of a specification or drawings corresponding to the document for estimation and at least a part of the claims corresponding to the document for training. An intellectual property management support system that uses the large-scale language model described in (1) or (2). (4) The large-scale language model is a second model retrained with second training data including a second input portion related to at least a portion of text of a specification or drawing corresponding to the training document, and a second output portion related to whether or not the at least a portion of text describes a feature related to an invention or device that should be claimed; a third model retrained with third training data including a third input portion relating to at least a part of the text of the specification or drawings corresponding to the training document and at least a part of the claim corresponding to the training document, and a third output portion relating to whether the content of the at least a part of the text has already been claimed; a fourth model to which outputs of the first model, the second model, and the third model are input; The other models include The input unit is inputting into the second model at least a portion of the text of a specification or drawing corresponding to the document for estimation; inputting into the third model at least a portion of the text of the specification or drawings corresponding to the document for estimation and at least a portion of the claims corresponding to the document for estimation; The output acquisition unit acquires the output from the fourth model. An intellectual property business support system that uses a large-scale language model according to any one of (1) to (3). (5) The large-scale language model is a second model retrained with second training data including a second input portion related to at least a portion of text of a specification or drawing corresponding to the training document, and a second output portion related to whether or not the at least a portion of text describes a feature related to an invention or device that should be claimed; a fourth model to which outputs of the first model and the second model are input; Including, The input unit inputs at least a part of the text of a specification or a drawing corresponding to the document for estimation into the second model; The output acquisition unit acquires the output from the fourth model. An intellectual property business support system using a large-scale language model according to any one of (1) to (4). (6) The large-scale language model is a third model retrained with third training data including a third input portion relating to at least a part of the text of the specification or drawings corresponding to the training document and at least a part of the claim corresponding to the training document, and a third output portion relating to whether the content of the at least a part of the text has already been claimed; a fourth model to which outputs of the first model and the third model are input; Including, The input unit inputs at least a portion of the text of a specification or drawing corresponding to the document for estimation and at least a portion of the claim corresponding to the document for estimation into the third model; The output acquisition unit acquires the output from the fourth model. An intellectual property business support system using a large-scale language model according to any one of (1) to (5). (7) The intellectual property business support system comprises: A generator that generates new training data based on at least a portion of the document for estimation and the output; a training unit that further performs retraining of the large-scale language model based on the new training data; An intellectual property management support system using the large-scale language model according to any one of (1) to (6), further comprising: (8) the training dossier is a training divisional application dossier, the third input portion also includes at least a portion of a claim of a training parent application that corresponds to the training divisional application; the third output portion indicating whether at least a portion of the training document has already been claimed in the training parent application; The said presumptive document is a presumptive divisional application document, The input unit also inputs at least a portion of the claims of the putative original application corresponding to the putative divisional application into the third model. An intellectual property business support system using the large-scale language model described in (3), (4), or (6). (9) The divisional training application has multiple other training applications, including the parent training application; the third output portion indicating whether at least a portion of the training document has already been claimed in each of the plurality of other training applications; The putative divisional application has multiple putative other applications, including the putative parent application, The input unit also inputs at least a portion of each claim of the plurality of other applications for estimation into the third model. (8) An intellectual property management support system that uses the large-scale language model described above. (10) The first input portion also includes technical content for training on claims up; The intellectual property management support system further includes a receiving unit for receiving an input of technical content for estimation regarding a claim-up, The input unit also inputs the technical content for estimation to the large-scale language model; The output acquisition unit acquires the output corresponding to at least a portion of the document for estimation and the technical content for estimation from the large-scale language model. An intellectual property business support system using a large-scale language model according to any one of (1) to (9). (11) the first input portion also includes training progress information corresponding to the training document; The input unit also inputs, to the large-scale language model, process information for estimation corresponding to the document for estimation; the output acquisition unit acquires the output corresponding to at least a portion of the document for estimation and the progress information for estimation from the large-scale language model. An intellectual property business support system using a large-scale language model according to any one of (1) to (10). [Explanation of symbols]

[0163] 1 Business support system, 10 Server, 11 Control unit, 12 Memory unit, 13 Communication unit, 20 User terminal, 21 Control unit, 22 Memory unit, 23 Communication unit, 24 Operation unit, 25 Display unit, 100 Data memory unit, 101 Generation unit, 102 Learning unit, 103 Input unit, 104 Output acquisition unit, 105 Business support unit, 106 Reception unit, 200 Data memory unit, 201 Operation reception unit, 202 Display control unit, N Network, M Large-scale language model, M1 First model, M2 Second model, M3 Third model, M4 Fourth model, DB0 Application document database, DB1 First training database, DB2 Second training database, DB3 Third training database, DB4 Fourth training database.

Claims

1. an input unit for inputting at least a portion of a document for estimation, the document being a patent application or utility model registration application, into a pre-trained large-scale language model including a first model retrained with first training data including a first input portion related to at least a portion of the document for training and a first output portion related to whether or not the at least a portion should be claimed up, and another model used supplementarily in estimating whether or not the at least a portion should be claimed up; an output acquisition unit that acquires an output from the large-scale language model regarding whether or not at least a portion of the estimation document should be claimed; a business support unit that determines a text that is a target of a claim-up from among the texts included in the document for estimation based on the output, and supports intellectual property business by recommending the text that is a target of the claim-up to a user; An intellectual property management support system that uses a large-scale language model including

2. the large-scale language model includes, as the other model, a second model retrained with second training data including a second input portion related to at least a portion of text of a specification or drawing corresponding to the training document, and a second output portion related to whether or not the at least a portion of text describes a feature related to an invention or device that should be claimed; The input unit inputs at least a part of the text of a specification or a drawing corresponding to the document for estimation into the second model.

2. An intellectual property business support system that utilizes the large-scale language model according to claim 1.

3. the large-scale language model includes, as the other model, a third model obtained by retraining third training data including a third input portion related to at least a part of text of a specification or drawing corresponding to the training document and at least a part of a claim corresponding to the training document, and a third output portion related to whether the content of the at least a part of the text has already been claimed; The input unit inputs, into the third model, at least a part of the text of a specification or drawings corresponding to the document for estimation and at least a part of the claims corresponding to the document for estimation.

3. An intellectual property business support system that utilizes the large-scale language model according to claim 1 or 2.

4. The large-scale language model is a second model retrained with second training data including a second input portion related to at least a portion of text of a specification or drawing corresponding to the training document, and a second output portion related to whether or not the at least a portion of text describes a feature related to an invention or device that should be claimed; a third model retrained with third training data including a third input portion relating to at least a part of the text of a specification or drawing corresponding to the training document and at least a part of the claim corresponding to the training document, and a third output portion relating to whether the content of the at least a part of the text has already been claimed; a fourth model to which outputs of the first model, the second model, and the third model are input; The other models include The input unit includes: inputting into the second model at least a portion of the text of a specification or drawing corresponding to the document for estimation; inputting into the third model at least a portion of the text of the specification or drawings corresponding to the document for estimation and at least a portion of the claims corresponding to the document for estimation; The output acquisition unit acquires the output from the fourth model.

3. An intellectual property business support system that utilizes the large-scale language model according to claim 1 or 2.

5. The large-scale language model is a second model retrained with second training data including a second input portion related to at least a portion of text of a specification or drawing corresponding to the training document, and a second output portion related to whether or not the at least a portion of text describes a feature related to an invention or device that should be claimed; a fourth model to which outputs of the first model and the second model are input; Including, The input unit inputs at least a part of the text of a specification or a drawing corresponding to the document for estimation into the second model; The output acquisition unit acquires the output from the fourth model.

3. An intellectual property business support system that utilizes the large-scale language model according to claim 1 or 2.

6. The large-scale language model is a third model retrained with third training data including a third input portion relating to at least a part of the text of a specification or drawing corresponding to the training document and at least a part of the claim corresponding to the training document, and a third output portion relating to whether the content of the at least a part of the text has already been claimed; a fourth model to which outputs of the first model and the third model are input; and Including, The input unit inputs at least a portion of the text of a specification or drawing corresponding to the document for estimation and at least a portion of the claim corresponding to the document for estimation into the third model; The output acquisition unit acquires the output from the fourth model.

3. An intellectual property business support system that utilizes the large-scale language model according to claim 1 or 2.

7. The intellectual property business support system comprises: A generator that generates new training data based on at least a portion of the document for estimation and the output; a training unit that further performs retraining of the large-scale language model based on the new training data; 3. An intellectual property business support system using a large-scale language model according to claim 1 or 2, further comprising:

8. the training dossier is a training divisional application dossier, the third input portion also includes at least a portion of a claim of a training parent application that corresponds to the training divisional application; the third output portion indicating whether at least a portion of the training document has already been claimed in the training parent application; The said presumptive document is a presumptive divisional application document, The input unit also inputs, into the third model, at least a portion of the claims of the putative original application corresponding to the putative divisional application; 4. An intellectual property business support system utilizing the large-scale language model according to claim 3.

9. The divisional training application has multiple other training applications, including the parent training application; the third output portion indicating whether at least a portion of the training document has already been claimed in each of the plurality of other training applications; The putative divisional application has multiple putative other applications, including the putative parent application, The input unit also inputs at least a portion of each claim of the plurality of other applications for estimation into the third model.

9. An intellectual property business support system utilizing the large-scale language model according to claim 8.

10. The first input portion also includes technical content for training on claims up; The intellectual property management support system further includes a receiving unit for receiving an input of technical content for estimation regarding a claim-up, The input unit also inputs the technical content for estimation to the large-scale language model; The output acquisition unit acquires the output corresponding to at least a portion of the document for estimation and the technical content for estimation from the large-scale language model.

3. An intellectual property business support system that utilizes the large-scale language model according to claim 1 or 2.

11. the first input portion also includes training progress information corresponding to the training document; The input unit also inputs, to the large-scale language model, process information for estimation corresponding to the document for estimation; the output acquisition unit acquires the output corresponding to at least a portion of the document for estimation and the progress information for estimation from the large-scale language model.

3. An intellectual property business support system that utilizes the large-scale language model according to claim 1 or 2.

12. an input step of inputting at least a portion of a document for estimation, the document being a patent application or utility model registration application, into a pre-trained large-scale language model including a first model retrained with first training data including a first input portion related to at least a portion of the document for training and a first output portion related to whether the at least a portion should be claimed up, and another model used supplementarily in estimating whether the at least a portion should be claimed up; obtaining an output from the large-scale language model regarding whether at least a portion of the estimation documents should be claimed; a business support step of determining a text that is a subject of a claim-up from among the texts included in the document for estimation based on the output, and supporting intellectual property business by recommending the text that is the subject of the claim-up to a user; The method includes:

13. an input unit for inputting at least a portion of a document for estimation, the document being a patent application or utility model registration application, into a pre-trained large-scale language model including a first model retrained with first training data including a first input portion related to at least a portion of the document for training and a first output portion related to whether or not the at least a portion should be claimed up, and another model used supplementarily in estimating whether or not the at least a portion should be claimed up; an output acquisition unit that acquires an output from the large-scale language model regarding whether at least a portion of the estimation document should be claimed; a business support unit that determines a text that is a subject of a claim-up from among the texts included in the document for estimation based on the output, and supports intellectual property business by recommending the text that is a subject of the claim-up to a user; A program that makes a computer function as a

Images