Terminal device and prediction system

Abstract

To provide a user interface and a prediction system which allow a user to easily determine whether a patent should be maintained or abandoned, based on a prediction result, even without any knowledge about a model.SOLUTION: A user interface displays a statistical state of at least patents to be abandoned in a display format allowing it to be grasped, based on data of each of predetermined data items indicated by a prediction result of maintenance or abandonment of a previously generated patent predicted by using a learned model for determining maintenance or abandonment of the patent.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present invention relates to a user interface and a forecasting system. [Background technology]

[0002] In order to maintain patent rights, patent fees, etc. must be paid. Patent holders, such as companies, periodically or irregularly evaluate their patent rights and decide whether to maintain or abandon them. In this regard, a technology has been proposed that generates a maintenance period calculation model that calculates an estimated value of the remaining maintenance period of a surviving patent right based on the actual maintenance period of an expired patent right, calculates an estimated value of the maintenance expenses until the surviving patent right expires as the remaining value of the patent right according to the estimated value of the remaining maintenance period calculated using the generated maintenance period calculation model, and evaluates only those patent rights whose calculated remaining value is equal to or less than a certain value or equal to or more than a certain value. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2013-101425 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in order to correctly understand the results predicted using a maintenance period calculation model or a trained model that predicts whether a patent right will be maintained or abandoned, the user is required to have a certain level of knowledge about the model. In addition, it can be difficult to determine how much the work time required to decide whether to maintain or abandon a patent right has actually been reduced.

[0005] Therefore, an object of the present invention is to provide a user interface and a prediction system that enables a user to easily decide whether to maintain or abandon a patent right based on the predicted results, even without knowledge of the model. [Means for solving the problem]

[0006] (1) The user interface of the present invention (e.g., user interface 200 described below) displays in a display format that allows the user to grasp at least the statistical status of the patent right to be abandoned, based on data for each specified data item indicated by the prediction result of whether to maintain or abandon the patent right, which is predicted using a pre-generated trained model that determines whether to maintain or abandon the patent right.

[0007] According to (1) above, even without knowledge of the model, it becomes possible to easily decide whether to maintain or abandon a patent right based on the predicted results.

[0008] (2) In the user interface described in (1) above, the specified data items may include at least the number of patent rights predicted to be maintained, the number of patent rights predicted to be abandoned, and a distribution of the confidence of the abandonment of patent rights predicted by the trained model.

[0009] According to (2) above, it will be possible to more appropriately determine whether to maintain or abandon a patent right.

[0010] (3) In the user interface described in (1) or (2) above, at least one question item for the user may be displayed, and the amount of work required to determine whether to maintain or abandon the patent right based on the data for each specified data item shown in the prediction result and the user's answer to the question may be displayed in a comparative manner to the amount of work required if the patent right were to be maintained or abandoned by a human being.

[0011] According to (3) above, it is possible to intuitively understand the effect of reducing the amount of work required to decide whether to maintain or abandon a patent right.

[0012] (4) In the user interface described in any of (1) to (3) above, an item for setting a threshold for the certainty of the abandonment of the patent right may be displayed, and data for each specified data item indicated by the prediction result may be displayed according to the set threshold.

[0013] According to (4) above, it will be possible to more appropriately determine whether to maintain or abandon a patent right.

[0014] (5) In the user interface described in (1) above, data for each specified data item indicating the predictive accuracy of the trained model may be displayed in a display format that allows the predictive accuracy of the trained model to be understood.

[0015] According to (5) above, it becomes possible to easily judge the predictive accuracy of a model even without knowledge of the model.

[0016] (6) In the user interface described in (5) above, the specified data items may include at least an overall accuracy rate, an accuracy rate of the prediction of the patent rights to be retained, an accuracy rate of the prediction of the patent rights to be abandoned, and a distribution of the degree of importance of explanatory variables in the trained model.

[0017] According to (6) above, it becomes possible to more appropriately judge the predictive accuracy of the model.

[0018] (7) In the user interface described in (1) above, based on the data for each specified data item shown in the prediction result, at least the incorrect answer rate for patent rights predicted to be abandoned, the number of patent rights that were overlooked to be abandoned, and the extra annual payment amount may be displayed for each different threshold value of the confidence level of the abandonment of the patent rights.

[0019] According to (7) above, depending on the confidence threshold, it is possible to estimate how many abandoned patent rights are likely to be overlooked and how much extra payments are likely to be incurred.

[0020] (8) A prediction system of the present invention (e.g., prediction system 50 described below) includes a prediction unit (e.g., prediction unit 34 described below) that predicts whether a patent right will be maintained or abandoned using a trained model that has been generated in advance to determine whether the patent right will be maintained or abandoned, and a display unit (e.g., display unit 12 described below) that displays a user interface (e.g., user interface 200 described below) in a display format that allows users to grasp at least the statistical state of the patent right to be abandoned, based on data for each specified data item indicated by the prediction result of the prediction unit.

[0021] According to (8) above, the same effect as (1) can be achieved. Effect of the Invention

[0022] According to the present invention, it becomes possible to easily determine whether to maintain or abandon a patent right based on the predicted results, even without knowledge of the model. [Brief description of the drawings]

[0023] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of a prediction system according to an embodiment. [Figure 2A] FIG. 2 is a diagram for explaining services to which the prediction system of FIG. 1 is applied. [Figure 2B] FIG. 2 is a diagram for explaining services to which the prediction system of FIG. 1 is applied. [Diagram 3] FIG. 2 is a functional block diagram showing an example of a functional configuration of a terminal device. [Figure 4] FIG. 2 is a functional block diagram illustrating an example of a functional configuration of a server. [Diagram 5] FIG. 11 is a diagram illustrating an example of teacher data received from a terminal device. [Figure 6] 4 is a diagram showing an example of a user interface of a web browser displayed on the display unit of FIG. 3. [Figure 7] 4 is a diagram showing an example of a user interface of a web browser displayed on the display unit of FIG. 3. [Figure 8]4 is a diagram showing an example of a user interface of a web browser displayed on the display unit of FIG. 3. [Figure 9] 11 is a flowchart illustrating a prediction process of the prediction system according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0024] Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings. Here, a case will be illustrated in which a user who is already a patent holder has been registered and a trained model for maintaining or abandoning a patent right has been generated. Note that the present invention is also applicable to a new user who is a patent holder. <One embodiment> FIG. 1 is a diagram illustrating an example of a configuration of a prediction system according to an embodiment. 1, a prediction system 50 is configured to include a terminal device 1 and a server 2. The terminal device 1 and the server 2 are connected to each other via a predetermined network 3 such as the Internet.

[0025] The terminal device 1 is, for example, a computer, a smartphone, a tablet terminal, etc., and connects to the network 3 via a base station (not shown) of a telecommunications carrier with which the user has a contract, and communicates with the server 2. Alternatively, the terminal device 1 may connect to the network 3 by wirelessly connecting to an access point (not shown) of a wireless LAN (for example, WiFi (registered trademark)), and communicate with the server 2. In FIG. 1, one terminal device 1 is connected to the network 3, but this is not limiting, and two or more terminal devices 1 may be connected to the network 3 and communicate with the server 2.

[0026] The server 2 is, for example, a cloud server, and in response to a request from the terminal device 1, provides, for example, updates (generation) of a trained model that predicts whether a patent right will be maintained or abandoned, verification results regarding the prediction accuracy of the updated (generated) trained model, and prediction results of the trained model regarding whether a patent right will be maintained or abandoned.

[0027] 2A and 2B are diagrams for explaining a service (hereinafter, also referred to as "this service") to which the prediction system 50 of FIG. 1 is applied. The terminal device 1 is managed by a user U who is the owner of the patent right. The server 2 is managed, for example, by a provider of this service.

[0028] In order to receive this service, an application program such as a web browser is pre-installed in the terminal device 1. If information about the user U (such as an ID or password) is not registered, the terminal device 1 executes the web browser based on an input operation by the user U, and registers the information about the user U in the server 2 from a new user registration screen based on a known method. The information about the user U may include the name, age, email address, etc.

[0029] As shown in Fig. 2A, in this service, for example, when information about a user U is registered, the terminal device 1 executes a web browser based on an input operation by the user U, and displays a login screen for authenticating the user U. The terminal device 1 inputs authentication information (e.g., an ID and a password) of the user U on the login screen based on an input operation by the user U, and transmits an authentication request including the input authentication information to the server 2 in order to obtain a verification result regarding the prediction accuracy of the trained model, a prediction result of the trained model regarding the maintenance or abandonment of a patent right, and the like.

[0030] The server 2 executes an authentication process to determine whether the authentication information of the user U received from the terminal device 1 matches the registration information of the registered user U. If the authentication information of the user U received from the terminal device 1 matches the registration information of the user U, the server 2 transmits a permission notification to the terminal device 1 to permit access to the server 2. Then, when the terminal device 1 receives a permission notification from the server 2, it displays a screen for obtaining updates (generation) of the trained model, verification results regarding the predictive accuracy of the trained model, and prediction results of the trained model regarding the maintenance or abandonment of patent rights, etc.

[0031] 2B, in this service, the terminal device 1 uploads to the server 2, based on an input operation by the user U, training data for updating (generating) the trained model, verification data for verifying the prediction accuracy of the updated (generated) trained model, and data on the patent right to be predicted. In addition, the terminal device 1 may receive settings of indexes and objective variables, answers to questions displayed on a web browser screen (user interface 200 described later), and the like from the user U, and transmit the received settings of indexes and objective variables, answers to questions, and the like to the server 2. The server 2, for example, executes supervised learning based on the received teacher data, and updates (generates) the trained model. The server 2 then executes a verification process to verify the prediction accuracy of the updated (generated) trained model by inputting the received verification data into the updated (generated) trained model. The server 2 also executes a prediction process to input the received data of the patent right to be predicted into the trained model, and predicts whether the patent right to be predicted will be maintained or abandoned according to the received indexes and settings of the objective variables, answers to questions, and the like. The server 2 transmits the verification results of the verification process and the prediction results of the prediction process to the terminal device 1. The terminal device 1 displays the verification results and prediction results received from the server 2 on a user interface screen, which will be described later, in a display format that allows the user to grasp at least the statistical status of the patent right to be abandoned, based on the data for each specified data item indicated by the verification results and prediction results.

[0032] In this way, with this service, User U can receive the predicted results of whether to maintain or abandon the patent right along with the verification results regarding the predictive accuracy of the trained model. This makes it possible for User U to easily decide whether to maintain or abandon the patent right based on the predicted results, even if he or she has no knowledge of the trained model. In addition, in this service, the prediction results may include information indicating how much work effort required to determine whether to maintain or abandon a patent right can be reduced. By doing so, the user U can intuitively grasp how much work man-hours can be reduced.

[0033] <Configuration of Terminal Device 1> FIG. 3 is a functional block diagram showing an example of the functional configuration of the terminal device 1. As shown in FIG. As shown in FIG. 3, the terminal device 1 is a computer or the like, and includes a control unit 10, an input unit 11, a display unit 12, a storage unit 13, and a communication unit .

[0034] The control unit 10 is a processor such as a CPU (Central Processing Unit), and is a part that controls the entire terminal device 1. The control unit 10 appropriately reads and executes various programs stored in the storage unit 13, thereby realizing various functions in this embodiment. Specifically, for example, when the control unit 10 executes a web browser based on an input operation of the user U, the control unit 10 displays a login screen for authenticating the user U on the display unit 12 described later, and transmits an authentication request including the authentication information (e.g., ID, password, etc.) of the user U to the server 2 via the communication unit 14 described later. When the control unit 10 receives a notification of authorization of authentication from the server 2, the control unit 10 displays a screen for uploading teacher data, etc. for updating (generating) the trained model on the display unit 12 described later, and uploads data such as teacher data to the server 2 via the communication unit 14 described later. Then, the control unit 10 displays screens of user interfaces 100 to 300 described later on the display unit 12 described later, which display the verification results and prediction results received from the server 2 in a display format that allows the user to grasp at least the statistical state of the patent right to be abandoned based on the data for each predetermined data item indicated by the verification results and prediction results.

[0035] The input unit 11 is realized by, for example, a keyboard, a mouse, a touch panel, etc., and receives various operations from the user U.

[0036] The display unit 12 is realized by, for example, a liquid crystal display, and displays various information.

[0037] The storage unit 13 is configured with, for example, a Read Only Memory (ROM), a Random Access Memory (RAM), or a Hard Disk Drive (HDD), and stores various data such as installed applications.

[0038] As described above, the communication unit 14 connects to the network 3 via a base station (not shown) of a telecommunications carrier with which the user U has a contract, and controls communication with the information providing server 2. Alternatively, the communication unit 14 may connect to the network 3 by wirelessly connecting to an access point of a wireless LAN (e.g., WiFi), and control communication with the information providing server 2.

[0039] <Server 2 configuration> FIG. 4 is a functional block diagram showing an example of the functional configuration of the server 2. As shown in FIG. 4, the server 2 is a cloud server or the like, and includes a communication unit 20, a control unit 21, and a storage unit 22.

[0040] The communication unit 20 is connected to the network 3 via a wired or wireless connection, and controls communication between the terminal device 1 and the network 3 .

[0041] The storage unit 22 is composed of a ROM, a RAM, a HDD, or the like, and stores user data 41, trained model data 42, and a teacher database 43. The user data 41 stores registration information (eg, ID, password, etc.) for each user U of the terminal device 1 that has been registered in advance. The trained model data 42 stores, for each user U, a trained model updated (generated) based on data received from the terminal device 1 by the learning unit 32 described later. The teacher database 43 stores the teacher data received from the terminal device 1 for each user U. FIG. 5 is a diagram showing an example of the teacher data received from the terminal device 1. As shown in FIG. As shown in FIG. 5, the training data received from the terminal device 1 includes attributes such as "application number", "application date", "publication date", "registration date", and "maintenance / abandonment". The "application number" in the training data received from the terminal device 1 stores the application number assigned to the patent right. The "application date" in the training data received from the terminal device 1 stores the date on which the patent application was filed. The "Publication Date" in the teacher data received from the terminal device 1 stores the date on which the patent application was published. The "registration date" in the training data received from the terminal device 1 stores the date on which the patent right was registered. "Maintained / Abandoned" in the training data received from the terminal device 1 stores information indicating whether the patent right is maintained or abandoned. In the training data in Fig. 5, "0" is stored if the patent right is maintained, and "1" is stored if the patent right is abandoned. In addition, in the training data of Figure 5, in addition to the "application number," "application date," "publication date," "registration date," and "maintenance / abandonment," the attribute information may also include the name of the invention, a rank indicating the importance of the patent right, keywords for the patent right, remaining term of validity, maintenance costs such as patent fees, etc.

[0042] The control unit 21 is, for example, a processor such as a CPU, and is a part that controls the entire server 2. The control unit 21 realizes various functions in this embodiment by appropriately reading and executing various programs stored in the storage unit 22. For example, the control unit 21 has the functions of an authentication unit 31, a learning unit 32, a verification unit 33, and a prediction unit 34.

[0043] The authentication unit 31 authenticates the terminal device 1 so that the terminal device 1 can receive this service. Specifically, the authentication unit 31 receives an authentication request including authentication information (e.g., ID, password, etc.) of the user U via the communication unit 20 so that the terminal device 1 can receive this service. Then, the authentication unit 31 determines whether the authentication information of the user U included in the authentication request received from the terminal device 1 matches the registration information of the user U registered in the user data 41 in the storage unit 22. If the authentication information of the user U received from the terminal device 1 matches the registration information of the user U, the authentication unit 31 transmits a permission notification to the terminal device 1 via the communication unit 20, which permits the terminal device 1 to access the information providing server 2.

[0044] The learning unit 32 performs supervised learning based on the teacher data received from the terminal device 1 via the communication unit 20, thereby updating (generating) the trained model. Specifically, the learning unit 32 performs supervised learning using, for example, the "application number," "application date," "publication date," "registration date," etc., from the teacher data shown in Figure 5, as input data with explanatory variables, and label data (correct answer data) with "maintain / abandon" as the objective variable, thereby updating (generating) a trained model that predicts the degree of abandonment of the patent right to be predicted with a value from "0" to "1" based on the data of the patent right to be predicted received as input data. The learning unit 32 stores the updated trained model in trained model data 42 in the memory unit 22. The trained model may be a classifier (e.g., a support vector machine) that classifies input data whose classification labels (label data) are known, or a neural network that calculates a predicted value of output data from input data. Furthermore, the learning unit 32 may update (generate) two or more different trained models. In this case, the verification unit 33 described later may input the received verification data to each of the trained models and execute a verification process of the prediction accuracy of each of the trained models, and the prediction unit 34 described later may select the trained model with the highest prediction accuracy based on the verification result.

[0045] The verification unit 33 inputs the received verification data as input data into the updated (generated) trained model, and executes a verification process to verify the prediction accuracy of the patent right to be abandoned output by the trained model. The format of the verification data is the same as that of the training data shown in FIG.

[0046] The prediction unit 34 inputs, for example, data on the patent right to be predicted received from the terminal device 1 as input data into the updated (generated) trained model, and predicts the degree of abandonment of the patent right as a value between "0" and "1." Note that the format of the data on the patent right to be predicted is the same as the training data shown in Fig. 5, and all values ​​of the "maintain / abandon" attribute are "0" (maintain).

[0047] <Web browser user interface> Figures 6 to 8 are diagrams showing an example of a user interface of a web browser displayed on the display unit 12 in Figure 3. Note that the user interfaces shown in Figures 6 to 8 can switch screens in response to an input operation by the user U via the input unit 11. As shown in FIG. 6, the user interface 100 displays, for example, a verification result regarding the prediction accuracy of the updated (generated) trained model. Specifically, the user interface 100 has, for example, an area 110 displaying the “number of learning data items,” “number of validation data items,” and “number of learning + validation data items in category” as predetermined data items, an area 120 displaying the “overall accuracy rate,” “accuracy rate of maintenance prediction,” “accuracy rate of abandonment prediction,” “incorrect rate of abandonment prediction,” and “confidence level” as predetermined data items, and an area 130 displaying the “explanatory variable (feature) importance ranking” as a predetermined data item.

[0048] "Number of training data items" indicates the number of patent rights contained in the training data, "Number of validation data items" indicates the number of patent rights contained in the validation data, and "Number in training + validation data category" indicates the number of patent rights contained in the training data and the number of patent rights contained in the validation data. The "overall accuracy rate" displays the percentage of agreement between the "maintain / abandon" of each patent right in the verification data and the verification result in which each patent right is determined to be maintained or abandoned by inputting the verification data as input data into the trained model by the verification unit 33. Note that in the verification result, for example, if the value of the degree of abandonment of the patent right output by the trained model is less than the certainty of abandonment (e.g., "0.5"), it is determined to be "maintained," and if it is equal to or greater than the certainty of abandonment (e.g., "0.5"), it is determined to be "abandoned." The "accuracy rate of maintenance prediction" shows the percentage (accuracy rate) of patents judged to be "maintained" based on the degree of patent abandonment output by the trained model for patents that are maintained in the validation data. The "accuracy rate of abandonment prediction" shows the percentage (accuracy rate) of abandoned patents in the validation data that were judged to be "abandoned" based on the degree of abandonment of the patent rights output by the trained model. The "Abandonment Prediction Error Rate" shows the percentage of abandoned patents in the validation data that were judged to be "maintained" based on the degree of patent abandonment output by the trained model. "Confidence" shows the distribution of how certain the trained model's predictions are. The "Explanatory variable (feature) importance ranking" displays the importance of each explanatory variable in the trained model.

[0049] Next, as shown in FIG. 7, the user interface 200 displays, for example, a prediction result based on the updated (generated) trained model. Specifically, the user interface 200 has an area 210 that displays, for example, predetermined data items such as "predicted number of data items," "number predicted to be kept," "number predicted to be abandoned," "certainty (of abandonment)," "predicted ratio of keeping and abandonment," and "certainty threshold," and an area 220 that displays, for example, "question (1)," "question (2)," "approximate time required to create the AI ​​learning model," and "reduction in labor hours" as predetermined data items. "Number of predicted data items" displays the number of patent rights included in the data of the patent right that is the target of prediction. "Number predicted to be maintained" displays the number of patent rights predicted by the prediction unit 34 to be maintained. "Number predicted to be abandoned" displays the number of patent rights predicted by the prediction unit 34 to be abandoned. "Certainty (of abandonment)" displays the distribution of the confidence of the abandonment of patent rights predicted by the prediction unit 34. "Predicted ratio of maintenance and abandonment" displays the ratio between the number of patent rights predicted by the prediction unit 34 to be maintained and the number of patent rights predicted by the prediction unit 34 to be abandoned.

[0050] The "certainty threshold" accepts the setting of a certainty threshold for determining abandonment based on an input operation by the user U via the input unit 11. Specifically, since the certainty varies greatly from 0.43 to 0.63 with reference to the distribution of "certainty (of abandonment)" in FIG. 7, for example, the "certainty threshold" may accept a setting such as "0.5" by direct input by the user U, or may accept a setting by the user U operating the slide 211. In this way, user U can easily set a confidence threshold for deciding whether to maintain or abandon a patent right based on the predicted results, even without any knowledge of the trained model.

[0051] In addition, the user interface 200 may display in area 210 the technical field and content of the patent, a recommended threshold value according to the distribution shape of the “certainty (of abandonment)”, or a predicted result of the rank of each patent right. By doing so, user U can more appropriately decide whether to maintain or abandon the patent right.

[0052] "Question (1)", for example, accepts the work time required for the user U to decide whether to maintain or abandon each patent based on an input operation by the user U via the input unit 11. Alternatively, the work time may be accepted by the user U operating the slide 221. Note that "Question (1)" may be configured to accept the work time required for the user U to decide whether to maintain or abandon each patent according to a rank indicating the importance of the patent right. The "question (2)" receives the preparation time required to prepare the analysis data based on an input operation by the user U via the input unit 11. The preparation time may be received by the user U operating the slide 222. "Approximate time required to create AI learning model" displays the time required for the learning unit 32 to update (generate) the learned model. "Reduction in work effort" displays, in a comparative manner, the work hours required for a user such as user U to reconfirm only the patent rights that have been determined to be abandoned in the prediction results, based on the "predicted number of data items," the "number of items predicted to be abandoned," and the answer to "Question (1)," and the work hours required for a user such as user U to reconfirm all of the patent rights in the "predicted number of data items." In this way, even if user U has no knowledge of the trained model, he or she can intuitively understand how much time it is possible to reduce the work required to decide whether to maintain or abandon a patent right based on the prediction results of the trained model.

[0053] Finally, as shown in FIG. 8, the user interface 300 displays the balance between annuity payments and labor reduction for different thresholds of confidence in the abandonment of patent rights, for example, based on the prediction results of an updated (generated) trained model. Specifically, the user interface 300 has, for example, an area 310 for setting the average annual amount per patent as a predetermined data item, an area 320 for displaying the “incorrect rate of abandonment prediction,” “number of abandoned cases overlooked,” and “extra annual amount to be paid” as predetermined data items when the threshold for the certainty of patent right abandonment is 0.3, an area 330 for displaying the “incorrect rate of abandonment prediction,” “number of abandoned cases overlooked,” and “extra annual amount to be paid” as predetermined data items when the threshold for the certainty of patent right abandonment is 0.5, and an area 340 for displaying the “incorrect rate of abandonment prediction,” “number of abandoned cases overlooked,” and “extra annual amount to be paid” as predetermined data items when the threshold for the certainty of patent right abandonment is 0.7. In the area 310, for example, the average annual amount per patent by the user U is set based on an input operation by the user U via the input unit 11. The average annual amount may be set by the user U operating the slide 311.

[0054] The "incorrect rate of abandonment prediction" in areas 320 to 340 displays the percentage (incorrect rate) of the predicted patent rights that the prediction unit 34 predicted to be "maintained" when the threshold of the certainty of abandonment is 0.3, 0.5, and 0.7, respectively. The "number of abandoned cases missed" in areas 320 to 340 displays the number of patent rights that the prediction unit 34 predicted to be "maintained" when the threshold of the certainty of abandonment is 0.3, 0.5, and 0.7, respectively. The "extra annual amount to be paid" in areas 320 to 340 displays the annual amount calculated based on the average annual amount set in area 310 and the "number of abandoned cases missed" when the threshold of the certainty of abandonment is 0.3, 0.5, and 0.7, respectively. In this way, user U can estimate how many abandoned patent rights are likely to be overlooked and how much extra payment is likely to be incurred depending on the confidence threshold.

[0055] <Prediction processing of prediction system 50> Next, the operation of the prediction process of the prediction system 50 according to this embodiment will be described. 9 is a flowchart illustrating a prediction process of the prediction system 50 according to an embodiment. The flow shown here is executed every time the terminal device 1 logs in to the server 2.

[0056] In step S101, the control unit 10 of the terminal device 1 executes the web browser and displays a login screen when the user U performs an operation such as clicking on a web browser icon displayed on the display unit 12 via the input unit 11. The control unit 10 transmits an authentication request including the authentication information (e.g., ID, password, etc.) of the user U inputted on the displayed login screen to the server 2 via the communication unit 14.

[0057] In step S201, the authentication unit 31 of the control unit 21 of the server 2 receives, via the communication unit 20, the authentication request for the user U transmitted by the terminal device 1 in step S101.

[0058] In step S202, the authentication unit 31 executes an authentication process to determine whether or not the authentication information of the user U included in the received authentication request matches the registration information of the user U registered in the user data 41. If the authentication information of the user U matches the registration information of the user U (YES), the process proceeds to step S204. On the other hand, if the authentication information of the user U does not match the registration information of the user U (NO), the process proceeds to step S203.

[0059] In step S203, the authentication unit 31 transmits an authentication failure to the terminal device 1 via the communication unit 20, indicating that access to the server 2 is not permitted. Then, the process returns to step S201. In this case, the authentication unit 31 waits until it receives an authentication request for the user U from the terminal device 1, for example.

[0060] In step S204, the authentication unit 31 transmits an authentication permission for permitting access to the server 2 to the terminal device 1 via the communication unit 20.

[0061] In step S102, the control unit 10 of the terminal device 1 determines whether or not authentication permission has been received from the server 2 via the communication unit 14. If authentication permission has been received (YES), the process proceeds to step S103. On the other hand, if authentication permission has not been received (NO), that is, if authentication failure has been received, the process returns to step S101. If the process returns to step S101, the control unit 10 retransmits an authentication request including, for example, authentication information of the user U (for example, an ID, a password, etc.) to the information providing server 2 via the communication unit 14.

[0062] In step S103, based on screen operations by user U, the control unit 10 uploads to the server 2 training data for updating (generating) the trained model, verification data for verifying the updated (generated) trained model, and data on the patent right to be predicted.

[0063] In step S205, the learning unit 32 of the server 2 executes supervised learning based on the teacher data uploaded in step S103, and updates (generates) the trained model.

[0064] In step S206, the verification unit 33 executes a verification process for the trained model updated (generated) in step S205, using the verification data uploaded in step S103.

[0065] In step S207, the prediction unit 34 inputs the data of the patent right to be predicted uploaded in step S103 into the trained model updated (generated) in step S205, and performs a prediction process of whether each of the patent rights to be predicted will be maintained or abandoned.

[0066] In step S208, the control unit 21 of the server 2 transmits to the terminal device 1 the verification result of the verification process executed in step S206 and the prediction result of the prediction process executed in step S206.

[0067] In step S104, the control unit 10 of the terminal device 1 determines whether or not the verification result and the prediction result have been received from the server 2 via the communication unit 14. If the verification result and the prediction result have been received (YES), the process proceeds to step S105. On the other hand, if the verification result and the prediction result have not been received (NO), the process waits in step S104 until the verification result and the prediction result are received.

[0068] In step S105, the control unit 10 displays the verification result and prediction result received in step S104 on the display unit 12 on any one of the screens of the user interfaces 100 to 300 shown in Fig. 6 to Fig. 8. The control unit 10 may display the verification result and prediction result by switching the screens of the user interfaces 100 to 300 in response to an input operation of the user U via the input unit 11. The control unit 10 may also change the display of the prediction result based on a threshold setting for the certainty of abandonment that is set in response to an input operation of the user U, or based on the answers to questions (1) and (2). The prediction system 50 described above makes it possible to easily determine whether to maintain or abandon a patent right based on the predicted results, even without knowledge of the trained model.

[0069] Although one embodiment has been described above, the prediction system 50 is not limited to the above-described embodiment, and includes modifications and improvements within the scope of achieving the object.

[0070] <Variation 1> In the embodiment described above, the user interface 200 receives answers to two questions (1) and (2) from the user U, but is not limited to this. For example, the user interface 200 may display one or three or more questions and receive answers to each question from the user U.

[0071] <Variation 2> Also, for example, in the above-described embodiment, the server 2 includes the authentication unit 31, the learning unit 32, the verification unit 33, and the prediction unit 34, but is not limited to this. For example, each function of the server 2 may be realized by using a virtual server function or the like on a cloud. Furthermore, the server 2 may be a distributed processing system in which the functions of the server 2 are appropriately distributed among a plurality of servers.

[0072] <Modification 3> Also, for example, in the above-mentioned embodiment, the user interface 200 displays the "predicted number of data items," "number of items predicted to be maintained," "number of items predicted to be abandoned," "certainty (of abandonment)," "predicted ratio of maintaining and abandoning," and "certainty threshold" in the area 210, but is not limited to this. For example, the user interface 200 may display the technical field or content of the patent, a recommended value of the threshold according to the distribution shape of the "certainty (of abandonment)," or a predicted result of the rank of each patent right, etc.

[0073] <Modification 4> Also, for example, in the above-described embodiment, the user interface 200 accepts the work time required to determine whether to maintain or abandon each patent in the "Question (1)" of the area 220, but is not limited to this. For example, the user interface 200 may accept, in the "Question (1)", the work time required to determine whether to maintain or abandon each patent according to the rank indicating the importance of the patent right.

[0074] Each function included in the prediction system 50 according to the embodiment described above can be realized by hardware, software, or a combination of these. Here, being realized by software means being realized by a computer reading and executing a program.

[0075] The program can be stored and provided to the computer using various types of non-transitory computer readable media. The non-transitory computer readable media includes various types of tangible storage media. Examples of the non-transitory computer readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs). The program may also be provided to the computer by various types of transitory computer readable media. Examples of the transitory computer readable media include electrical signals, optical signals, and electromagnetic waves. The transitory computer readable media can provide the program to the computer via wired communication paths such as electric wires and optical fibers, or wireless communication paths.

[0076] In addition, the steps of writing a program to be recorded on a recording medium include not only processes that are performed chronologically according to the order, but also processes that are not necessarily performed chronologically but are executed in parallel or individually. [Explanation of symbols]

[0077] 1 Terminal equipment 2 Server 3. Network 10 Control section 11 Input section 12 Display section 13 Storage section 14 Communications Department 20 Communications Department 21 Control section 22 Memory section 31 Authentication Section 32 Learning Department 33 Verification Department 34 Prediction Department 41 User Data 42 Trained model data 43 Teacher Database 50 Prediction System 100, 200, 300 User Interface

Claims

1. A display unit is provided that displays data for each predetermined data item, including at least the number of patent rights predicted to be maintained, the number of patent rights predicted to be abandoned, and a distribution of the confidence of the abandonment of the patent rights predicted by the trained model in the prediction results of the maintenance or abandonment of the patent rights predicted using a trained model that determines whether to maintain or abandon the patent rights that has been generated in advance, The display unit is Displaying an item for setting a threshold for the certainty of abandoning the patent right; A terminal device that displays data for each of the predetermined data items according to the set threshold value.

2. The display unit is displaying at least one item of question to a user; The terminal device according to claim 1, which displays in a comparative manner the amount of work required to determine whether to maintain or abandon a patent right based on the data for each of the specified data items and the user's answers to the questions, and the amount of work that would be required if the decision to maintain or abandon the patent right were made by all persons.

3. A display unit is provided that displays data for each predetermined data item including at least an overall accuracy rate, an accuracy rate for predicting whether a patent right will be maintained or abandoned, and a distribution of the importance of explanatory variables in the trained model, the data indicating the predictive accuracy of the trained model using a trained model that determines whether a patent right will be maintained or abandoned, and the overall accuracy rate, the accuracy rate for predicting whether a patent right will be abandoned, and the distribution of the importance of explanatory variables in the trained model; The display unit is Displaying a screen for uploading teacher data for generating or updating the trained model; A terminal device that displays, as a distribution of the degree of importance of the explanatory variables, the importance of each explanatory variable in the teacher data uploaded by a user when generating or updating the trained model in a ranked format for comparison.

4. The terminal device according to claim 1, wherein the display unit displays at least the incorrect answer rate of patent rights predicted to be abandoned, the number of patent rights that were overlooked to be abandoned, and the extra pension amount to be paid, for each different threshold value of the certainty of the abandonment of the patent rights, based on data for each specified data item indicated by the prediction result.

5. A prediction unit that predicts whether the patent right will be maintained or abandoned using a trained model that has been generated in advance to determine whether the patent right will be maintained or abandoned; A display unit that displays data for each predetermined data item including at least the number of patent rights predicted to be maintained, the number of patent rights predicted to be abandoned, and a distribution of the confidence of the abandoned patent rights predicted by the trained model in the prediction results of the prediction unit; A prediction system comprising:

Images