Patent value evaluation method based on Markov decision

Abstract

The invention relates to the technical field of patent value evaluation. The invention provides a Markov decision-based patent value evaluation method, which is used for constructing a state-action-reward-transfer decision-making model for patent life cycle dynamic characteristics. Through multi-source data processing, state construction, strategy screening, value function iteration and optimal strategy generation, quantitative evaluation of patent dynamic values and operation suggestion output are realized, and the problem that strategy evolution and dynamic earnings cannot be described by an existing method is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of patent value evaluation, in particular to a patent value evaluation method based on Markov decision. BACKGROUND

[0002] In the field of patent operation and management, quantifying the value of patent assets has become an important basis for decision-making in mergers and acquisitions, investment and financing, and litigation. Current methods are mostly based on expert scoring, analytic hierarchy process, or static index weighting models, focusing on linear combination of technical, legal, and market information at a single time slice, thus giving a one-time value score.

[0003] With the rapid growth of patent data and the successful application of artificial intelligence in financial risk control, academia and industry have begun to explore the use of reinforcement learning, graph neural networks, and other means to view the life cycle evolution of patents as a time-series decision problem, dynamically predicting potential earnings, licensing opportunities, and risk exposure. Among them, Markov Decision Process (MDP) is of interest because it can describe the "state-action-reward-transition" closed loop, providing a unified framework for continuous value assessment and strategy optimization.

[0004] However, existing MDP-based attempts are still at the theoretical verification level, lacking systematic solutions to the unique data heterogeneity, high-dimensional sparsity of states, and non-stationarity of legal events specific to patent assets. Existing evaluation results are difficult to balance immediate returns and long-term strategic returns, and cannot support multi-strategy collaborative decision-making such as "maintaining, licensing, pledging, and abandoning." Therefore, there is a need for an evaluation method that can adaptively construct a state space from multi-source data within a complete MDP framework and obtain the full-cycle expected value through the Bellman optimality principle. SUMMARY

[0005] To overcome the shortcomings of the prior art, the present application aims to provide a patent value evaluation method based on Markov decision, which solves the problem that existing patent evaluation methods cannot depict the strategy evolution process and cannot quantify dynamic value changes.

[0006] To achieve the above-mentioned purpose, the present application provides the following solutions:

[0007] A patent value evaluation method based on Markov decision, comprising:

[0008] synchronously acquiring original patent data of a target patent, and performing missing value filling, feature normalization, and time series alignment on the original patent data to obtain multi-dimensional features;

[0009] mapping the multi-dimensional features to a state S of a patent life cycle; the state S describes at least technical maturity, market heat, legal robustness, and competition density;

[0010] setting a set of actions A for each of the states S; the set of actions A includes multiple selectable strategies;

[0011] setting an immediate reward R(S, a) according to an expected cash flow, litigation compensation, licensing rate, holding cost, and risk exposure generated by performing any action in the set of actions A under each of the states S; wherein a ∈ A, and setting a discount factor γ ∈ (0, 1);

[0012] using multi-task Bayesian learning and time-dependent Wiener processes to fit historical events and estimate a probability P(S' | S, a) of transitioning to a state S' after performing an action a in the set of actions A under a state S;

[0013] calculating an optimal value V*(S) and an optimal strategy π*(S) corresponding to each state S based on a Bellman optimality equation through a policy iteration algorithm;

[0014] taking the optimal expected value V*(S0) corresponding to a current state S0 of the patent as a patent value evaluation result, and giving an optimal operation strategy suggestion corresponding to the action π*(S0).

[0015] Preferably, the selectable strategies include: maintaining, licensing, pledge financing, cooperative development, filing a lawsuit, defense against a lawsuit, and abandonment.

[0016] Preferably, the original patent data includes: literature full text, legal status, international patent classification number, citation-referencing relationship, licensing, pledge, litigation, transfer and marketing event record, patentee and related subject.

[0017] Preferably, the original patent data of the target patent is synchronously acquired, and the original patent data is subjected to missing value filling, feature normalization and time series alignment to obtain multi-dimensional features, including:

[0018] the literature full text, legal status, international patent classification number and citation-referencing relationship core fields of the target patent are acquired through an official patent public interface;

[0019] the licensing, pledge, litigation, transfer and marketing event record directly associated with the target patent are acquired through a commercial intelligence database;

[0020] the basic information of the patentee and related subject of the target patent is acquired through an enterprise information platform;

[0021] The core field, license, pledge, litigation, transfer and marketing event record and the basic information are structured and parsed according to a unified field mapping table, and are combined to generate original patent correlation data with a unified timestamp;

[0022] Missing values in the continuous type field in the original patent correlation data are filled in a time series interpolation manner, and missing values in the category type field in the original patent correlation data are filled in a nearest neighbor estimation manner;

[0023] The filled continuous type field in the original patent correlation data is scaled by minimum-maximum, and the filled category type field in the original patent correlation data is processed by one-hot encoding to obtain preprocessed data;

[0024] According to the unified timestamp, the preprocessed data is aligned according to a preset observation window to generate the multi-dimensional features with equal time intervals;

[0025] Fields with inconsistent granularity in the preset observation window are uniformly encoded by a sliding aggregation strategy to obtain the multi-dimensional features satisfying the consistency of time effectiveness.

[0026] Preferably, the multi-dimensional features are mapped to a state S of a patent life cycle, including:

[0027] According to the patent life cycle management requirements, the state S includes at least the following four dimensions: technical maturity T t , market heat M t , legal stability L t and competition density C t ; wherein, the technical maturity T t reflects the research and development depth and industrial application degree of the patent technology; the market heat M t reflects the market attention and economic activity intensity of the patent related products or services; the legal stability L t reflects the stability, legal survival and historical objection risk of the patent right; and the competition density C t reflects the aggregation degree and technical barrier strength of similar or alternative patents in the technical field.

[0028] A feature vector x t is constructed according to the multi-dimensional features, so that the feature vector x t is used as an input at the current time t.

[0029] A supervised learning model is used to train four mapping functions f1, f2, f3 and f4 using historical label data, and x t is mapped to corresponding index values T t , M t , L t and Ct ; the mapping function is automatically trained according to historical event labels; the historical event labels include achievement transformation records, transaction behaviors, litigation situations, and citation dynamics;

[0030] The four indicator values obtained by mapping are assembled into the state S; the expression of the state S is: S=[T t , M t , L t , C t ];

[0031] The dimension values in the state S are normalized according to predefined intervals, so that the dimension values fall within the closed interval [0, 1].

[0032] Preferably, for each state S, a set of actions A is set, including:

[0033] For each of the selectable strategies, a applicable condition rule is preset; the applicable condition rule includes a numerical threshold or interval limit for each dimension indicator in the state S;

[0034] The indicator values of the current state S are read and compared with the applicable condition rules corresponding to each strategy;

[0035] The strategy that meets the corresponding applicable condition is included in the action set A corresponding to the current state S.

[0036] Preferably, according to the expected cash flow, litigation compensation, license fee rate, holding cost, and risk exposure generated by executing any action in the action set A under each state S, an immediate reward R(S, a) is set, including:

[0037] An reward function R(S, a) is set for quantifying the immediate economic return and risk cost brought by executing action a in state S; the function form is: R(S, a)=E[CF(S, a)]+E[LC(S, a)]+E[PC(S, a)]-C(S, a)-RSK(S, a); wherein, E[CF(S, a)] represents the expected cash flow income, E[LC(S, a)] represents the expected license income brought by the license behavior, E[PC(S, a)] represents the compensation income that may be brought by litigation or claim for compensation, C(S, a) represents the direct or indirect cost required for executing action a in state S, and RSK(S, a) represents the risk exposure index accompanied by the action;

[0038] A data sample set containing a plurality of historical "state-action-result" triplets is obtained; the data sample set includes technical indicators, market feedback, legal changes, and economic consequences of operation strategy behaviors of historical patents;

[0039] Training a prediction model for each index in the R(S, a), including:

[0040] Using a preset supervised learning method to fit the prediction function of E[CF(S, a)], E[LC(S, a)], E[PC(S, a)]; the supervised learning method is any one of linear regression, decision tree or neural network;

[0041] Using a cost aggregation model or expert estimation method to establish a cost evaluation function of C(S, a);

[0042] Based on the Bayesian probability graph or risk event frequency modeling method to construct the risk expectation model of RSK(S, a);

[0043] Taking each dimension index of the current state S and the action a as input variables, substituting them into each of the prediction models, outputting the expected value of the corresponding index and substituting it into the reward function R(S, a);

[0044] For each action a∈A in the state S and the corresponding action set A, the calculation process of the prediction model is performed to obtain the value of R(S, a).

[0045] Preferably, the calculation formula of the probability P(S'|S, a) is:

[0046]

[0047] Where P(S'|S, a) is the probability of transitioning to the next state S' after performing action a in the current state S; γ is the discount factor, used to discount the next state value; V(S') is the expected value of state S' predicted by the value function network; η is the distance penalty coefficient, used to suppress unreachable or too high cost transitions; D(E(S'), E(S)+E a ) is the Euclidean distance between the state embedding vector E(S') and the target vector obtained by superimposing the current state vector E(S) and the action vector E a ; exp is the exponential function; ∑ S″∈Ω represents the normalized summation of all possible next states in the state space Ω, to ensure that the probability sum is 1.

[0048] Preferably, based on the Bellman optimality equation, the optimal value V*(S) and the optimal policy π*(S) corresponding to each state S are calculated by a policy iteration algorithm, including:

[0049] a) Initialize the policy π0(S), for each state S in the state space Ω, randomly select an executable action a∈A(S) as the initial action of the current policy π0(S);

[0050] b) For the current policy πk (S) computing state value function V k (S); the V k (S) satisfies the Bellman optimality equation: V k (S) = R(S, π k (S)) + γ∑ S′ P(S' | S, π k (S)) · V k (S');

[0051] c) solving the Bellman optimality equation by using an iterative method or a matrix solution method until a preset convergence condition ε is met, to obtain a stable state value function V k (S);

[0052] d) for each state S, updating the policy π k+1 (S) to a new action that can maximize the state value:

[0053]

[0054] e) if π k+1 (S) ≠ π k (S), setting k <- k + 1, and repeating steps b) to d); otherwise, terminating the iteration, and outputting the converged state optimal value function V * (S) and the corresponding policy π * (S); wherein k represents an index of a round of policy iteration.

[0055] Preferably, the optimal expected value V*(S0) corresponding to the current state S0 of the patent is taken as a patent value evaluation result, and an optimal operation policy suggestion corresponding to the action π*(S0) is given, including:

[0056] Based on the current feature data of the patent, the position of the feature data in the state space is determined, to obtain the current state S0.

[0057] The optimal value function V*(S) calculated in advance is called to obtain the optimal expected value V*(S0) corresponding to the current state S0.

[0058] The optimal policy function π*(S) generated in advance is called to obtain the optimal action π*(S0) corresponding to the current state S0.

[0059] V*(S0) is taken as the value evaluation result of the patent, and the policy content corresponding to the action π*(S0) is converted into a literal operation suggestion and displayed or recorded.

[0060] According to the specific embodiments provided by the present application, the following technical effects are disclosed:

[0061] The patent value evaluation method based on Markov decision provided by the application can effectively realize dynamic value prediction and strategy recommendation of the patent at different life cycle stages through multi-dimensional feature state modeling, action strategy space setting, reward function quantification and Bellman optimality solving. The method fuses patent data, market performance and legal events, realizes whole-process modeling from raw data to operation decision, has more explainability and foresight compared with traditional static evaluation methods, can adapt to value evolution differences under different strategy paths, and improves practicality and operation guidance of the evaluation results. BRIEF DESCRIPTION OF DRAWINGS

[0062] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description only represent some of the embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0063] Figure 1 The method flowchart provided for the embodiments of the present application is as follows:

[0064] Figure 2 The action set setting flowchart provided for the embodiments of the present application is as follows. DETAILED DESCRIPTION

[0065] The technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments only represent some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the present application.

[0066] The purpose of the present application is to provide a patent value evaluation method based on Markov decision, which solves the problems that the existing patent evaluation methods are difficult to depict the strategy evolution process and cannot quantify the dynamic value change.

[0067] In order to make the above-mentioned purposes, features and advantages of the present application more apparent and easy to understand, the present application will be further described in detail with reference to the drawings and specific embodiments.

[0068] Figure 1 The method flowchart provided for the embodiments of the present application is as follows: Figure 1 As shown in the method flowchart provided for the embodiments of the present application, the present application provides a patent value evaluation method based on Markov decision, which comprises:

[0069] Step 100: synchronously acquiring original patent data of a target patent, and performing missing value filling, feature normalization and time sequence alignment on the original patent data to obtain multi-dimensional features;

[0070] Step 200: mapping the multi-dimensional features into a state S of the patent life cycle; the state S describes at least technical maturity, market heat, legal robustness, and competition density;

[0071] Step 300: setting a set of actions A for each state S; the set of actions A includes multiple optional strategies;

[0072] Step 400: setting an immediate reward R(S, a) according to the expected cash flow, litigation compensation, licensing rate, holding cost, and risk exposure generated by executing any action in the set of actions A under each state S; where a∈A, and setting a discount factor γ∈(0, 1);

[0073] Step 500: fitting historical events using multi-task Bayesian learning and time-dependent Wiener process to estimate the probability P(S′|S, a) of transitioning to a state S′ after executing an action a in the set of actions A under a state S;

[0074] Step 600: calculating the optimal value V*(S) and optimal strategy π*(S) corresponding to each state S based on the Bellman optimality equation through a policy iteration algorithm;

[0075] Step 700: taking the optimal expected value V*(S0) corresponding to the current state S0 of the patent as the patent value evaluation result, and giving the optimal operation strategy suggestion corresponding to the action π*(S0).

[0076] Preferably, the optional strategies include: maintaining, licensing, financing through pledge, cooperative development, filing a lawsuit, defensive response to a lawsuit, and abandonment.

[0077] Preferably, the original patent data includes: full text of literature, legal status, international patent classification number, citation-referencing relationship, licensing, pledge, litigation, transfer and marketing event record, patentee and related subject.

[0078] Preferably, the original patent data of the target patent is synchronously acquired, and the original patent data is subjected to missing value filling, feature normalization, and time series alignment to obtain multi-dimensional features, including:

[0079] The full text of literature, legal status, international patent classification number, and citation-referencing relationship of the target patent are acquired through an official patent public interface;

[0080] The licensing, pledge, litigation, transfer, and marketing event record directly associated with the target patent are acquired through a commercial intelligence database;

[0081] The basic information of the patentee and related subject of the target patent is acquired through an enterprise information platform;

[0082] The core field, license, pledge, litigation, transfer and market promotion event record and the basic information are structured and parsed according to a unified field mapping table, and are combined to generate original patent correlation data with a unified timestamp;

[0083] The continuous type field in the original patent correlation data is filled with missing values by using a time series interpolation method, and the category type field in the original patent correlation data is filled with missing values by using a nearest neighbor estimation method;

[0084] The continuous type field in the original patent correlation data is filled with missing values by using a time series interpolation method, and the category type field in the original patent correlation data is filled with missing values by using a nearest neighbor estimation method;

[0085] According to the unified timestamp, the preprocessed data is aligned according to a preset observation window to generate the multi-dimensional features with equal time intervals;

[0086] The fields with inconsistent granularity in the preset observation window are uniformly encoded by using a sliding aggregation strategy to obtain the multi-dimensional features that satisfy the time consistency.

[0087] Specifically, in the original patent data processing stage, the embodiment focuses on the systematic construction of the structure fusion and time synchronization of multi-source information, and the core is to parse and integrate the heterogeneous fields from the patent disclosure interface, the commercial intelligence database and the enterprise information platform through the field mapping table, to form the original patent correlation data in a unified format. In the processing process, only the key fields related to the patent value evaluation are retained, including the legal status change record, the license and pledge behavior, the litigation event details, the right holder information and the cited-citing link, etc., while the redundant fields are removed to ensure that the input features have causal correlation and modeling significance. The field fusion and standardization process provides a computable, comparable and time-indexed data basis for subsequent state modeling, which is an irreplaceable core link of the method.

[0088] In the missing value filling and normalization stage, the embodiment adopts a differentiated processing strategy according to the field type to ensure data integrity and numerical stability. For continuous indicators (such as the sequence of citation frequency increasing over time), the time series interpolation algorithm is used to fill in the observation gaps to ensure the continuity of the trend; for category type fields (such as missing legal status labels), the nearest known value of the nearest sample is selected for estimation to avoid introducing cross-period errors. Normalization uses minimum-maximum scaling and one-hot encoding to ensure that all features fall within a unified scale, ensuring the weight balance between different dimensions and the model interpretability, and all processing is synchronized based on the original timestamp.

[0089] To achieve the time consistency of the final feature input, the embodiment aligns the preprocessed data according to a unified time index and a preset observation window to generate an isochronous feature sequence. In fields with inconsistent time granularity, a sliding aggregation strategy is used to extract window statistics (such as maximum value, mean value, event count, etc.), and the behavior data with unequal intervals is converted into fixed-interval and semantically consistent input features. This process solves the problem that asynchronous data cannot directly participate in state modeling and is a prerequisite for implementing downstream processes such as state transition and policy evaluation.

[0090] Preferably, the multi-dimensional features are mapped to a state S of the patent life cycle, including:

[0091] According to the patent life cycle management requirements, the state S includes at least the following four dimensions: technical maturity T t , market heat M t , legal stability L t and competition density C t ; wherein the technical maturity T t reflects the research and development depth and industrial application degree of the technology to which the patent belongs; the market heat M t reflects the market attention and economic activity intensity of the products or services related to the patent; the legal stability L t reflects the stability, legal survival and historical objection risk of the patent right; and the competition density C t reflects the aggregation degree and technical barrier strength of similar or alternative patents in the technical field.

[0092] A feature vector x t is constructed according to the multi-dimensional features, so that the feature vector x t is used as the input at the current time t;

[0093] A supervised learning model is used to train four mapping functions f1, f2, f3 and f4 using historical label data, and x t is mapped to corresponding index values T t , M t , L t , C t using f1, f2, f3 and f4 respectively; the mapping functions are automatically trained according to historical event labels; and the historical event labels include achievement transformation records, transaction behaviors, litigation situations and citation dynamics.

[0094] The four index values obtained by mapping are assembled into the state S; the expression of the state S is: S = [T t , M t , L t , C t ];

[0095] The dimension values in the state S are normalized according to predefined intervals, so that the dimension values fall within the closed interval [0, 1].

[0096] In the process of constructing the patent life cycle state, in order to ensure the accuracy and operability of the state representation, based on the core dimensions of patent value evaluation, the state is divided into four key indicators of technology maturity, market heat, legal stability and competition density. The above-mentioned indicators all need to have the ability of dynamic update, quantifiable expression and predictability, so they are not directly assigned or scored by expert rules, but are modeled by supervised learning method facing real historical samples, to ensure the objectivity and calculability of state expression. Among them, the sample sources include historical patents with complete life cycle information, covering the whole process behavior from authorization, operation to transfer, litigation, etc., to ensure that the model training has causal correlation data basis.

[0097] The embodiment realizes the numerical conversion from the feature vector to the state indicator by constructing the mapping function. The determination method of the mapping function adopts the regression modeling process in supervised learning, which specifically includes: selecting historical samples showing significant transformation or risk events in the training set, extracting the processed features at each time node in the early stage as input variables, and taking the observed indicator changes before and after the event as label signals, and training multiple regression models facing different dimensions. In order to improve the prediction stability and generalization ability, the embodiment preferably uses an integrated learning structure with nonlinear fitting ability, combined with cross-validation technology to avoid overfitting. Feature selection and model parameter optimization are automatically completed during the training process, without the need for a large amount of manual intervention, to ensure that the applicable range of the mapping function covers multiple patent types and industry situations.

[0098] After the model is trained, the embodiment solidifies the mapping function as a core component in the feature processing process, and directly calls the trained mapping model to predict the multi-dimensional feature vector at the current time point when constructing the state of the patent to be evaluated, to obtain the indicator values of the four dimensions. In order to improve the explainability of the state and the consistency of subsequent decision-making, the embodiment unifies the intervals of the indicator values, so that they have relative comparability and can be used as a standard expression form in the Markov state space. In the whole process, the establishment and calling of the mapping function do not depend on the setting of scoring rules or experience parameters, to ensure the data-driven and repeatability of state construction.

[0099] Preferably, as shown in Figure 2 , an action set A is set for each of the states S, including:

[0100] An applicable condition rule is preset for each of the selectable strategies; the applicable condition rule includes a numerical threshold or interval limit for each dimension indicator in the state S;

[0101] read the indicator value of the current state S, and compare the indicator value with the applicable condition rule corresponding to each strategy;

[0102] The strategy that meets the corresponding applicable condition is included in the action set A corresponding to the current state S.

[0103] Specifically, in order to realize the automation and rationality of strategy screening, the embodiment defines applicable condition rules for each strategy in advance, which are set according to the state indicators of patents in the dimensions of technology, market, law and competition, and specifically include the value range or combination constraint conditions allowed by each indicator. After the current state is generated, the embodiment reads the indicator values of each dimension in the state, and automatically compares them with the applicable rules corresponding to each strategy in turn. If the indicator values meet all the prerequisite conditions of a strategy, the strategy is determined as a feasible strategy under the current state, and is included in the action set corresponding to the state. All rule settings in this process are generated based on historical operation data statistical analysis results or rule learning algorithm, avoiding human experience intervention, ensuring the repeatability and business rationality of the selection logic of the strategy space, and being the basic link for realizing the structured modeling of state-action matching.

[0104] Preferably, the expected cash flow, litigation compensation, license fee rate, holding cost and risk exposure generated by executing any action in the action set A under each state S are used to set the immediate reward R(S, a), including:

[0105] The reward function R(S, a) is set to quantify the immediate economic return and risk cost brought by executing action a in state S; the function form is: R(S, a) = E[CF(S, a)] + E[LC(S, a)] + E[PC(S, a)] - C(S, a) - RSK(S, a); wherein, E[CF(S, a)] represents the expected cash flow income, E[LC(S, a)] represents the expected license income brought by the license behavior, E[PC(S, a)] represents the compensation income that may be brought by litigation or claim for compensation, C(S, a) represents the direct or indirect cost required for executing action a in state S, and RSK(S, a) represents the risk exposure indicator accompanied by the action;

[0106] Obtain a data sample set containing a plurality of historical "state-action-result" triplets; the data sample set includes the technical indicators, market feedback, legal changes and operation strategy behaviors and economic consequences of historical patents;

[0107] Train a prediction model for each indicator in R(S, a), including:

[0108] fitting prediction functions of E[CF(S, a)], E[LC(S, a)], E[PC(S, a)] by using preset supervised learning methods; the supervised learning methods are any one of linear regression, decision tree or neural network;

[0109] a cost evaluation function of C(S, a) is established by using a cost aggregation model or an expert estimation method;

[0110] a risk expectation model of RSK(S, a) is constructed based on a Bayesian probability graph or a risk event frequency modeling method;

[0111] each dimension indicator of the current state S and the action a are taken as input variables, and are substituted into each prediction model, and an expected value of a corresponding indicator is output and substituted into the reward function R(S, a);

[0112] for each action a e A in the state S and the corresponding action set A, a calculation process of the prediction model is performed, and a value of R(S, a) is obtained.

[0113] In the construction of the immediate reward, the embodiment takes the reflection of the real historical behavior on the economic results of the patent as the modeling basis, and extracts the mapping relationship between the technology, market, law and other indicators and the revenue or risk after the execution of the strategy by statistically aggregating the data samples of the relationship between a plurality of known states, strategies and results. In the model construction stage, the embodiment respectively trains the positive revenue such as cash flow, license income and litigation compensation by using a supervised learning method with generalization ability. The method used can select a regression algorithm or a deep network structure according to the data distribution; for the cost and risk items, the corresponding evaluation model is constructed according to the cost aggregation logic or the risk event frequency in the historical data, so as to avoid subjective estimation error. When the state to be evaluated and the selected strategy are obtained, the embodiment automatically takes them as inputs, substitutes them into the trained prediction model, outputs the expected values of each revenue and cost item, and completes the unified calculation of the immediate reward.

[0114] Preferably, the calculation formula of the probability P(S'|S, a) is:

[0115]

[0116] wherein P(S'|S, a) is the probability of transitioning to the next state S' after performing the action a in the current state S; γ is a discount factor for discounting the value of the next state; V(S') is the expected value of the state S' predicted by the value function network; η is a distance penalty coefficient for suppressing unreachable or too high cost transitions; D(E(S'), E(S)+E a ) is a distance between the state embedding vector E(S') and the vector E athe Euclidean distance between the resulting target vectors; exp is the exponential function; ∑ S″∈Ω denotes a normalized sum over all possible next states in the state space Ω to ensure a probability sum of 1.

[0117] The embodiment focuses on solving the problems of directionality, profitability and reachability evaluation of state transition when constructing the state transition probability model. The discount factor involved is derived from the time value weight set during model initialization, which is set according to the discounting demand of future income in the target scenario. The state value is output by the value function network obtained through pre-training, and is repeatedly iteratively optimized based on the constructed state-policy-reward system. The distance penalty coefficient is optimized by comparing the deviation between the predicted path and the real evolution path during model training, to ensure that unreasonable transition probabilities are suppressed. The vector representation of states and actions is embedded and mapped in the feature construction stage, and the embedding space has the ability to maintain the consistency of semantic structure after unified training. The embodiment integrates the above key parameters into the state transition probability formula, and ensures the consistency of the probability sum through the exponential normalization mechanism, so as to realize the accurate modeling of the system evolution trend after the strategy is executed.

[0118] Preferably, based on the Bellman optimality equation, the optimal value V*(S) and the optimal policy π*(S) corresponding to each state S are calculated by a policy iteration algorithm, including:

[0119] a) initializing a policy π0(S), for each state S in the state space Ω, randomly selecting an executable action a∈A(S) as the initial action of the current policy π0(S);

[0120] b) calculating the state value function V k (S) for the current policy π k (S); the V k (S) satisfies the Bellman optimality equation: V k (S) = R(S, π k (S))+γΣ S′ P(S′∣S, πk(S))·V k (S′);

[0121] c) solving the Bellman optimality equation by using an iterative method or a matrix solution method until a preset convergence condition ε is met, to obtain a stable state value function V k (S);

[0122] d) updating the policy π k+1 (S) for each state S to a new action that can maximize the state value:

[0123]

[0124] e) if π k+1 (S)≠π k (S), then let k←k+1, repeat steps b) to d); otherwise terminate the iteration and output the converged state optimal value function V * (S) and the corresponding policy π * (S); wherein k represents the iteration index of the policy iteration.

[0125] In the policy solving phase, the policy iteration algorithm based on the Bellman optimality equation is used to estimate the value of each state and select the optimal action, so as to obtain the global optimal expected return under all possible policy paths. First, the state space is initialized in the policy, and in the absence of prior knowledge, a random action is assigned to each state as the initial policy selection, thereby forming the first round of policy distribution. The initial policy is not directly used for decision output, but serves as a starting reference point for subsequent iterative solving.

[0126] After the policy initialization is completed, the policy evaluation operation is performed, that is, the expected value of each state under the current policy is calculated based on the current policy structure. In this process, the value of each state depends not only on the immediate reward brought by the action selected by the current policy, but also on the state transition path caused by the action and the value level of the subsequent state. Therefore, the trained immediate reward function and state transition probability function are called and substituted into the Bellman optimality equation for iterative solving. In order to ensure the stability and convergence of the solving result, the relaxation iteration or sparse matrix solving mechanism is used, and when the state value variation of two consecutive rounds is lower than the set convergence threshold, it is considered that the state value function under the current policy has been stabilized.

[0127] After completing the policy evaluation of the current round, the policy improvement phase is entered, that is, for each state, the action that can maximize the expected value is reselected, and it is updated as the policy mapping of the next round. In this process, based on the state value obtained at present, the value of all candidate actions is calculated, and the action with the highest return is selected as the optimal action for the next iteration. When the policy update is the same as the last round, it means that the policy has reached the optimal, and there is no need to continue iteration; otherwise, it will enter the next round of evaluation and update cycle. Through the above policy iteration mechanism, the optimal solution of the state-policy space is solved within a limited number of rounds, and a stable policy output structure for all states is formed.

[0128] Preferably, the optimal expected value V*(S0) corresponding to the current state S0 of the patent is taken as the patent value evaluation result, and the optimal operation policy suggestion corresponding to the action π*(S o ) is given, including:

[0129] Based on the current feature data of the patent, the position of the feature data in the state space is determined, and a current state S0 is obtained;

[0130] The optimal value function V*(S) calculated in advance is called to obtain the optimal expected value V*(S0) corresponding to the current state S0;

[0131] The optimal policy function π*(S) generated in advance is called to obtain the optimal action π*(S0) corresponding to the current state S0;

[0132] V*(S0) is output as the value evaluation result of the patent, and the strategy content corresponding to the action π*(S0) is converted into a textual operation suggestion and displayed or recorded.

[0133] In the output of the patent value evaluation result and operation strategy suggestion, based on the completed state modeling and strategy optimization result, the mapping position of the patent in the state space is directly determined from the current feature data, and the corresponding current state is obtained. The mapping process relies on the previously constructed feature-state conversion mechanism to ensure that the current state recognition result has timeliness and comparability in semantics, which is the prerequisite for subsequent calling of the optimal strategy and value function. The embodiment converts the original feature input into a standard state expression through a structured coding method, avoiding state recognition deviation caused by feature noise or discretization error.

[0134] After completing the state positioning, the optimal value function and the optimal policy function converged through the previous policy iteration process are called to obtain the optimal expected value and the best executable action corresponding to the state. The numerical value of the patent is directly output by the optimal value function, which has the discounted income significance under the whole cycle strategy expectation, and can be used as the basis for horizontal evaluation and sorting; the corresponding optimal action is obtained by looking up the strategy function table, and further combined with the action type, execution condition and resource demand to generate a text suggestion, such as maintaining, transferring, licensing or abandoning, etc. Operation direction.

[0135] The beneficial effects of the present application are as follows:

[0136] (1) The present application constructs a dynamic mapping relationship between the patent life cycle state and the strategy behavior based on Markov decision process, breaks through the limitation of traditional patent evaluation method based on static index scoring, can reflect the value change trend and optimal operation path of the patent at different development stages, and significantly improves the timeliness and strategy guidance of the evaluation.

[0137] (2) The application establishes a mapping function between features and states through a supervised learning method, and combines historical patent operation data to build an instant reward and state transition probability model, so that the value evaluation result is not dependent on artificial subjective judgment, but is automatically learned based on real economic behavior and risk performance, ensuring the objectivity and accuracy of the evaluation.

[0138] (3) The application adopts a policy iteration method to solve the global optimal solution based on the Bellman optimality principle, can select the one with the maximum long-term revenue among multiple candidate strategy paths, and provides executable, interpretable and landable operation suggestions for decision makers, is suitable for investment evaluation, license negotiation, risk review and other application scenarios, and has strong practicability.

[0139] (4) The application can be flexibly adapted to different industries, different regions and different types of patent data, support the expansion and customization of feature dimensions, strategy sets and evaluation periods, have good universality and engineering deployment value, and are suitable for integration into various intellectual property management and investment research platforms.

[0140] Each embodiment in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments.

[0141] The principles and implementation modes of the application are described by applying specific examples in this paper, and the above description of the embodiments is only used to help understand the method of the application and its core idea; at the same time, for those skilled in the art, according to the idea of the application, the specific implementation mode and application range will be changed. In view of the above, the content of the specification should not be understood as a limitation of the application.

Claims

1. A method for patent value assessment based on Markov decision making, characterized in that: include: Synchronously obtain the original patent data of the target patent, and perform missing value filling, feature normalization, and time series alignment on the original patent data to obtain multidimensional features; Mapping the multidimensional features into a state S of the patent life cycle; the state S at least describes the technical maturity, market popularity, legal robustness and competition density; For each state S, an action set A is set; the action set A includes a plurality of optional strategies; Set an immediate reward R(S,a) based on the expected cash flow, litigation compensation, licensing rate, holding cost, and risk exposure generated by executing any action in the action set A in each state S; where a∈A and a discount factor γ∈(0,1); Multi-task Bayesian learning and time-dependent Wiener process are used to fit historical events and estimate the probability P(S′|S,a) of transitioning to state S′ after executing an action a in action set A in state S. Based on the Bellman optimality equation, the optimal value V*(S) and the optimal strategy π*(S) corresponding to each state S are calculated through the policy iteration algorithm; The optimal expected value V*(S0) corresponding to the current state S0 of the patent is taken as the patent value evaluation result, and the optimal operation strategy recommendation corresponding to the action π*(S0) is given.

2. The method for patent value assessment based on Markov decision making according to claim 1, characterized in that: The available strategies include: maintenance, licensing, pledge financing, joint development, litigation, litigation defense, and abandonment.

3. The method for patent value assessment based on Markov decision making according to claim 1, characterized in that: The original patent data includes: full text of the document, legal status, international patent classification number, citation-citing relationship, licensing, pledge, litigation, transfer and marketing event records, patent owners and related entities.

4. The method for patent value assessment based on Markov decision making according to claim 3, characterized in that: The original patent data of the target patent is obtained simultaneously, and the missing value filling, feature normalization and time series alignment are performed on the original patent data to obtain multi-dimensional features, including: Obtain the full text, legal status, international patent classification number, and core fields of the cited-citing relationship of the target patent through the official patent disclosure interface; Obtain records of licensing, pledges, litigation, transfers, and marketing events directly related to the target patent through business intelligence databases; Obtain basic information on the patentee and related entities of the target patent through the enterprise information platform; Performing structured analysis on the core fields, licensing, pledge, litigation, transfer and marketing event records and the basic information according to a unified field mapping table, and merging them to generate original patent-related data with a unified timestamp; Filling missing values ​​for continuous fields in the original patent-linked data using time series interpolation, and filling missing values ​​for categorical fields in the original patent-linked data using nearest neighbor estimation; Applying min-max scaling to the filled continuous fields in the original patent-related data, and applying one-hot encoding to the filled categorical fields in the original patent-related data, to obtain preprocessed data; According to a unified timestamp, the pre-processed data are aligned according to a preset observation window to generate the multi-dimensional features with equal time intervals; The fields with inconsistent granularity within the preset observation window are uniformly encoded using a sliding aggregation strategy to obtain the multidimensional features that meet the time consistency.

5. The method for patent value assessment based on Markov decision making according to claim 3, characterized in that: Mapping the multidimensional features into the state S of the patent life cycle includes: According to the patent life cycle management requirements, the state S is set to include at least the following four dimensions: technology maturity T t 、Market heat M t , Legal robustness L t and competition density C t ; Among them, the technology maturity T t Reflects the R&D depth and industrial application level of the technology to which the patent belongs; Market popularity M t Reflects the market attention and economic activity intensity of patent-related products or services; Legal robustness L t Reflects the stability, legal continuity and historical objection risk of patent ownership; competition density C t Reflects the degree of aggregation of similar or alternative patents in the technical field and the strength of technical barriers; Construct a feature vector x based on the multidimensional features t , to transform the eigenvector x t As the input at the current time t; Use the supervised learning model to train four mapping functions f1, f2, f3 and f4 with historical label data, and use f1, f2, f3 and f4 to transform x t Mapped to the corresponding index value T t 、M t , L t 、C t The mapping function is automatically trained based on historical event labels; the historical event labels include achievement transformation records, transaction behaviors, litigation situations, and citation dynamics; The four index values ​​obtained by mapping are assembled into the state S; the expression of the state S is: S=[T t , M t , L t , C t ]; Each dimension value in the state S is normalized according to a predefined interval so that the dimension value falls into the closed interval [0, 1].

6. The method for patent value assessment based on Markov decision making according to claim 1, characterized in that: For each state S, an action set A is set, including: Preset applicable condition rules for each optional strategy; the applicable condition rules include numerical thresholds or interval limits for each dimension indicator in the state S; Read the indicator value of the current state S and compare the indicator value with the applicable condition rules corresponding to each strategy; The strategies that meet the corresponding applicable conditions are included in the action set A corresponding to the current state S.

7. The method for patent value assessment based on Markov decision making according to claim 1, characterized in that: The immediate reward R(S,a) is set based on the expected cash flow, litigation compensation, licensing rate, holding cost and risk exposure generated by executing any action in the action set A in each state S, including: A reward function R(S,a) is set to quantify the immediate economic return and risk cost of performing action a in state S. The function is expressed as follows: R(S,a) = E[CF(S,a)] + E[LC(S,a)] + E[PC(S,a)] – C(S,a) – RSK(S,a); where E[CF(S,a)] represents the expected cash flow, E[LC(S,a)] represents the expected licensing revenue from licensing, E[PC(S,a)] represents the potential compensation from litigation or asserting a claim, C(S,a) represents the direct or indirect cost of performing action a in state S, and RSK(S,a) represents the risk exposure associated with the action. Obtain a data sample set containing multiple historical "state-action-result" triplets; the data sample set includes: technical indicators of historical patents, market feedback, legal changes and operational strategy behaviors and economic consequences; The prediction model is trained for each indicator in R(S,a), including: Using a preset supervised learning method to fit the prediction functions of E[CF(S,a)], E[LC(S,a)], and E[PC(S,a)]; the supervised learning method is any one of linear regression, decision tree, or neural network; Use cost aggregation model or expert estimation method to establish cost evaluation function of C(S,a); Construct a risk expectation model of RSK(S,a) based on Bayesian probability graph or risk event frequency modeling method; The dimensional indicators of the current state S and action a are used as input variables, substituted into the prediction models, and the expected values ​​of the corresponding indicators are output and substituted into the reward function R(S,a); For all the states S and each action a∈A in the corresponding action set A, the calculation process of the prediction model is executed to obtain the R(S,a) value.

8. The method for patent value assessment based on Markov decision making according to claim 1, characterized in that: The calculation formula of the probability P(S′|S,a) is: Where P(S′|S,a) is the probability of transitioning to the next state S′ after executing action a in the current state S; γ is the discount factor used to discount the value of the next state; V(S′) is the expected value of state S′ predicted by the value function network; η is the distance penalty coefficient used to suppress unreachable or too costly transitions; D(E(S′),E(S)+E a ) is the state embedding vector E(S′) and the current state vector E(S) and the action vector E a The Euclidean distance between the superimposed target vectors; exp is the exponential function; ∑ S″∈Ω It represents the normalized summation of all possible next states in the state space Ω to ensure that the sum of the probabilities is 1.

9. The method for patent value assessment based on Markov decision making according to claim 8, characterized in that: Based on the Bellman optimality equation, the optimal value V*(S) and the optimal strategy π*(S) corresponding to each state S are calculated through the policy iteration algorithm, including: a) Initialize the strategy π0(S). For each state S in the state space Ω, randomly select an executable action a∈A(S) as the initial action of the current strategy π0(S). b) For the current strategy π k (S) Calculate the state value function V k (S); the V k (S) satisfies the Bellman optimality equation: V k (S)=R(S,π k (S))+γ∑ S′ P(S′|S,π k (S))·V k (S′); c) Solve the Bellman optimality equation using an iterative method or a matrix solution method until the preset convergence condition ε is met, and a stable state value function V is obtained k (S); d) For each state S, update the strategy π k+1 (S) is the new action that maximizes the state value: e) If π k+1 (S)≠π k (S), let k←k+1 and repeat steps b) to d); otherwise, terminate the iteration and output the optimal value function V after convergence. * (S) and the corresponding strategy π * (S); where k represents the round index of the policy iteration.

10. The method for patent value assessment based on Markov decision making according to claim 8, characterized in that: The optimal expected value V*(S0) corresponding to the current state S0 of the patent is used as the patent value evaluation result, and the optimal operation strategy corresponding to the action π*(S0) is given, including: Based on the current feature data of the patent, determine the position of the feature data in the state space and obtain the current state S0; Call the pre-calculated optimal value function V*(S) to obtain the optimal expected value V*(S0) corresponding to the current state S0; Call the pre-generated optimal policy function π*(S) to obtain the optimal action π*(S0) corresponding to the current state S0; V*(S0) is output as the value assessment result of the patent, and the strategy content corresponding to action π*(S0) is converted into textual operational suggestions and displayed or recorded.

Images