System for Deriving Technology Development Trajectories Based on Patent Citation Network Analysis Using Reinforcement Learning and Method for Deriving Technology Development Trajectories Using the same

Abstract

A system for deriving a technology development trajectory based on patent citation network analysis using reinforcement learning and a method for deriving a technology development trajectory using the same are disclosed. The system for deriving a technology development trajectory based on patent citation network analysis using reinforcement learning according to the present invention comprises: a patent data storage unit that stores patent data collected from a database; a patent citation network map generation unit that generates a patent citation network map using patent citation information among the patent data stored in the patent data storage unit; a user input unit that receives a user command to apply machine learning to the patent citation network map generated by the patent citation network map generation unit; an artificial intelligence learning unit that learns a major path derivation algorithm for the corresponding patent citation network map according to the user command input from the user input unit; and a major path derivation unit that derives a technology development trajectory for the corresponding patent citation network map using a learning model learned by the artificial intelligence learning unit.

Description

Technology Field

[0001] The present invention relates to a system for deriving a technology development trajectory and a method for deriving a technology development trajectory using the same, and more specifically, to a system for deriving a technology development trajectory based on a patent citation network utilizing reinforcement learning and a method for deriving a technology development using the same. Background Technology

[0003] Patent information contains specific and detailed information regarding an invention, which is the result of a technical concept utilizing natural laws. Among the types of patent information, cited information is classified into cited information, which refers to prior art literature to complete the patented invention, and cited information, which refers to the content of the relevant patented invention.

[0004] Multiple patents are interconnected through this patent citation information, and these connections form a patent citation network. Patent citation networks are utilized as a highly useful analytical tool for identifying the flow of technical knowledge. However, because these networks contain too much information, they can cause difficulties in grasping the flow of technical knowledge.

[0005] Using patent citation network analysis to identify the flow of technical knowledge and to track the evolutionary trajectory of technical or scientific knowledge using major path analysis has the advantage of reducing the complexity of patent citation networks and identifying important knowledge paths.

[0006] Conventional methods relied on rule-based approaches that calculated the weights of links constituting major paths within complex patent citation networks and searched for paths. However, it has been pointed out that such rule-based methods have technical limitations in reducing path complexity within complex citation networks. Prior art literature

[0008] Patent Document 1: Korean Published Patent No. 10-2016-0124386 (Publication Date: Oct. 27, 2016) Patent Document 2: Korean Registered Patent No. 10-2632595 (Publication Date: Jan. 31, 2024) Patent Document 3: Korean Registered Patent No. 10-1278135 (Publication Date: June 27, 2013)

[0009] Non-patent Document 1: Batagelj, Vladimir. 2003. “Efficient algorithms for citation network analysis.” arXiv preprint cs / 0309023. Verspagen, Bart. 2007. “Mapping technological trajectories as patent citation networks: A study on the history of fuel cell research.” Advances in complex systems 10 (01) The problem to be solved

[0010] The present invention, devised based on the aforementioned necessity, aims to provide a system for deriving a technology development trajectory based on patent citation network analysis utilizing reinforcement learning and a method for deriving a technology development trajectory using the same, so as to accurately derive the most important knowledge path in a patent citation network analysis method that collects a large number of patent documents and analyzes them using citation information of the patent documents as node and edge information. means of solving the problem

[0012] To achieve the above objective, a system for deriving a technology development trajectory based on patent citation network analysis using reinforcement learning according to one embodiment of the present invention comprises: a patent data storage unit (110) for storing patent data collected from a database; a patent citation network map generation unit (120) for generating a patent citation network map using patent citation information among the patent data stored in the patent data storage unit (110); a user input unit (130) for receiving a user command to apply machine learning to the patent citation network map generated by the patent citation network map generation unit (120); an artificial intelligence learning unit (140) for learning a major path derivation algorithm for the corresponding patent citation network map according to the user command input from the user input unit (130); and a major path derivation unit (150) for deriving a technology development trajectory for the corresponding patent citation network map using a learning model learned by the artificial intelligence learning unit (140).

[0013] In this case, the patent citation network map generation unit (120) can select a patent belonging to a technical area according to a user-defined criterion among the patent data stored in the patent data storage unit (110), extract patent citation information and patent identification information from the selected patent data, and generate a patent citation network map using the extracted patent identification information and patent citation information.

[0014] In this case, the artificial intelligence learning unit (140) can define a state by setting the patent identification information as a node, define an action by setting the patent citation information as an edge between the nodes, and establish a reinforcement learning environment by defining a reward matrix for the entire patent citation network map.

[0015] In this case, the artificial intelligence learning unit (140) can define the state of all nodes constituting the patent citation network map to be analyzed as the input of the learning model, define the action for the interconnection between all nodes as the output of the learning model, define the reward (r) that the agent of the learning model can obtain for the action (a) that can be taken for the input state(s), and define a Q-value composed of numbers for all state-action pairs (s, a) consisting of all states and actions.

[0016] In this case, the artificial intelligence learning unit (140) is characterized in that when a specific action (at) is performed randomly on a specific node at a specific time (t) and a specific state (st), the node is moved to the next state (st+1), and when a reward R(st, at) for the action (at) is obtained, the time difference (TD) obtained due to the action (at) is defined by the following mathematical formula.

[0017]

[0018] Here, Is class Defined by the difference of, silver state s t Operation a in t It refers to the reward obtained by performing, is future state s t+1 It refers to the discounted value of the queue value of the best operation performed in, and silver state s t Actions performed in (a t It means the queue value of ).

[0019] In this case, the artificial intelligence learning unit (140) can train the agent of the learning model to update the queue value according to the following mathematical formula, and can repeat the learning a specific number of times until the queue value is no longer updated.

[0020]

[0021] Here, is the learning rate and means a real value between 0 and 1, and The smaller the value, the smaller the update to the Q-value, and the slower the learning speed may be.

[0022] In this case, the main path derivation unit (150) constructs a matrix of queue values ​​that have been learned by the artificial intelligence learning unit (140), and for a path moving from an arbitrary starting node to a destination node, a specific state (s t The operation of the highest queue value in ) (a t The operation at a specific node can be determined according to the following formula to perform ).

[0023] .

[0024] A method for deriving a technology development trajectory according to another embodiment of the present invention is a method for deriving a technology development trajectory using a system comprising a patent data storage unit (110), a citation network map generation unit (120), a user input unit (130), an artificial intelligence learning unit (140), and a main path derivation unit (150), comprising: a step of loading patent data for a specific technology field among patent data previously stored by the patent data storage unit (110); a step of generating a patent citation network map using patent identification information and patent citation information of the loaded patent data by the citation network map generation unit (120); a step of receiving a discount rate, a learning rate, and a number of iterations input by the user input unit (130); and a step of performing reinforcement learning on the patent citation network map according to the previously input discount rate, learning rate, and number of iterations by the artificial intelligence learning unit (140). The method includes the step of calculating a technical main path by applying a learning model calculated according to the reinforcement learning by the main path derivation unit (150) to a previously generated patent citation network map; wherein the step of performing the reinforcement learning includes performing an operation from a specific node to another node on a previously generated citation network map, calculating a reward when another node is reached, calculating a time difference for the calculated reward, updating a current queue value using the calculated time difference and an existing queue value, and terminating the learning when the update of the queue value is no longer performed. Effects of the invention

[0026] According to various embodiments of the present invention, by using patent citation information to concisely identify the evolutionary trajectory of technical or scientific knowledge in a specific technical field, it is possible to predict the paths of technological trends and the convergence and diffusion of technologies, and

[0027] Furthermore, by optimizing the technological evolution path of the patent citation network, it is effective in providing a technology development roadmap for efficient technology development. Brief explanation of the drawing

[0029] FIG. 1 is a block diagram exemplarily illustrating a system for deriving a technology development trajectory based on patent citation network analysis using reinforcement learning according to an embodiment of the present invention, FIG. 2 is a flowchart exemplarily illustrating a method for deriving a technological development trajectory according to another embodiment of the present invention, FIG. 3 is a flowchart exemplarily illustrating the process of deriving a technological development trajectory according to another embodiment of the present invention, FIG. 4 is a diagram exemplarily illustrating a patent citation network map and a reward matrix applied to a method according to another embodiment of the present invention, FIG. 5 is a flowchart exemplarily illustrating the procedure for performing reinforcement learning (Q-learning) during the process illustrated in FIG. 3, FIG. 6 is a diagram exemplarily illustrating an example of a reward matrix learned by a method according to another embodiment of the present invention. Figure 7 is a diagram visualizing a network map by major technology path analysis (SPLC) for a specific technology field (photovoltaic technology). FIG. 8 is a diagram visualizing the main technical path with noise removed from the network map shown in FIG. 7. FIG. 9 is a diagram visualizing major technology paths by GKPM (Genetic Knowledge Persistence Measurement) for a specific technology field (photovoltaic technology), and, FIG. 10 is a diagram visualizing a major technology path for a specific technology field (photovoltaic technology) by applying the method according to the present invention. Specific details for implementing the invention

[0030] Preferred embodiments of the present invention will be described below with reference to the drawings. The preferred embodiments described below may be modified in various ways within the scope of the technical concept of the present invention.

[0032] FIG. 1 is a block diagram illustrating, in an exemplary manner, a system for deriving a technology development trajectory based on patent citation network analysis utilizing reinforcement learning according to an embodiment of the present invention. Referring to FIG. 1, the technology development trajectory derivation system (100) includes a patent data storage unit (110), a patent citation network map generation unit (120), a user input unit (130), an artificial intelligence learning unit (140), and a main path derivation unit (150).

[0034] The patent data storage unit (110) collects and stores patent data from a patent database or web page connected to the web. The patent data storage unit (110) may be configured as a storage device provided on a local computer or as a storage device provided in a cloud manner on the web.

[0036] The patent citation network map generation unit (120) loads patent data stored in the patent data storage unit (110). The patent citation network map generation unit (120) assigns each node using patent identification information (e.g., patent application number, patent publication number, patent registration number, etc.) included in the patent data. The patent citation network map generation unit (120) generates a patent citation network map by forming links between multiple nodes using citation information.

[0038] Specifically, the patent citation network map generation unit (120) selects patents belonging to a technical area according to user-defined criteria from among the patent data stored in the patent data storage unit (110). The patent citation network map generation unit (120) extracts patent citation information and patent identification information from the selected patent data. The patent citation network map generation unit (120) defines each node using the extracted patent identification information and generates a patent citation network map by connecting each defined node using the patent citation information.

[0040] The user input unit (130) receives a user command to apply machine learning to the patent citation network map generated by the patent citation network map generation unit (120). The user input unit (130) receives information such as a discount rate, learning rate, and number of iterations for machine learning from the user.

[0042] The artificial intelligence learning unit (140) learns a main path derivation algorithm for the corresponding patent citation network map according to a user command input from the user input unit (130). The artificial intelligence learning unit (140) defines a node defined by patent identification information as a state, and defines an action for the connection (edge) between each node based on patent citation information. The artificial intelligence learning unit (140) constructs a data model environment for reinforcement learning by defining a reward matrix by constructing the state-action in rows and columns based on the connection relationships of the nodes constituting the entire network.

[0044] Next, the artificial intelligence learning unit (140) defines the state of all nodes constituting the patent citation network map to be analyzed as the input to the learning model. The artificial intelligence learning unit (140) defines the action for the interconnection between all nodes as the output of the learning model. The artificial intelligence learning unit (140) defines the reward (r) that the agent of the learning model can obtain for the action (a) that can be taken for the input state(s). The artificial intelligence learning unit (140) defines a Q-value composed of numbers for all state-action pairs (s, a) consisting of all states and actions.

[0046] Then, the artificial intelligence learning unit (140) moves the node (Node X) to the next state (st+1) when a specific action (at) is performed randomly on a specific node (Node X) at a specific time (t) and a specific state (st). The artificial intelligence learning unit (140) obtains a reward R(st, at) for the action (at) that moves the node (Node X) from the first state to the second state. The artificial intelligence learning unit (140) calculates the time difference (TD) obtained by the action (at) according to the following mathematical formula 1.

[0048]

[0049] Here, Is class Defined by the difference of, silver state s t Operation a in t It refers to the reward obtained by performing, is future state s t+1 It refers to the discounted value of the queue value of the best operation performed in, and silver state s t Actions performed in (a t It means the queue value of ).

[0051] Then, the artificial intelligence learning unit (140) trains the agent of the learning model to update the queue value according to the following mathematical formula 2. At this time, the artificial intelligence learning unit (140) can repeat the learning a specific number of times until the queue value is no longer updated.

[0053]

[0054] Here, is the learning rate and means a real value between 0 and 1, and The smaller the value, the smaller the update to the Q-value, and the slower the learning speed may be.

[0055] The main path derivation unit (150) constructs a matrix of queue values ​​that have been learned by the artificial intelligence learning unit (140). The main path derivation unit (150) determines a specific state (s) for a path moving from an arbitrary starting node to a destination node. t The operation of the highest queue value in ) (a t The operation at a specific node can be determined according to the following mathematical formula 3 to perform ).

[0057]

[0059] The main path derivation unit (150) can derive a technology development trajectory for the corresponding patent citation network map using a learning model learned by the artificial intelligence learning unit (140).

[0061] FIG. 2 is a flowchart illustrating an exemplary method for deriving a technology development trajectory according to another embodiment of the present invention. Referring to FIG. 2, the method for deriving a technology development trajectory includes the steps of loading patent data (S210), generating a patent citation network map (S220), receiving user input for a learning model (S230), performing reinforcement learning on the patent citation network map (S240), and calculating a major technology path in the patent citation network by the completed reinforcement learning model (S250).

[0063] FIG. 3 is a flowchart exemplarily illustrating the process of deriving a technology development trajectory according to another embodiment of the present invention. Referring to FIG. 3, patent data for a specific technology field (e.g., solar power technology field) is collected (S310). To increase suitability to the technology field, patent data is selected using patent information (e.g., IPC, UPC, etc.). For the selected patent data, a patent citation network map is constructed using patent identification information and patent citation information (S320). Patent identification information utilizes information that uniquely identifies each node. For example, patent publication numbers can be organized in ascending order and sequentially assigned numbers to assign identification information to each node. Additionally, a patent citation network map is generated by completing links based on citation information regarding connectivity between nodes to which identification information has been assigned. Then, states and actions are defined for the completed patent citation network map, a reward matrix for the state-action relationship is constructed, and reinforcement learning is performed on the patent citation network map based on a Q-learning algorithm (S330).

[0065] Q-learning, a model-free reinforcement learning algorithm for learning action values ​​of specific states, is used to identify key paths in a patent citation network. Q-learning operates in a predefined environment including states (inputs), actions (outputs), and rewards, and is modeled by a Markov decision process. Q-learning uses a learning mode and an inference mode, and during the learning mode, it can learn a parameter called the Q-value.

[0067] The first step in performing Q-learning is to build the environment. Building the environment requires three steps. First, define the states (inputs). Second, define the actions that can be performed. Third, define the reward obtained after performing an action in a given state.

[0069] The first step is to define the state. The state (St) at a specific time (t) is the position of the virtual agent at that time (t). If the primary pathfinding problem in a patent citation network is considered as finding the optimal path from a starting patent node (Node X) to a destination patent node (Node Y), each patent node can be defined as a state.

[0071] Referring to FIG. 4(a), each patent node (node ​​A to node I) can be defined as a state. An action that allows moving from a patent node located at time t to the next state can be defined as a patent node filed after time t. In a situation where a given agent starts at patent node A and reaches I, the reward matrix can be composed of a 9x8 matrix. Here, the rows correspond to the state and the columns correspond to the action. The reward matrix is ​​constructed by assigning a value of 1 to the reward for performing the first action (a1) in the first state (s1) if an actual citation party exists. The reward matrix is ​​configured to provide a reward when moving to a node where an actual citation occurred.

[0073] In the second step, an action is defined. An action refers to the next possible destination that the agent can move to. In a patent citation network, the next possible destination refers to a patent node that was filed later than the patent node at time (t).

[0075] The agent receives a reward for moving to patent nodes B, C, and D while located at patent node A. The final reward matrix is ​​constructed as shown in FIG. 4(b). In this example, a much higher reward (1000) is given than when performing other tasks to make it clear that node I is the final destination.

[0077] Referring to FIG. 4(b), when an agent is randomly located at node A, the movable nodes are B to I because it can be cited by nodes B to I, which are patented after node A. However, when moving from node A to nodes B, C, and D where actual citations occur, a greater reward is given. Even when an agent is located at node C, a higher reward may be given when moving to nodes G and H where actual citations occur.

[0079] Finally, a reward system is defined. Specifically, a function R is defined as shown in the following mathematical equation 4, which takes an input state (s) and an action (a) and returns a reward consisting of numbers that the agent can obtain by performing action (a).

[0081]

[0083] A reinforcement learning environment for the patent citation network map is constructed by defining these states, actions, and reward systems. Then, Q-learning is performed on the constructed reinforcement learning environment. The process of performing Q-learning is explained in more detail below with reference to separate drawings.

[0085] The Q-learning model is established after building the environment by clearly defining the goal of the relevant reward system. It consists of three important concepts: Q-value, time difference, and the Bellman equation.

[0087] First, for every state and action pair (s, a), a numeric value Q(s, a) is defined as follows.

[0089]

[0090] In the above mathematical formula 5, Q(s, a) represents the Q-value of the operation performed in state s.

[0092] Second, temporal difference refers to a method for estimating the value function or action-value function of a given state-action pair. A specific state s at a specific time t. t When in, you can choose to perform a random action. This action is the next state s t+1 We take it to, and we reward R(s t , a t ) obtains. In this situation, TD t (s t , a t The time difference at time t represented by ) is equal to the mathematical formula 1 described above.

[0094] The first part of mathematical equation 1 is state s t Operation a in t Reward R(s obtained by performing t , a t ) and future state s t+1 The sum of the Q-values ​​of the best operations performed is the value discounted by the factor γ∈[0, 1], which is called the discount factor.

[0096] The second part of mathematical equation 1 is state s t Action a performed in t It is the Q-value. The time difference (TD) indicates how well the agent is learning. When learning begins, the Q-value is set to 0. Since the agent aims to receive better rewards, it looks for the case where the TD is the largest.

[0098] When a good reward is received, the specific Q-value of the state (state, action) that received a large reward increases, allowing the agent to remember how to obtain high rewards. This increased Q-value is important information because it tells the agent which transition is better for receiving good rewards. Through many iterations, the agent learns all transitions that lead to good rewards and high Q-values. The Q-values ​​of these transitions have already increased over time, and ultimately, the transition duration (TD) decreases. Therefore, as the agent approaches the final goal, the transition duration becomes smaller.

[0100] In order for the agent to perform a better action that leads to goal achievement, it must increase the Q-value of the action when it discovers a high TD. To do this, the Q-value is updated using the Bellman equation as described in Equation 2 above.

[0102] In mathematical equation 2, α∈R is the learning rate and usually has a value between 0 and 1. The smaller the value of α, the smaller the update of the Q-value and the longer Q-learning takes, and vice versa.

[0104] FIG. 5 is a flowchart that exemplarily illustrates the procedure for performing reinforcement learning (Q-learning) during the process illustrated in FIG. 3. Referring to FIG. 5, the purpose of the Q-learning process is to update the Q-value through a specific number of iterations until the Q-value is no longer updated. The Q-learning process corresponds to a learning mode for learning the Q-value, and after learning is complete, prediction is performed in the inference part. In the first iteration, the Q-value is initialized to 0 for all state and action pairs to start the learning mode. From the next iteration onwards, the learning mode is terminated by repeating the following steps a specified number of times selected by the user.

[0106] 1. First state(s t In the second state(s) t+1Randomly select from the following possible states (S510).

[0107] 2. In that state, R(s t , a t Any operation a to the next possible state satisfying ) > 0 t is performed (S520).

[0108] 3. Next state S t+1 After reaching R(s t , a t ) is given as a reward (S530).

[0109] 4. Time difference TD t (s t , a t ) is calculated as in mathematical formula 1 (S540).

[0110] 5. The Q-value is updated by applying the Bellman equation (S550).

[0111] 6. Determine whether the Q-value is updated (S560)

[0112] 7. When the Q-value is updated (S560-YES), execute step S520 to perform an arbitrary action from the current state.

[0113] 8. If the Q-value is not updated (S560-NO), terminate reinforcement learning and create a training model (S570).

[0115] When the learning mode ends, the learned Q-values ​​are obtained to complete the learning model. Figure 6 is an example of a learned Q-value matrix. In inference mode, the fully learned model is used to predict the action to be taken to move from the starting patent node to the destination patent node. In Q-learning, a specific state s t Prediction is performed by executing the operation with the highest Q-value (see Equation 3). Therefore, the best path from the starting point to the destination, which can be called the main path, is proposed.

[0117] FIG. 7 is a diagram visualizing a network map based on a major technology path analysis (SPLC) for a specific technology field (photovoltaic technology). Referring to FIG. 7, 5,203 patents belonging to the field of photovoltaic technology are loaded from a patent data storage unit (110). A patent citation network map generation unit (120) visualizes the 5,203 patent data as a network map. The path analysis method applied in FIG. 7 is the result obtained by applying the SPLC (Search Path Link Count) technique. The patents shown in FIG. 7 are the result of visualizing by assigning layers according to the filing year.

[0119] Figure 8 is a diagram visualizing the main technology path with noise removed from the network map shown in Figure 7. Referring to Figure 8, it can be seen that the main technology path in solar power technology is visualized.

[0121] Figure 9 is a diagram visualizing major technology paths by Genetic Knowledge Persistence Measurement (GKPM) for a specific technology field (photovoltaic technology). Referring to Figure 9, common major paths that overlap with the results visualized in Figure 8 are highlighted in the same color.

[0123] FIG. 10 is a diagram visualizing a major technical path for a specific technical field (photovoltaic technology) by applying the method according to the present invention. Referring to FIG. 10, it can be seen that it is generally similar to the major path for photovoltaic technology shown in FIGs. 8 and 9. However, the result of the major path derived by the method according to the present invention exhibits a technical feature in that the path becomes new and shorter by introducing a new node (indicated by a red rectangle in FIG. 10) that replaces the common path obtained through the existing methodology. In addition, it can be seen that a shorter path is presented by skipping some nodes that were passed through in the existing path.

[0125] The components disclosed in the preferred embodiments described above may be implemented as a single integrated module depending on the specifications, uses, and functions, or a single component may be implemented as two or more individual modules. Explanation of the symbols

[0127] 100: Technology development trajectory derivation system 110: Patent data storage unit 120: Patent citation network map generation unit 130: User input unit 140: AI Learning Unit 150: Main Path Derivation Unit

Claims

Claim 1 A system for deriving a technology development trajectory based on patent citation network analysis using reinforcement learning, comprising: a patent data storage unit (110) for storing patent data collected from a database; a patent citation network map generation unit (120) for generating a patent citation network map using patent citation information among the patent data stored in the patent data storage unit (110); a user input unit (130) for receiving a user command to apply machine learning to the patent citation network map generated by the patent citation network map generation unit (120); an artificial intelligence learning unit (140) for learning a major path derivation algorithm for the corresponding patent citation network map according to the user command input from the user input unit (130); and a major path derivation unit (150) for deriving a technology development trajectory for the corresponding patent citation network map using a learning model learned by the artificial intelligence learning unit (140). Claim 2 A system for deriving a technology development trajectory based on patent citation network analysis using reinforcement learning, wherein, in claim 1, the patent citation network map generation unit (120) selects a patent belonging to a technology area according to a user-defined criterion among the patent data stored in the patent data storage unit (110), extracts patent citation information and patent identification information from the selected patent data, and generates a patent citation network map using the extracted patent identification information and patent citation information. Claim 3 In claim 2, the artificial intelligence learning unit (140) is characterized by defining a state by setting the patent identification information as a node, defining an action by setting the patent citation information as an edge between the nodes, and establishing a reinforcement learning environment by defining a reward matrix for the entire patent citation network map. Claim 4 In claim 3, the artificial intelligence learning unit (140) is characterized by defining the state of all nodes constituting the patent citation network map to be analyzed as the input of the learning model, defining the action for the interconnection between all nodes as the output of the learning model, defining the reward (r) that the agent of the learning model can obtain for the action (a) that can be taken for the input state(s), and defining a Q-value composed of numbers for all state-action pairs (s, a) consisting of all states and actions, a system for deriving a technology development trajectory based on patent citation network analysis using reinforcement learning. Claim 5 In claim 4, the artificial intelligence learning unit (140) has a specific time (t) and a specific state (s) for a specific node. t Randomly on a specific action (a t If ) is executed, the corresponding node becomes the next state (s t+1 Move to ), and the corresponding action (a t Reward R(s) for ) t , a t If ) is acquired, the corresponding action (a t A system for deriving a technology development trajectory based on patent citation network analysis utilizing reinforcement learning, characterized in that the Time Difference (TD) obtained due to ) is defined by the following formula. (Here, Is class Defined by the difference of, silver state s t Operation a in t It refers to the reward obtained by performing, is future state s t+1 It refers to the discounted value of the queue value of the best operation performed in, and silver state s t Actions performed in (a t (refers to the queue value of ). Claim 6 In claim 5, the artificial intelligence learning unit (140) is characterized by training an agent of a learning model to update a queue value according to the following formula, and repeating the learning a specific number of times until the queue value is no longer updated, in a system for deriving a technology development trajectory based on patent citation network analysis using reinforcement learning. (Here, is the learning rate and means a real value between 0 and 1, and The smaller the value, the smaller the update to the Q-value, and the slower the learning speed. Claim 7 In claim 6, the main path derivation unit (150) constructs a matrix of queue values ​​that have been learned by the artificial intelligence learning unit (140), and for a path moving from an arbitrary starting node to a destination node, a specific state (s t The operation of the highest queue value in ) (a t A system for deriving a technology development trajectory based on patent citation network analysis utilizing reinforcement learning, characterized by determining an action at a specific node according to the following formula to perform ). . Claim 8 A method for deriving a technology development trajectory using a system comprising a patent data storage unit (110), a citation network map generation unit (120), a user input unit (130), an artificial intelligence learning unit (140), and a main path derivation unit (150), comprising: a step of loading patent data for a specific technology field among patent data previously stored by the patent data storage unit (110); a step of generating a patent citation network map using patent identification information and patent citation information of the loaded patent data by the citation network map generation unit (120); a step of receiving a discount rate, a learning rate, and a number of iterations input by the user input unit (130); a step of performing reinforcement learning on the patent citation network map according to the previously input discount rate, learning rate, and number of iterations by the artificial intelligence learning unit (140); and a step of calculating a technology main path by applying a learning model calculated according to the reinforcement learning to the previously generated patent citation network map by the main path derivation unit (150); wherein the step of performing reinforcement learning involves performing an operation from a specific node to another node on the previously generated citation network map, and A method characterized by including the process of calculating a reward when another node is reached, calculating a time difference for the calculated reward, updating the current queue value using the calculated time difference and the existing queue value, and terminating the learning when the queue value is no longer updated.

Images