Lung nodule positioning model training method and device, lung nodule positioning device

Abstract

This invention provides a method and device for training a lung nodule localization model, as well as a lung nodule localization device. It is applied in the field of medical image processing. The method includes: training a lung nodule localization model using two-dimensional slice images containing lung nodules; the agent starts from the current state s... t Initially, based on the network structure, DQN outputs the current state s. t The corresponding action value and the corresponding action a are selected. t ; Perform action a t Obtain the corresponding reward r t and new state s t+1 And store the experience in the experience pool; randomly sample N batch The group calculates the time-difference objective value based on empirical calculations; the parameter w is updated based on the time-difference objective value. t Until the agent moves N steps steps The lung nodule localization model converges, completing this round of learning. After repeating the learning process M times on the two-dimensional slice image, the localization training ends, allowing the lung nodule localization model to output the optimal path for the agent to move to the lung nodule grid. This invention requires only a small amount of training data to obtain an accurate lung nodule localization model.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of medical image processing, in particular to a lung nodule positioning model training method and device and a lung nodule positioning device. BACKGROUND

[0002] Deep learning, as an important branch of machine learning, has shown great potential and broad application prospects in the field of medical image processing. Based on neural network structure, it automatically extracts features by learning a large amount of data, avoiding the tedious process of manually designing features, and significantly improving learning efficiency and accuracy. In particular, in disease diagnosis, treatment evaluation and drug discovery, deep learning technology has achieved remarkable results.

[0003] However, although deep learning has made great progress in the application of medical image processing, there are still some inherent limitations. The training of deep learning model is highly dependent on a large amount of labeled data. However, in the field of medical image processing, high-quality labeled data is often difficult to obtain. Deep learning model may have large prediction differences on data from different institutions or scanners. This is mainly because medical image data is often affected by many factors such as acquisition equipment, scanning parameters, patient individual differences, etc., resulting in large differences between data. The decision-making process of deep learning model often lacks transparency, which makes it difficult for doctors to understand and trust the prediction results of the model. The output results of deep learning model often lack intuitiveness and are difficult to be directly used for clinical diagnosis and decision-making. Doctors need to spend additional time and effort to understand the prediction results of the model, which increases the complexity and uncertainty of clinical application. SUMMARY

[0004] Therefore, the present application provides a lung nodule positioning model training method, which comprises:

[0005] The lung nodule positioning model is trained using two-dimensional slice images containing lung nodules, wherein the training learning period is M rounds, and for each round of learning, the agent of the lung nodule positioning model moves from the current state s t Start, N steps step sampling, in each step, the network structure DQN outputs the action value t of the current state s And use the greedy algorithm to select the corresponding action a t , wherein the action a t is randomly selected from the action space A with a probability of ε, and the action corresponding to the maximum value of the action value is selected with a probability of 1-ε, and the action a t is used to guide the movement direction of the agent on the two-dimensional slice image;

[0006] Perform the action a t Obtain the corresponding reward r t and new state s t+1 and the experience at the current time t Stored in the experience pool, wherein the reward r t According to the action a performed by the intelligent agent t The direction of movement and the action performed by the agent. t The determination is made based on whether the grid in which the nodule is located contains lung nodules;

[0007] When the experience pool capacity reaches a preset value, N samples are randomly taken from the experience pool. batch Group experience

[0008] Calculate each experience The time-series difference target value;

[0009] The parameters w of the lung nodule localization model are updated based on the temporal difference target value. t and update the status s t+1 Until the agent moves N steps steps The step or lung nodule localization model converges, thus completing this round of learning;

[0010] After M rounds of learning on the two-dimensional slice image, the localization training ends, so that the lung nodule localization model outputs the optimal path for the agent to move to the lung nodule grid.

[0011] Optionally, the reward r t Including the first reward r t1 Second reward r t2 Third reward r t3 Fourth reward r t4 And the fifth reward r t5 The first reward r t1 The agent moves towards the boundary and performs action a. t After touching the grid boundary, the second reward r t2 To keep the agent stationary while the agent performs action a t The grid in which the third reward r is located does not contain lung nodules. t3 To keep the agent stationary while the agent performs action a t The grid containing the fourth reward r contains lung nodules. t4 The fifth reward r is given when the agent moves to a grid containing lung nodules. t5 For the agent to move to a grid that does not contain lung nodules, where r t1 <r t2 <r t5< 0 < r t3 = r t4 .

[0012] Optionally, each experience 's timing difference target value is calculated in the following way:

[0013]

[0014] wherein TD target y j is the timing difference target Q value of the jth experience, r j is the reward in the jth experience, s j+1 is the new state after the jth experience, γ is the discount factor, and max a′ Q(s t+1 , a') is the maximum value of the action value function, that is, the maximum value of the Q value of all actions a' under the state s j+1 .

[0015] Optionally, the parameters w t of the lung nodule positioning model are updated according to the timing difference target value, including:

[0016] A loss function L batch is calculated according to all the timing difference target values;

[0017] A gradient descent algorithm is performed according to the loss function L batch to update the parameters w t of the lung nodule positioning model, obtaining the parameters w t+1 of the lung nodule positioning model under the next state s t+1 .

[0018] Optionally, a loss function L batch is calculated according to all the timing difference target values, including:

[0019]

[0020]

[0021] wherein represents the action value of the action corresponding to the action value function DQN at time t in the ith experience, represents the timing difference target value of the ith experience at time t.

[0022] Optionally, a gradient descent algorithm is performed according to the loss function L batch to update the parameters w t of the lung nodule positioning model, including:

[0023] The loss function Lbatch gradient g t :

[0024]

[0025] wherein, is a symbol of partial derivative, and w is a parameter of the lung nodule positioning model;

[0026] The parameter w of the lung nodule positioning model is updated in the following manner t to obtain a next state s t+1 The parameter w of the lung nodule positioning model is updated in the following manner t+1

[0027] w t+1 =w t -α·g t

[0028] wherein, a represents a learning rate.

[0029] Optionally, the network structure DQN outputs a current state s t corresponding action value is:

[0030]

[0031] wherein, represents an action value corresponding to the i-th action at time t, i = 1, 2, 3, 4, 5.

[0032] Optionally, the corresponding action a t is selected by using the following greedy algorithm:

[0033]

[0034] wherein, ε represents an exploration rate, an initial value is a set value, and after completing the current round of learning, the exploration rate is attenuated at a preset rate, 0 < ε < 1.

[0035] Optionally, before training the lung nodule positioning model by using the two-dimensional slice image containing the lung nodule, the method further comprises:

[0036] obtaining a two-dimensional slice image containing a lung nodule;

[0037] resampling the two-dimensional slice image containing the lung nodule;

[0038] dividing a resampled image space into a plurality of grid units with a preset pixel interval, and each grid unit is a basic unit for the agent to move and search on the two-dimensional slice image.

[0039] Optionally, the network structure DQN comprises a convolution Elu layer, a full connection layer and an output layer.

[0040] The second aspect of the present application provides a lung nodule positioning model training device, comprising a processor and a memory connected to the processor; wherein the memory stores instructions executable by the processor, and the instructions are executed by the processor to enable the processor to perform the lung nodule positioning model training method described above.

[0041] The third aspect of the present application provides a lung nodule positioning device, comprising a processor and a memory connected to the processor; wherein the memory stores instructions executable by the processor, and the instructions are executed by the processor to enable the processor to use the lung nodule positioning model trained by the method to perform lung nodule positioning on lung images,

[0042] The lung nodule positioning model training method of the present application uses only a small amount of lung CT slices as training data, and each two-dimensional slice contains a lung nodule. In the framework of the grid world, accurate lung nodule positioning can be obtained by training DQN using only the original image and its corresponding nodule mask. Here, a small amount means that the amount of training samples required by the method is at least an order of magnitude lower than the amount of training samples required by traditional radiology artificial intelligence. Since current radiology artificial intelligence is dominated by supervised deep learning, this method relies on a large amount of annotated data. Given that current radiology artificial intelligence mainly relies on supervised deep learning, the method usually requires hundreds or even thousands of annotated images to achieve high performance. In contrast, the present application significantly reduces the data requirements and provides an intuitive and efficient operating environment for medical image analysis in the framework of the grid world.

[0043] The lung nodule positioning device of the present application can use the trained lung nodule positioning model to accurately identify lung nodules in lung images, and use the lung nodule positioning model to quickly find the optimal path with the least number of steps and the maximum comprehensive reward to locate the lung nodule. It can realize automatic analysis and processing of lung images, and complete the positioning of lung nodules without manual intervention, which further improves the efficiency and convenience of diagnosis. BRIEF DESCRIPTION OF DRAWINGS

[0044] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the drawings needed to be used in the specific embodiments or prior art description. Obviously, the drawings described below are some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0045] Figure 1 It is a flowchart of the lung nodule positioning model training method in the embodiments of the present application.

[0046] Figure 2 A network structure diagram of the DQN in the embodiment of the present application;

[0047] Figure 3 A grid result diagram of two-dimensional slice image division in the embodiment of the present application. DETAILED DESCRIPTION

[0048] The technical solutions of the present application will be described clearly and completely below in conjunction with the drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0049] In the description of the present application, it should be noted that the orientations or positional relationships indicated by the terms “center”, “upper”, “lower”, “left”, “right”, “vertical”, “horizontal”, “inner”, “outer” and the like are based on the orientations or positional relationships shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and therefore cannot be understood as indicating or implying that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application. In addition, the terms “first”, “second”, “third” are only for the purpose of description, and cannot be understood as indicating or implying relative importance.

[0050] In the description of the present application, it should be noted that unless otherwise explicitly specified and limited, the terms “mounting”, “connection”, “connecting” should be understood broadly, for example, it can be fixed connection, or detachable connection, or integrally connected; it can be mechanical connection, or electrical connection; it can be directly connected, or indirectly connected through an intermediate medium, or the internal communication of two elements, it can be wireless connection, or wired connection. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0051] In addition, the technical features involved in the different embodiments of the present application described below can be combined with each other as long as there is no conflict.

[0052] As Figure 1 shown, the embodiment of the present application provides a lung nodule positioning model training method, which is executed by a computer or a server and the like electronic device, wherein the training data needs to be preprocessed and DQN network training is performed before the lung nodule positioning model training.

[0053] Specifically, the preprocessing of the training data includes:

[0054] Obtaining training data, the training data being a small number of lung CT slice images, and each being a two-dimensional slice image containing a lung nodule.

[0055] Resampling the two-dimensional slice image containing the lung nodule.

[0056] Dividing the resampled image space into a plurality of grid cells with a preset pixel interval, each grid cell being a basic unit for the agent to move and search on the two-dimensional slice image. For example, the space is divided into grids with an interval of 32 pixels, so that the agent occupies a position of a 32*32 pixel block, and the specific division is as shown in FIG. a. Figure 3 In FIG. a, the upper left corner of the entire two-dimensional slice image is divided into 256 grids, and the position of a grid occupied by the agent is shown.

[0057] The DQN network training specifically includes:

[0058] Defining the two-dimensional slice image containing the lung nodule as a training learning environment, and the size, state s, action space A, strategy π, reward, network structure DQN and experience pool of the agent required for training learning. Among them, the environment is a two-dimensional slice image containing a lung nodule; the agent is a point moving in the two-dimensional slice image; the state s is the position of the agent in the two-dimensional slice image, and the agent is set to the initial state when it is at the upper left corner of the pixel block, for example, for a 512*512 image, there are states, that is, the state space s∈N 256 ; the action space A contains a plurality of actions a for guiding the moving direction of the agent, such as up, right, down, left and staying in place, and the action space A∈N 5 , and the specific definition is as follows: The strategy π is the action basis according to the state s, and the goal of the DQN is to find an optimal strategy π * that maximizes the sum of the rewards obtained, and in the present application, the best strategy is to make the agent move to the nodule as quickly as possible and stay there to mark / predict the nodule; the reward is the feedback given to the action of the agent.

[0059] The network structure DQN is used to output the action value corresponding to the strategy π and the state, and the DQN uses a CNN (which can be represented by Q π (s, a; w)) to approximate the state value function Q π (s, a). The network structure DQN includes a convolution Elu layer, a fully connected layer and an output layer, and the specific network structure is as shown in FIG. b. Figure 2, the network adopts four successive convolution Conv+Elu (exponential linear unit) operations, such as Conv1, Conv2, Conv3, and Conv4 in the figure, the convolution kernel size is 3, the number of convolution kernels is 32, the step is 2, and the padding operation is used to ensure that the feature map size remains unchanged after convolution, and the output of the last convolution layer is expanded into a 1*512 vector into the full connection layer FC1, followed by two full connection layers FC2 and FC3, the dimensions are 256 and 128 respectively, and finally a 5-dimensional result Q(s, a) is output, which represents the action value of each action in the action space A. By calculating the action value corresponding to different actions, the action with the maximum value is generally selected for execution. In order to seek the balance between exploration and exploitation, effectively learn and optimize the strategy, at time t, the action a t is selected according to the ε-greedy greedy algorithm π Q (s,a) is:

[0060]

[0061] Q π (s,a) is the action value function, defined as the expectation of the discounted return, which is a function of state-action pair (s, a) and related to the policy π, the greater the action value, the better the current action; R t is the discounted return starting from time t (i.e. the weighted value of the return at times t, t+1, t+2, …, t+k, …, R t = r t + γr t+1 + γ 2 r t+2 + …), γ is the discount factor, which determines the contribution of future rewards to the discounted return; the present invention can be set to 0.99; s t is the state at time t, a t is the action taken at time t; E π operation is the expectation of the discounted return R t obtained by executing action a t in a given state s t ; t+k is any time after t, is the combined expression of R t+1 = r 2 + γr t+2 + γ t r t + ….

[0062] The experience pool is used to place continuous experiences Experience includes state s t , action at , reward r t , new state s t+1 , state s at time t t , take action a t , receive reward r and new state s t , store the tuple in a memory buffer t+1 , called a transition, denoted as To make full use of each set of experience and remove the adverse effects of correlation between consecutive states, the present application uses experience replay technology. Specifically: for each consecutive time, consecutive can be stacked together to form a transition matrix Set the maximum number of rows of the transition matrix to N memory , which represents the replay memory buffer size, which allows DQN to sample and learn from past experiences. When the buffer is full, the transition stored in the buffer first is deleted to ensure that new is stored in the buffer. It should be noted that N memory is a tunable hyperparameter. On the one hand, the larger N memory , the larger the buffer, the more transitions can be used for training, and the agent can learn more about the environment. On the other hand, the larger N memory , the more memory it will occupy, and the probability of each set of transitions being selected will decrease, so N memory needs to be set reasonably according to the requirements.

[0063] Further, the experience pool is initialized with a capacity of N memory , and the parameters w of the DQN network in the lung nodule positioning model are randomly initialized. Then the training begins, and the training process specifically includes:

[0064] S1, use the two-dimensional slice image containing the lung nodule to train the lung nodule positioning model, wherein the training learning period is M rounds, and for each round of learning, the lung nodule positioning model agent starts from the current state s t , and performs N steps step sampling, in each step, the network structure DQN outputs the action value t corresponding to the current state s , and the greedy algorithm is used to select the corresponding action a t , wherein the action a t is randomly selected from the action space A with a probability of ε, and the action corresponding to the maximum value in the action value is selected with a probability of 1-ε, and the action at Used to guide the movement direction of the agent on a two-dimensional slice image.

[0065] S2, Execute action a t Obtain the corresponding reward r t and new state s t+1 and the experience at the current time t Stored in the experience pool, where the reward r t Based on the action a performed by the agent t The direction of movement and the action performed by the agent. t The determination is made by whether the grid in which the nodule is located contains lung nodules.

[0066] S3, when the experience pool capacity reaches a preset value, randomly sample N from the experience pool. bch Group experience

[0067] S4, calculate each experience The time-series difference target value.

[0068] S5, Update the parameters w of the lung nodule localization model based on the temporal difference target value. t and update the status s t+1 Until the agent moves N steps steps The learning cycle is complete when the step or lung nodule localization model converges.

[0069] S6: After repeating the learning process on the 2D slice image for M rounds, the localization training ends, allowing the lung nodule localization model to output the optimal path for the agent to move to the lung nodule grid. The optimal path is the path from the agent's initial position to any position in the grid containing the lung nodule, with the fewest steps required and the highest overall reward.

[0070] Specifically, a complete training process on a small number of CT slices requires, for example, M episodes. For each episode, the agent can sample N... steps Step, in each step, randomly select from the transition matrix Medium sampling N batch Group transitions. Update DQN using a single-step temporal difference algorithm. The initial state is s1, as follows: Figure 3 Starting from the top left corner of the two-dimensional slice image, at a certain position at time t, i.e., in state s t Next, select action a according to the greedy strategy. t Then execute action a. t Observe the environment and receive an immediate reward. t and new state s t+1 Then store this set of values ​​in the transition. and put into the experience pool ; when the data in the experience pool is sufficient (for example, reaches N batch groups), randomly sample N batch groups of transitions from the experience pool , and calculate the timing difference target value (TD target) of each group to update the model parameters w t of the DQN network in the lung nodule positioning model according to the timing difference target value, and update the new state s t+1 of the agent. If the data in the experience pool is not sufficient, the above steps for model parameter updating are not required, and only the state of the agent needs to be updated. Until the state of the agent is a terminal state (i.e., the lung nodule is positioned) or the number of movement steps of the agent reaches N steps or the lung nodule positioning model converges, an episode is completed, and then the above steps S1-S5 are repeated for M rounds of learning, and the positioning training is ended, so that the lung nodule positioning model outputs the optimal path for the agent to move to the lung nodule grid.

[0071] The present application only needs to use a small amount of lung CT slices as training data and each two-dimensional slice contains a lung nodule, and in the framework of the grid world, accurate lung nodule positioning can be obtained by training DQN only using the original image and the corresponding nodule mask, wherein the small amount refers to at least one order of magnitude lower than the number of training samples required by traditional radiology artificial intelligence. Since the current radiology artificial intelligence is always dominated by supervised deep learning, this method relies on a large amount of annotated data. Given that the current radiology artificial intelligence mainly relies on supervised deep learning, this method usually needs hundreds or even thousands of annotated images to achieve high performance. In contrast, the present application significantly reduces the data requirement and provides an intuitive and efficient operating environment for medical image analysis in the framework of the grid world.

[0072] Further, the reward r t in step S2 includes a first reward r t1 , a second reward r t2 , a third reward r t3 , a fourth reward r t4 , and a fifth reward r t5 , the first reward r t1 is that the agent moves in the boundary direction, and the agent touches the grid boundary after performing the action a t , the second reward r t2 is that the agent remains in place, and the grid where the agent is located does not contain a lung nodule after performing the action a t , the third reward r t3To keep the agent stationary while the agent performs action a t The grid containing the lung nodules, the fourth reward r t4 For the agent to move to a grid containing lung nodules, the fifth reward r t5 For the agent to move to a grid that does not contain lung nodules, where r t1 <r t2 <r t5 <0<r t3 =r t4 .

[0073] For example, r t1 =-4, r t2 =-2, r t5 =-0.5, r t3 =r t4 =+1. For example... Figure 3 As shown in Figure a, if the agent in the top-left grid moves towards the grid's world boundary, its reward is -4. In Figure b, if the agent moves up, down, left, or right, the reward for each of these four actions is -0.5, since the resulting grid does not contain lung nodules. In Figure b, if the agent in the bottom-right grid remains stationary and this grid does not contain lung nodules, its reward is -2. In Figure c, if the agent moves to the right, it will move to a grid containing lung nodules, and its reward is +1. In Figure d, if the agent remains stationary and this grid contains lung nodules, its reward is +1.

[0074] In one embodiment, step S4 calculates each experience using the following method. The time-series difference target value:

[0075]

[0076] Among them, TD target y j Let r be the temporal difference objective Q value of the j-th experience. j For the reward in the j-th experience, s j+1 Let γ be the new state after the j-th experience, and γ be the discount factor. a′ Q(s t+1 (a′) represents the maximum value of the action value function, i.e., in state s. j+1 Below, for all actions a ′ The Q value is taken as the maximum value.

[0077] Specifically, if the state s of the j-th experience in the experience pool j+1 If the final state is reached, the temporal difference objective value is determined solely by the current reward (immediate return); otherwise, the temporal difference objective value is determined by both the current reward and the new state s. t+1The maximum Q-value among all actions is determined (both the immediate reward and the maximum potential future reward). This formula combines the immediate reward and the maximum potential future reward (weighted by a discount factor) to calculate a more accurate temporal difference target value to guide policy updates during the learning process.

[0078] Furthermore, in step S5, the parameters w of the lung nodule localization model are updated based on the temporal difference target value. t Specifically, it includes:

[0079] The loss function L is calculated based on all the temporal difference target values. batch ;

[0080] Based on the loss function L batch The gradient descent algorithm is used to update the parameters w of the lung nodule localization model. t To obtain the next state s t+1 Parameters w of the lower lung nodule localization model t+1 .

[0081] Specifically, the loss function L is calculated based on all the temporal difference target values. batch ,include:

[0082]

[0083] in, This represents the action value of the DQN network structure at time t, corresponding to the i-th experience. Let represent the temporal difference target value of the i-th experience at time t.

[0084] The loss function L is applied in the following manner. batch Find the gradient g t :

[0085]

[0086] in, The sign represents the partial derivative, and w represents the parameters of the lung nodule localization model.

[0087] The parameters w of the lung nodule localization model are updated using the following method. t To obtain the next state s t+1 The parameters w of the lung nodule localization model described below t+1

[0088] w t+1 =w t -α·g t

[0089] Where α represents the learning rate.

[0090] In one embodiment, step S1 outputs the current state s based on the DQN network structure. t Corresponding action value

[0091]

[0092] in, Let represent the action value corresponding to the i-th action at time t, where i = 1, 2, 3, 4, 5.

[0093] Specifically, It is the output Q(s,a) of the DQN network structure.

[0094] Furthermore, in step S1, the corresponding action a is selected using the following greedy algorithm. t :

[0095]

[0096] Where ε represents the exploration rate, the initial value is a set value, and it decays at a preset rate after the completion of this round of learning, 0<ε<1.

[0097] Specifically, ε is a positive number less than 1. During the decision-making process, there is an ε probability of randomly selecting one action from |A| actions in the action space. If an action is randomly selected from |A| actions in the action space, the probability of each action being selected is ε / |A|, where |A| is the number of actions. The remaining actions have a probability of 1-ε. The action corresponding to the largest Q value (that is, the optimal action at that moment) has a probability of being selected as 1-ε+ε / |A|.

[0098] The key to the ε-greedy algorithm lies in gradually decreasing ε during the learning process. This allows the agent to explore extensively in the early stages of training, gradually increasing the opportunities for exploitation as training progresses until it converges to a smaller ε. This helps the agent gradually optimize its policy during learning, while simultaneously exploring the environment to discover better strategies. In this invention, the initial value of ε is set to 0.7, and for each episode, the parameter ε is preset to, for example, 1×10-1. -4 The rate decays until ε min =1×10 -4 By gradually decreasing ε, the agent can gradually converge to a stable policy while maintaining a certain level of exploratory ability. This helps the algorithm exhibit stronger adaptability and robustness in different environments and tasks, improving learning efficiency and accuracy.

[0099] Compared to the traditional method of selecting actions based on the maximum value of Q, this invention selects the corresponding action a using the method described above.t There is a certain probability to select an action that is not the Q maximum value, which increases the exploratory of the entire grid space and improves the opportunity to find the optimal path to locate the lung nodule.

[0100] The embodiment of the present application also provides a lung nodule positioning device for positioning a lung nodule in a lung image according to the trained lung nodule positioning model.

[0101] The device can use the trained lung nodule positioning model to accurately identify the lung nodule in the lung image, and use the lung nodule positioning model to quickly find the optimal path with the least number of steps and the maximum comprehensive reward to locate the lung nodule. The automatic analysis and processing of the lung image can be realized, and the positioning of the lung nodule can be completed without manual intervention, which further improves the efficiency and convenience of diagnosis.

[0102] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0103] The present application is described with reference to flowcharts and / or block diagrams of the method, device (system), and computer program product according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one flow or multiple flows and / or blocks Figure 1 The means for performing the functions specified in one block or multiple blocks.

[0104] These computer program instructions can also be stored in a computer readable storage medium that can guide the computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer readable storage medium produce a product including instruction means, which implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one flow or multiple flows and / or blocks Figure 1 The means for performing the functions specified in one block or multiple blocks.

[0105] These computer program instructions can also be loaded into a computer or other programmable data processing devices, so that a series of operational steps are performed on the computer or other programmable data processing devices to generate computer-implemented processes, thus the instructions executed on the computer or other programmable data processing devices provide processes for implementing the functions specified in the flowchart Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0106] Obviously, the above embodiments are only examples for clearly illustrating the present application, and are not intended to limit the embodiments. Based on the above description, other different forms of changes or variations can also be made by those skilled in the art. Here, all the embodiments are not required to be exhausted, and the obvious changes or variations derived therefrom are still within the protection scope of the present application.

Claims

1. A method for training a lung nodule localization model, characterized in that, include: A lung nodule localization model is trained using two-dimensional slice images containing lung nodules, with a training cycle of M rounds. For each round, the agent of the lung nodule localization model starts from the current state s. t Begin, proceed with N steps The sampling is performed step by step, and in each step, the current state s is output according to the network structure DQN. t Corresponding action value And use a greedy algorithm to select the corresponding action a t Among them, action a t It means randomly selecting an action from the action space A with probability ε, and selecting the action value with probability 1-ε. The action corresponding to the maximum value in the middle, the action a t Used to guide the movement direction of the intelligent agent on a two-dimensional slice image; Perform the action a t Obtain the corresponding reward r t and new state t+1 and the experience at the current time t Stored in the experience pool, wherein the reward r t According to the action a performed by the intelligent agent t The direction of movement and the action performed by the agent. t The determination is made based on whether the grid in which the nodule is located contains lung nodules; When the experience pool capacity reaches a preset value, N samples are randomly taken from the experience pool. batch Group experience Calculate each experience The time-series difference target value; The parameters w of the lung nodule localization model are updated based on the temporal difference target value. t and update the status s t+1 Until the agent moves N steps steps The step or lung nodule localization model converges, thus completing this round of learning; After M rounds of learning on the two-dimensional slice image, the localization training ends, so that the lung nodule localization model outputs the optimal path for the agent to move to the lung nodule grid.

2. The method according to claim 1, characterized in that, The reward r t Including the first reward r t1 Second reward r t2 Third reward r t3 Fourth reward r t4 And the fifth reward r t5 The first reward r t1 The agent moves towards the boundary and performs action a. t After touching the grid boundary, the second reward r t2 To keep the agent stationary while the agent performs action a t The grid in which the third reward r is located does not contain lung nodules. t3 To keep the agent stationary while the agent performs action a t The grid containing the fourth reward r contains lung nodules. t4 The fifth reward r is given when the agent moves to a grid containing lung nodules. t5 For the agent to move to a grid that does not contain lung nodules, where r t1 <r t2 <r t5 <0<r t3 =r t4 .

3. The method according to claim 1, characterized in that, Calculate each experience using the following method The time-series difference target value: Among them, TDtarget y j Let r be the temporal difference objective Q value of the j-th experience. j For the reward in the j-th experience, s j+1 Let γ be the new state after the j-th experience, and γ be the discount factor. a′ Q(s t+1 (a′) represents the maximum value of the action value function, i.e., in state s. j+1 Under these conditions, the Q value of all actions a′ is taken as the maximum value.

4. The method according to claim 1, characterized in that, The parameters w of the lung nodule localization model are updated based on the temporal difference target value. t ,include: The loss function L is calculated based on all the aforementioned temporal difference target values. batch ; According to the loss function L batch The gradient descent algorithm is used to update the parameters w of the lung nodule localization model. t To obtain the next state s t+1 The parameters w of the lung nodule localization model described below t+1 .

5. The method according to claim 4, characterized in that, The loss function L is calculated based on all the aforementioned temporal difference target values. batch ,include: in, This represents the action value corresponding to the action in the i-th experience at time t in the DQN network structure. Let represent the temporal difference target value of the i-th experience at time t.

6. The method according to claim 5, characterized in that, According to the loss function L batch The gradient descent algorithm is used to update the parameters w of the lung nodule localization model. t , include: The loss function L is applied in the following manner. batch Find the gradient g t : in, is the sign for partial derivative, and w represents the parameters of the lung nodule localization model. The parameters w of the lung nodule localization model are updated using the following method. t To obtain the next state s t+1 The parameters w of the lung nodule localization model described below t+1 w t+1 =w t -α·g t Where α represents the learning rate.

7. The method according to claim 1, characterized in that, Based on the network structure, DQN outputs the current state s. t Corresponding action value in, Let represent the action value corresponding to the i-th action at time t, where i = 1, 2, 3, 4, 5.

8. The method according to claim 1, characterized in that, Use the following greedy algorithm to select the corresponding action a t : Where ε represents the probability of selecting an action, with an initial value set and decreasing at a preset rate after the current learning round is completed, 0 < ε < 1.

9. The method according to claim 1, characterized in that, Before training the lung nodule localization model using two-dimensional slice images containing lung nodules, the following steps are also included: Obtain two-dimensional slice images containing lung nodules; Resampling of two-dimensional slice images containing lung nodules; The resampled image space is divided into several grid units with preset pixel intervals. Each grid unit is the basic unit for the agent to move and search on the two-dimensional slice image.

10. The method according to claim 1, characterized in that, The network structure DQN includes a convolutional Elu layer, a fully connected layer, and an output layer.

11. A training device for a lung nodule localization model, characterized in that, include: A processor and a memory connected to the processor; wherein the memory stores instructions executable by the processor, the instructions being executed by the processor to cause the processor to perform the lung nodule localization model training method as described in any one of claims 1-10.

12. A lung nodule localization device, characterized in that, include: A processor and a memory connected to the processor; wherein the memory stores instructions executable by the processor, the instructions being executed by the processor to cause the processor to perform lung nodule localization on a lung image using a lung nodule localization model trained by the method as described in any one of claims 1-10.

Images