Pulmonary nodule positioning model training method and device and pulmonary nodule positioning device

Abstract

The invention provides a pulmonary nodule positioning model training method and device and a pulmonary nodule positioning device. The method is applied to medical image processing. The method comprises the following steps: training a pulmonary nodule positioning model by using a two-dimensional slice image containing a pulmonary nodule, starting from a current state st, outputting an action value # imgabs0 # corresponding to the current state st by an intelligent agent according to a network structure DQN, and selecting a corresponding action at; the action at is executed, a corresponding reward rt and a new state st + 1 are obtained, and the experience # imgabs1 # is stored in an experience pool; randomly sampling experience # imgabs2 # of the Nbatch group to calculate a time sequence difference target value; the parameter wt is updated according to the time sequence difference target value until the number of moving steps of the intelligent agent reaches Nsteps or the pulmonary nodule positioning model converges, and the round of learning is completed; and after M rounds of learning are repeatedly carried out on the two-dimensional slice image, positioning training is ended, so that the pulmonary nodule positioning model outputs an optimal path for the intelligent agent to move to the pulmonary nodule grid. According to the method, the accurate pulmonary nodule positioning model can be obtained only through a small amount of training data.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing, and in particular to a lung nodule positioning model training method and equipment, and a lung nodule positioning equipment. Background Art

[0002] Deep learning, as an important branch of machine learning, has shown great potential and broad application prospects in the field of medical image processing. Based on the neural network structure, it automatically extracts features by learning a large amount of data, avoiding the tedious process of manually designing features, and significantly improving learning efficiency and accuracy. In particular, deep learning technology has achieved remarkable results in disease diagnosis, treatment evaluation, and drug discovery.

[0003] However, despite the great progress made in the application of deep learning in the field of medical image processing, there are still some inherent limitations. The training of deep learning models is highly dependent on a large amount of annotated data. However, in the field of medical image processing, high-quality annotated data is often difficult to obtain. Deep learning models may have large prediction differences on data from different institutions or scanners. This is mainly because medical image data is often affected by multiple factors such as acquisition equipment, scanning parameters, and individual differences among patients, resulting in large differences between data. The decision-making process of deep learning models often lacks transparency, which makes it difficult for doctors to understand and trust the prediction results of the model. The output results of deep learning models often lack intuitiveness and are difficult to use directly for clinical diagnosis and decision-making. Doctors need to spend extra time and effort to understand the prediction results of the model, which increases the complexity and uncertainty of clinical applications. Summary of the invention

[0004] In view of this, the present invention provides a lung nodule localization model training method, comprising:

[0005] The pulmonary nodule localization model is trained using two-dimensional slice images containing pulmonary nodules, wherein the training learning cycle is M rounds. For each round of learning, the agent of the pulmonary nodule localization model changes from the current state s t Start, proceed to N steps Step sampling, in each step, according to the network structure DQN output current state s t The corresponding action value And use the greedy algorithm to select the corresponding action a t , where action a t is to randomly select an action in the action space A with probability ε and select the action value with probability 1-ε The action corresponding to the maximum value in t Used to guide the moving direction of the intelligent agent on the two-dimensional slice image;

[0006] Perform the action a t , get the corresponding reward r t and the new state s t+1 , and the experience at the current time t Stored into the experience pool, where the reward r t According to the agent to perform the action a t The moving direction and the agent's action a t The latter is determined by whether the grid contains lung nodules;

[0007] When the experience pool capacity reaches a preset value, N random samples are sampled from the experience pool. batch Group Experience

[0008] Calculate each experience The target value of the timing difference of

[0009] Update the parameter w of the lung nodule positioning model according to the time series difference target value t , and update the state s t+1 , until the agent moves N steps steps The step or lung nodule localization model converges, and this round of learning is completed;

[0010] After repeating M rounds of learning on the two-dimensional slice image, the positioning training is terminated, so that the lung nodule positioning model outputs the optimal path for the intelligent agent to move to the lung nodule grid.

[0011] Optionally, the reward r t Including the first reward t1 , Second Reward t2 、Third Reward t3 , the fourth reward r t4 and the fifth reward r t5 , the first reward r t1 The agent moves toward the boundary and performs action a t After touching the grid boundary, the second reward r t2 The agent remains stationary and performs action a t After the grid contains no lung nodules, the third reward r t3 The agent remains stationary and performs action a t After the grid contains lung nodules, the fourth reward r t4 For the agent to move to the grid containing the lung nodules, the fifth reward r t5 The agent moves to the grid that does not contain lung nodules, where r t1 <r t2 <r t5<0<r t3 =r t4 .

[0012] Optionally, calculate each experience as follows The timing difference target value of:

[0013]

[0014] Among them, TD target y j is the temporal difference target Q value of the jth experience, r j is the reward in the jth experience, s j+1 is the new state after the jth experience, γ is the discount factor, max a′ Q(s t+1 ,a′) is the maximum value of the action value function, that is, in state s j+1 Under this condition, the Q value of all actions a′ is maximized.

[0015] Optionally, the parameter w of the pulmonary nodule localization model is updated according to the time series difference target value. t ,include:

[0016] Calculate the loss function L based on all the time series difference target values batch ;

[0017] According to the loss function L batch Execute the gradient descent algorithm to update the parameters w of the pulmonary nodule localization model t , get the next state s t+1 The parameters w of the lung nodule localization model are as follows: t+1 .

[0018] Optionally, the loss function L is calculated based on all the time series difference target values batch ,include:

[0019]

[0020]

[0021] in, It represents the action value corresponding to the action in the i-th experience of the network structure DQN at time t, Represents the time-series difference target value of the i-th experience at time t.

[0022] Optionally, according to the loss function L batch Execute the gradient descent algorithm to update the parameters w of the pulmonary nodule localization model t ,include:

[0023] The loss function L is analyzed in the following way:batch Find the gradient g t :

[0024]

[0025] in, is the symbol for partial derivative, w is the parameter of the pulmonary nodule localization model;

[0026] The parameter w of the lung nodule localization model is updated in the following way: t , get the next state s t+1 The parameters w of the lung nodule localization model are as follows: t+1

[0027] w t+1 =w t -α·g t

[0028] Among them, α represents the learning rate.

[0029] Optionally, output the current state s according to the network structure DQN t The corresponding action value for:

[0030]

[0031] in, It represents the action value corresponding to the i-th action at time t, i=1,2,3,4,5.

[0032] Optionally, use the following greedy algorithm to select the corresponding action a t :

[0033]

[0034] Among them, ε represents the exploration rate, the initial value is the set value, and it decays at a preset rate after completing this round of learning, 0<ε<1.

[0035] Optionally, before training the lung nodule localization model using the two-dimensional slice image containing the lung nodules, the method further includes:

[0036] Acquire two-dimensional slice images containing lung nodules;

[0037] Resampling the 2D slice images containing lung nodules;

[0038] The resampled image space is divided into a number of grid units with preset pixel intervals, and each grid unit is a basic unit for the intelligent agent to move and search on the two-dimensional slice image.

[0039] Optionally, the network structure DQN includes a convolutional Elu layer, a fully connected layer and an output layer.

[0040] The second aspect of the present invention provides a lung nodule localization model training device, which includes: a processor and a memory connected to the processor; wherein the memory stores instructions executable by the processor, and the instructions are executed by the processor so that the processor executes the above-mentioned lung nodule localization model training method.

[0041] A third aspect of the present invention provides a lung nodule localization device, the device comprising: a processor and a memory connected to the processor; wherein the memory stores instructions executable by the processor, the instructions are executed by the processor, so that the processor uses the lung nodule localization model trained by the method to perform lung nodule localization on a lung image,

[0042] The lung nodule localization model training method of the present invention uses only a small number of lung CT slices as training data and each two-dimensional slice contains a lung nodule. In the framework of the grid world, accurate lung nodule localization can be obtained by training DQN using only the original image and its corresponding nodule mask, wherein a small number refers to the number of training samples required by the method compared to the number of training samples required for traditional radiology artificial intelligence. The number of training samples required is at least one order of magnitude lower. Since the current radiology artificial intelligence has been dominated by supervised deep learning, this method relies on a large amount of annotated data. Given that current radiology artificial intelligence mainly relies on supervised deep learning, this method usually requires hundreds or even thousands of annotated images to achieve high performance. In contrast, the present invention significantly reduces data requirements and provides an intuitive and efficient operating environment for medical image analysis in the grid world framework.

[0043] The lung nodule positioning device of the present invention can use the trained lung nodule positioning model to accurately identify the lung nodules in the lung image, and use the lung nodule positioning model to quickly find the optimal path with the least steps and the greatest comprehensive reward to locate the lung nodules. It can realize automatic analysis and processing of lung images, and complete the positioning of lung nodules without manual intervention, which further improves the efficiency and convenience of diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 Schematic diagram of the process of the lung nodule localization model training method in an embodiment of the present invention;

[0046] Figure 2 is a network structure diagram of a DQN in an embodiment of the present invention;

[0047] Figure 3 This is a grid result diagram of the two-dimensional slice image division in an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0050] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, it can also be the internal connection of two components, it can be a wireless connection, or it can be a wired connection. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0051] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0052] like Figure 1 As shown, an embodiment of the present invention provides a lung nodule localization model training method, which is executed by an electronic device such as a computer or a server, wherein the training data needs to be preprocessed and the DQN network training needs to be performed before the lung nodule localization model training.

[0053] Specifically, preprocessing the training data includes:

[0054] The training data is obtained, which is a small number of lung CT slice images, and each of the slice images is a two-dimensional slice image containing a lung nodule.

[0055] Resample the 2D slice images containing lung nodules.

[0056] The resampled image space is divided into a number of grid units with preset pixel intervals, and each grid unit is the basic unit for the agent to move and search on the two-dimensional slice image. For example, the space is divided into grids with an interval of 32 pixels, so that the agent occupies a 32×32 pixel block. Figure 3 The entire two-dimensional slice image is divided into 256 grids, and the upper left corner of Figure a is the position of a grid occupied by the intelligent agent.

[0057] DQN network training specifically includes:

[0058] Define the two-dimensional slice image containing lung nodules as the training learning environment, as well as the size of the agent, state s, action space A, strategy π, reward, network structure DQN and experience pool required for training learning. Among them, the environment is a two-dimensional slice image containing lung nodules; the agent is a point moving in the two-dimensional slice image; the state s is the position of the agent in the two-dimensional slice image, and the position of the pixel block in the upper left corner of the agent is set as the initial state. For example, for a 512*512 image, there is states, that is, the state space s∈N 256 ; The action space A contains multiple actions a, which are used to guide the movement direction of the agent, such as up, right, down, left and stay still. The action space A∈N 5 , the specific definition is: Strategy π is the action taken according to state s. The goal of DQN is to find an optimal strategy π * , so that the sum of the rewards obtained is maximized. In the present invention, the optimal strategy is to make the agent move to the node as quickly as possible and stay there to mark / predict the node; the reward is the feedback given to the action taken by the agent.

[0059] The network structure DQN is used to output the action value corresponding to the strategy π and the state. DQN uses CNN (CNN can use Q π (s, a; w) represents) to approximate the state value function Q π (s,a). The network structure DQN includes convolutional Elu layer, fully connected layer and output layer. The specific network structure is as follows Figure 2The network uses 4 consecutive convolution Conv+Elu (exponential linear unit) operations, as shown in the figure Conv1, Conv2, Conv3, Conv4, the convolution kernel size is 3, the number of convolution kernels is 32, the step size is 2, and the padding operation is used to ensure that the size of the feature map after convolution remains unchanged. The output of the last convolution layer is expanded into a 1*512 vector and enters the fully connected layer FC1, followed by two fully connected layers FC2 and FC3, with dimensions of 256 and 128 respectively, and finally outputs a 5-dimensional result Q(s,a), which represents the action value of each action in the action space A. By calculating the action values ​​corresponding to different actions, the action with the greatest value is generally selected for execution. In order to seek a balance between exploration and utilization, the present invention effectively learns and optimizes strategies. At time t, action a is selected according to the ε-greedy greedy algorithm. t . Specific Q π (s,a) is:

[0060]

[0061] Among them, Q π (s,a) is the action value function, which is defined as the expectation of discounted return. It is a function of the state-action pair (s,a) and is related to the strategy π. The larger the action value, the better the current action. t is the discounted return starting from time t (i.e., the weighted value of the return at time t, t+1, t+2, ..., t+k, ..., R t =r t +γr t+1 +γ 2 r t+2 +…), γ is the discount factor, which determines the contribution of future rewards to the discounted return; the present invention can set it to 0.99; s t is the state at time t, a t is the action taken at time t; E π The operation is in a given state s t , perform action a t The discount return R t t+k is any time after time t, YesR t =r t +γr t+1 +γ 2 r t+2 +… combined expression.

[0062] The experience pool is used to place continuous experience experience Including status t 、Action at , Reward t , new state s t+1 , for the state s at time t t , take action a t After that, you will receive a reward r from the environment t and the new state s t+1 , store this set of values ​​into a tuple, called a set of transitions, denoted by To make the most of each set of experience In order to remove the adverse effects of the correlation between continuous states, the present invention adopts the experience playback technology. Specifically: for each continuous moment, the continuous Stacked together to form a transfer matrix Set the maximum number of rows in the transfer matrix to N memory , which represents the replay memory buffer size, which allows DQN to sample and learn from past experience. When the buffer is filled, the transition first stored in the buffer is deleted to ensure that the new Stored in the buffer. Note that: N memory is an adjustable hyperparameter. On the one hand, N memory The larger the value, the larger the buffer, the more transitions can be used for its training, and the agent can understand the environment more. On the other hand, N memory The larger the value, the more memory will be occupied, and the probability of each group transition being selected will decrease, so N memory It needs to be set reasonably according to needs.

[0063] Furthermore, we initialize the experience pool The capacity is N memory , and randomly initialize the parameters w of the DQN network in the lung nodule localization model. Then start training. The training process specifically includes:

[0064] S1, using the two-dimensional slice images containing lung nodules to train the lung nodule localization model, where the training learning cycle is M rounds. For each round of learning, the agent of the lung nodule localization model changes from the current state s t Start, proceed to N steps Step sampling, in each step, according to the network structure DQN output current state s t The corresponding action value And use the greedy algorithm to select the corresponding action a t , where action a t is to randomly select an action in the action space A with probability ε and select the action value with probability 1-ε The action corresponding to the maximum value is action at Used to guide the moving direction of the intelligent agent on the two-dimensional slice image.

[0065] S2, execute action a t , get the corresponding reward r t and the new state s t+1 , and the experience at the current time t Stored in the experience pool, where the reward is r t According to the agent to perform action a t The moving direction and the agent's action a t The final result is determined by whether the grid contains lung nodules.

[0066] S3, when the experience pool capacity reaches the preset value, randomly sample N from the experience pool bch Group Experience

[0067] S4, calculate each experience The timing difference target value of .

[0068] S5, update the parameter w of the lung nodule localization model according to the time series difference target value t , and update the state s t+1 , until the number of steps the agent moves reaches N steps The step or lung nodule localization model converges, and this round of learning is completed.

[0069] S6, after repeating M rounds of learning on the two-dimensional slice image, the positioning training is terminated, so that the lung nodule positioning model outputs the optimal path for the agent to move to the lung nodule grid. The optimal path is the path from the initial position of the agent to the position of any grid containing the lung nodule, and the agent moves the least number of steps and has the highest comprehensive reward.

[0070] Specifically, a complete training process for a small number of CT slices requires executing, for example, M episodes. For each episode, the agent can sample N steps Step, in each step, randomly select the transition matrix Sampling N batch Group transitions. Use the single-step temporal difference algorithm to update DQN. The starting state is s1, such as Figure 3 Starting from the upper left corner of the two-dimensional slice image, at a certain position at time t, that is, in state s t Next, according to the greedy strategy, action a is selected t ; Then perform action a t , observe the environment, and get immediate reward r t and the new state s t+1 ; Then store this set of values ​​in transition And put it into the experience pool In the experience pool When there is enough data (for example, N batch group), from Random sampling N batch The transition of each group is calculated, and the time difference target value (TD target) of each group is calculated to update the model parameter w of the DQN network in the lung nodule localization model according to the TD target value. t , and update the new state s of the agent t+1 If the experience pool When the data is insufficient, the above steps are not required to update the model parameters, and only the state of the agent needs to be updated. This is done until the state of the agent is the terminal state (i.e., the lung nodules are located) or the number of steps of the agent reaches N. steps Or if the lung nodule localization model converges, an episode is completed, and then the above steps S1-S5 are repeated for M rounds of learning, and the positioning training is ended, so that the lung nodule localization model outputs the optimal path for the intelligent agent to move to the lung nodule grid.

[0071] The present invention only needs to use a small amount of lung CT slices as training data and each two-dimensional slice contains a lung nodule. In the framework of the grid world, accurate lung nodule positioning can be obtained by training DQN using only the original image and its corresponding nodule mask, where a small amount means that compared with the number of training samples required for traditional radiological artificial intelligence, the amount of training samples required by this method is at least one order of magnitude lower. Since the current radiological artificial intelligence has been dominated by supervised deep learning, this method relies on a large amount of annotated data. Given that current radiological artificial intelligence mainly relies on supervised deep learning, this method usually requires hundreds or even thousands of annotated images to achieve high performance. In contrast, the present invention significantly reduces data requirements and provides an intuitive and efficient operating environment for medical image analysis in the grid world framework.

[0072] Furthermore, the reward r in step S2 t Including the first reward t1 , Second Reward t2 、Third Reward t3 , the fourth reward r t4 and the fifth reward r t5 , the first reward r t1 The agent moves toward the boundary and performs action a t After touching the grid boundary, the second reward is r t2 The agent remains stationary and performs action a t The grid does not contain lung nodules, and the third reward is r t3The agent remains stationary and performs action a t The grid contains lung nodules, and the fourth reward is r t4 For the agent to move to the grid containing the lung nodules, the fifth reward r t5 The agent moves to the grid that does not contain lung nodules, where r t1 <r t2 <r t5 <0<r t3 =r t4 .

[0073] For example, t1 =-4, r t2 =-2, r t5 =-0.5, r t3 =r t4 =+1. Figure 3 As shown, in the grid where the agent in the upper left corner of Figure a is located, if the agent performs an action of moving to the border of the grid world, its reward is -4. In Figure b, if the agent performs an action of moving up, down, left or right, since the grid after the action does not contain lung nodules, the rewards for these four actions are all -0.5; as in Figure b, the agent in the grid in the lower right corner remains in place, and the grid does not contain lung nodules, its reward is -2. In Figure c, if the agent performs an action to the right, after this action, the agent will move to the grid containing lung nodules, and its reward is +1. In Figure d, the agent remains in place, and the grid contains lung nodules, and its reward is +1.

[0074] In one embodiment, step S4 calculates each experience in the following manner: The timing difference target value of:

[0075]

[0076] Among them, TD target y j is the temporal difference target Q value of the jth experience, r j is the reward in the jth experience, s j+1 is the new state after the jth experience, γ is the discount factor, max a′ Q(s t+1 ,a′) is the maximum value of the action value function, that is, in state s j+1 Next, for all actions a ′ The Q value takes the maximum value.

[0077] Specifically, if the state s of the jth experience in the experience pool j+1 is the final state, then the TD target value is determined only by the current reward (immediate return), otherwise, the TD target value is determined by the current reward and the new state s t+1The maximum Q value among all actions is determined by the current reward and the maximum possible reward in the future. This formula combines the immediate reward and the maximum possible reward in the future (weighted by the discount factor) to calculate a more accurate temporal difference target value, which is used to guide the strategy update during the learning process.

[0078] Furthermore, in step S5, the parameter w of the lung nodule localization model is updated according to the time series difference target value. t , specifically including:

[0079] Calculate the loss function L based on all the time-series difference target values batch ;

[0080] According to the loss function L batch Execute the gradient descent algorithm to update the parameters w of the pulmonary nodule localization model t , get the next state s t+1 Parameters w of the lower lung nodule localization model t+1 .

[0081] Specifically, the loss function L is calculated based on all the time-series difference target values. batch ,include:

[0082]

[0083] in, Represents the action value corresponding to the action in the i-th experience of the network structure DQN at time t, Represents the time-series difference target value of the i-th experience at time t.

[0084] The loss function L is analyzed in the following way: batch Find the gradient g t :

[0085]

[0086] in, represents the symbol for partial derivative, w is the parameter of the pulmonary nodule localization model;

[0087] The parameter w of the lung nodule localization model is updated in the following way: t , get the next state s t+1 The parameters w of the lung nodule localization model are as follows: t+1

[0088] w t+1 =w t -α·g t

[0089] Among them, α represents the learning rate.

[0090] In one embodiment, in step S1, the current state s is output according to the network structure DQN. t The corresponding action value

[0091]

[0092] in, It represents the action value corresponding to the i-th action at time t, i=1,2,3,4,5.

[0093] Specifically, It is the output Q(s,a) of the network structure DQN.

[0094] Furthermore, in step S1, the following greedy algorithm is used to select the corresponding action a t :

[0095]

[0096] Among them, ε represents the exploration rate, the initial value is the set value, and it decays at a preset rate after completing this round of learning, 0<ε<1.

[0097] Specifically, ε is a positive number less than 1. In the decision-making process, there is a probability of ε to randomly select an action from |A| actions in the action space. If an action is randomly selected from |A| actions in the action space, then the probability of each action being selected is ε / |A|, where |A| is the number of actions; the remaining probability of 1-ε is selected. The action corresponding to the largest Q value in (that is, the optimal action corresponding to that moment), the probability of this action being selected is 1-ε+ε / |A|.

[0098] The key to the ε-greedy algorithm is to gradually reduce ε during the learning process so that the agent can conduct a lot of exploration in the early stage of training, and gradually increase the opportunities for utilization as the training time goes by until it converges to a smaller ε. This helps the agent to gradually optimize its strategy during the learning process and explore the environment to find better strategies. In this invention, the initial value of ε is set to 0.7. Each time an episode is executed, the parameter ε is preset to 1×10 -4 The rate of decay is until ε min =1×10 -4 By gradually reducing ε, the agent can gradually converge to a stable strategy while maintaining a certain exploration ability, which helps the algorithm to show stronger adaptability and robustness in different environments and tasks, and improves learning efficiency and accuracy.

[0099] Compared with the traditional method of selecting actions based on the maximum value of Q, the present invention uses the above method to select the corresponding action at There is a certain probability that an action that is not the maximum value of Q will be selected, which increases the exploratory nature of the entire grid space and improves the chance of finding the best path to locate lung nodules.

[0100] An embodiment of the present invention further provides a lung nodule localization device, which is used to localize lung nodules in lung images according to the lung nodule localization model obtained through the above training.

[0101] The device can use the trained lung nodule localization model to accurately identify lung nodules in lung images, and use the lung nodule localization model to quickly find the optimal path with the least steps and the greatest comprehensive reward to locate the lung nodules. It can automatically analyze and process lung images and complete the location of lung nodules without human intervention, which further improves the efficiency and convenience of diagnosis.

[0102] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0103] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0104] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0105] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0106] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the invention.

Claims

1. A lung nodule localization model training method, characterized in that: include: The pulmonary nodule localization model is trained using two-dimensional slice images containing pulmonary nodules, wherein the training learning cycle is M rounds. For each round of learning, the agent of the pulmonary nodule localization model changes from the current state s t Start, proceed to N steps Step sampling, in each step, according to the network structure DQN output current state s t The corresponding action value And use the greedy algorithm to select the corresponding action a t , where action a t is to randomly select an action in the action space A with probability ε and select the action value with probability 1-ε The action corresponding to the maximum value in t Used to guide the moving direction of the intelligent agent on the two-dimensional slice image; Perform the action a t , get the corresponding reward r t and the new state s t+1 , and the experience at the current time t Stored into the experience pool, where the reward r t According to the agent to perform the action a t The moving direction and the agent's action a t The latter is determined by whether the grid contains lung nodules; When the experience pool capacity reaches a preset value, N random samples are sampled from the experience pool. batch Group Experience Calculate each experience The target value of the timing difference of Update the parameter w of the lung nodule positioning model according to the time series difference target value t , and update the state s t+1 , until the agent moves N steps steps The step or lung nodule localization model converges, and this round of learning is completed; After repeating M rounds of learning on the two-dimensional slice image, the positioning training is terminated, so that the lung nodule positioning model outputs the optimal path for the intelligent agent to move to the lung nodule grid.

2. The method according to claim 1, characterized in that The reward r t Including the first reward t1 , Second Reward t2 、Third Reward t3 , the fourth reward r t4 and the fifth reward r t5 , the first reward r t1 The agent moves toward the boundary and performs action a t After touching the grid boundary, the second reward r t2 The agent remains stationary and performs action a t After the grid contains no lung nodules, the third reward r t3 The agent remains stationary and performs action a t After the grid contains lung nodules, the fourth reward r t4 For the agent to move to the grid containing the lung nodules, the fifth reward r t5 The agent moves to the grid that does not contain lung nodules, where r t1 <r t2 <r t5 <0<r t3 =r t4 .

3. The method according to claim 1, characterized in that Each experience is calculated as follows The target value of timing difference is: Among them, TD target y j is the temporal difference target Q value of the jth experience, r j is the reward in the jth experience, s j+1 is the new state after the jth experience, γ is the discount factor, max a′ Q(s t+1 ,a′) is the maximum value of the action value function, that is, in state s j+1 Under this condition, the Q value of all actions a′ is maximized.

4. The method according to claim 1, characterized in that: Update the parameter w of the lung nodule positioning model according to the time series difference target value t ,include: Calculate the loss function L based on all the time series difference target values batch ; According to the loss function L batch Execute the gradient descent algorithm to update the parameters w of the pulmonary nodule localization model t , get the next state s t+1 The parameters w of the lung nodule localization model are as follows: t+1 .

5. The method according to claim 4, characterized in that Calculate the loss function L based on all the time series difference target values batch ,include: in, It represents the action value corresponding to the action in the i-th experience of the network structure DQN at time t, Represents the time-series difference target value of the i-th experience at time t.

6. The method according to claim 5, characterized in that According to the loss function L batch Execute the gradient descent algorithm to update the parameters w of the pulmonary nodule localization model t , include: The loss function L is calculated as follows: batch Find the gradient g t : in, is the symbol for partial derivative, w is the parameter of the pulmonary nodule localization model; The parameter w of the lung nodule localization model is updated in the following way: t , get the next state s t+1 The parameters w of the lung nodule localization model are as follows: t+1 w t+1 =w t -α·g t Among them, α represents the learning rate.

7. The method according to claim 1, characterized in that According to the network structure DQN outputs the current state s t The corresponding action value in, It represents the action value corresponding to the i-th action at time t, i=1,2,3,4,5.

8. The method according to claim 1, characterized in that Use the following greedy algorithm to select the corresponding action a t : Among them, ε represents the exploration rate, the initial value is the set value, and it decays at a preset rate after completing this round of learning, 0<ε<1.

9. The method according to claim 1, characterized in that: Before training the lung nodule localization model using a two-dimensional slice image containing lung nodules, the following steps are also included: Acquire two-dimensional slice images containing lung nodules; Resampling the 2D slice images containing lung nodules; The resampled image space is divided into a number of grid units with preset pixel intervals, and each grid unit is a basic unit for the intelligent agent to move and search on the two-dimensional slice image.

10. The method according to claim 1, characterized in that The network structure DQN includes a convolutional Elu layer, a fully connected layer and an output layer.

11. A lung nodule positioning model training device, characterized in that: include: A processor and a memory connected to the processor; wherein the memory stores instructions executable by the processor, and the instructions are executed by the processor so that the processor executes the lung nodule positioning model training method as described in any one of claims 1-10.

12. A lung nodule localization device, characterized in that: include: A processor and a memory connected to the processor; wherein the memory stores instructions executable by the processor, and the instructions are executed by the processor so that the processor executes the lung nodule localization model trained by the method according to any one of claims 1-10 to locate lung nodules in lung images.

Images