A method and apparatus for generating cutting trajectory of a target to be cut in a medical image

Abstract

The present invention belongs to the field of computer vision technology, and provides a method and device for generating a cutting trajectory of a target to be cut in a medical image, the method comprising: obtaining an image to be processed, the image to be processed comprising a chest X-ray image, a cardiac MRI image, and a dermatoscope detection image; selecting an initial point of the image to be processed, and using the initial point as the starting point of a navigation agent, guiding the navigation agent to generate trajectory points until a cutting trajectory containing the target to be cut is generated; according to the generated trajectory points and the sampling area corresponding to each sampling operation, the deviation of each sampling is determined in real time, the direction of each sampling is corrected, and while generating a cutting trajectory on the image to be processed, the generated cutting trajectory is optimized to obtain a target image containing the target to be cut after cutting. The present invention solves the problem of instability such as segmented areas in existing medical images, and significantly improves the segmentation accuracy and stability of the target to be cut.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method and device for generating a cutting trajectory of a target to be cut in a medical image. Background Art

[0002] The application of deep learning in the field of medical image segmentation is an important progress in the intersection of medical imaging and artificial intelligence in recent years. This technology uses complex neural network models, especially convolutional neural networks (CNNs), to automatically identify and segment specific structures in medical images, such as organs, tumors or other physiological features.

[0003] Although the existing methods have achieved a certain segmentation accuracy, they still have the following problems. Since the generated segmentation boundary is not smooth enough, after extracting the pixel information of the rectangular area around its location, the trained neural network is usually used directly to predict the displacement of the next step. However, the prediction of this process is independent of the displacement of the previous step. Therefore, in the final result, the generated trajectory is not smooth enough, which affects the accuracy of the final image segmentation. In addition, the neural network is used to learn the correspondence between the image block information and the displacement information. The trained neural network can guide the "agent" to move on the image to be segmented, which can be understood as a process of numerically solving differential equations. This type of numerical method is usually accompanied by cumulative errors, which are mainly caused by the difference between discrete systems and continuous systems. Therefore, there is a problem of low segmentation accuracy due to the cumulative errors caused by the difference between discrete systems and continuous systems introduced by the neural network method.

[0004] Therefore, it is necessary to provide a new method for generating cutting trajectories of medical image objects to be cut to solve the above problems. Summary of the invention

[0005] The present invention aims to provide a method and device for generating a cutting trajectory of a target to be cut in a medical image, so as to solve the problem that the trajectory generated by the existing method is not smooth enough, thereby affecting the image segmentation accuracy, and the technical problem that the neural network method introduces cumulative errors caused by the differences between discrete systems and continuous systems, resulting in low segmentation accuracy. The technical problem to be solved by the present invention is achieved through the following technical solutions.

[0006] In a first aspect, the present invention provides a method for generating a cutting trajectory of a target to be cut in a medical image, comprising: obtaining an image to be processed, wherein the image to be processed includes a chest X-ray image, a cardiac MRI image, and a dermatoscope detection image, and the image to be processed includes a target to be cut of a related medical anatomical structure; selecting an initial point of the image to be processed, and using the initial point as the starting point of a navigation agent, guiding the navigation agent to generate trajectory points until a cutting trajectory including the target to be cut is generated, specifically comprising: performing a sampling operation on each trajectory point to obtain a sampling area (for example, a square image block intercepted from the original image of the image to be processed at the current trajectory point), inputting the obtained sampling area into a pre-trained deep learning model to obtain a displacement from each trajectory point to the next trajectory point, so that the navigation agent generates continuous trajectory points to include the target to be cut; according to the generated trajectory point and the sampling area corresponding to each sampling operation, real-time calculation determines the deviation of each sampling to correct the direction of each sampling, so that while generating a cutting trajectory on the image to be processed, the generated cutting trajectory is optimized; cutting the image to be processed according to the optimized cutting trajectory to obtain a target image including the target to be cut.

[0007] According to an optional implementation, an initial point of the image to be processed is selected, recursive sampling is performed on the initial point, and correction processing is performed a predetermined number of times to obtain a displacement corresponding to the initial point after the correction processing, specifically including performing a sampling operation on the initial point a predetermined number of times, and adjusting the sampling direction of the initial sampling area formed based on the initial point a predetermined number of times to obtain the sampling direction of the corrected initial sampling area; recursive sampling is performed on each trajectory point to be formed on the image to be processed, and correction processing is performed a predetermined number of times to obtain the displacement corresponding to each corrected trajectory point.

[0008] According to an optional implementation manner, when the predetermined number of times is C, the sampling direction of each trajectory point on the image to be processed is corrected C times, and the corrected displacement difference between the current time step t and the time step ti is calculated using the following expression:

[0009]

[0010] Among them, CDO t It represents the corrected displacement difference that the navigation agent needs to correct from the current trajectory point corresponding to the current time step t on the image to be processed to the next trajectory point, where t represents the current time step; S(C) is a sigmoid function with log, that is, It is used to characterize the weight at each time step, C represents the number of recursive sampling at the same trajectory point; v t is the displacement at time t or the current time step, v t-irepresents the displacement at time ti, i is a positive integer, specifically 1, 2, ..., n, i represents the number of steps of displacement from the current time t forward, when i = n, it means that the number of steps of displacement from the current time t forward is n; Δv t-i Indicates v t-i With v t-i-1 difference.

[0011] According to an optional embodiment, when the image to be processed is a chest X-ray image, the predetermined number of recursive samplings C is in the range of 5 to 20 times, preferably 15 times; when the image to be processed is a cardiac MRI image, the predetermined number of recursive samplings C is in the range of 5 to 20 times, preferably 15 times; when the image to be processed is a dermoscopy image, the predetermined number of recursive samplings C is in the range of 10 to 20 times, preferably 15 times.

[0012] According to an optional implementation, according to the current trajectory point corresponding to the current time step t, the calculated corrected displacement difference CDO t , correct the displacement from the current trajectory point to the next trajectory point:

[0013] v' t =v t +CDO t

[0014] Among them, v t Indicates the displacement from the current trajectory point to the next trajectory point; v' t Indicates the corrected displacement from the current track point to the next track point; CDO t It represents the corrected displacement difference that the navigation agent needs to correct from the current trajectory point corresponding to the current time step t on the image to be processed to the next trajectory point, where t represents the current time step.

[0015] According to an optional implementation manner, further comprising: the recursive sampling specifically comprises executing the following steps:

[0016] Step S201: when the navigation agent moves to the position of the current track point, the navigation agent first cuts out the image block corresponding to the sampling area from the image to be processed according to the displacement of the previous track point along the direction of the previous displacement;

[0017] Step S202: input the image block corresponding to the intercepted sampling area into a pre-trained deep learning model, output a temporary sampling direction, and adjust the sampling area corresponding to the current trajectory point to a sampling direction consistent with the temporary sampling direction;

[0018] Step S203: According to the number of recursive sampling, step S202 is repeatedly executed for the current trajectory point to output the temporary sampling direction of the corresponding number of times. When the predetermined number of recursive sampling is reached, the temporary displacement output for the last time, i.e., the temporary sampling direction, is used as the moving direction to the next trajectory point and the sampling direction of the next trajectory point.

[0019] According to an optional embodiment, it further comprises: determining whether the generation process of the cutting trajectory including the target to be cut is completed according to the convergence limiting condition, wherein:

[0020] The convergence limitation conditions include determining a detection line. In the process of generating a cutting trajectory including the target to be cut, an interval line is formed according to the intersection of the generated trajectory line and the detection line. The drawn interval line is further compared with a preset distance to determine whether the process of generating the cutting trajectory including the target to be cut is completed.

[0021] According to an optional implementation, the method further includes: moving to the next step according to the current trajectory point t , the calculated exponential moving average EMA t , calculate the corrected displacement v' corresponding to the current trajectory point to the next trajectory point t :

[0022] v' t =v t +EMA t

[0023] Among them, v t Indicates the displacement of the current trajectory to the next step; v' t Indicates the corrected displacement from the current trajectory point to the next trajectory point; EMA t It represents the corrected displacement difference that the navigation agent needs to correct for the current trajectory point corresponding to the current time step t on the image to be processed, where t represents the current time step.

[0024] The second aspect of the present invention proposes a device for generating a cutting trajectory of a target to be cut in a medical image, which adopts the method for generating a cutting trajectory of a target to be cut in a medical image described in the first aspect of the present invention. The device for generating a cutting trajectory of a target to be cut in a medical image comprises: a creation module for acquiring an image to be processed, wherein the image to be processed comprises a chest X-ray image, a cardiac MRI image, and a dermatoscope detection image, and the image to be processed comprises a target to be cut of a related medical anatomical structure; a sampling operation module for selecting an initial point of the image to be processed and using the initial point as the starting point of a navigation agent to guide the navigation agent to generate trajectory points until a cutting trajectory containing the target to be cut is generated, specifically comprising: sampling each trajectory point A sampling operation is performed to obtain a sampling area (for example, a square image block intercepted from the original image of the image to be processed at the current trajectory point), and the obtained sampling area is input into a pre-trained deep learning model to obtain the displacement from each trajectory point to the next trajectory point, so that the navigation agent generates continuous trajectory points to include the target to be cut; a correction module calculates and determines the deviation of each sampling in real time according to the generated trajectory points and the sampling area corresponding to each sampling operation to correct each sampling direction, so that the generated cutting trajectory is optimized while generating the cutting trajectory on the image to be processed; a cutting module cuts the image to be processed according to the optimized cutting trajectory to obtain a target image including the target to be cut.

[0025] The third aspect of the present invention provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in the first aspect of the present invention.

[0026] A fourth aspect of the present invention provides a computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method described in the first aspect of the present invention.

[0027] The embodiments of the present invention include the following advantages:

[0028] Compared with the prior art, the present invention selects the initial point of the image to be processed, and takes the initial point as the starting point, takes the initial point as the starting point of the navigation agent, guides the navigation agent to generate trajectory points, until a cutting trajectory containing the target to be cut is generated, and determines the deviation of each sampling in real time according to the generated sampling area corresponding to each sampling operation to correct each sampling direction, so that while generating the cutting trajectory on the image to be processed, the generated cutting trajectory is optimized; the image to be processed is cut according to the optimized cutting trajectory to obtain a target image containing the target to be cut, and the segmentation accuracy is improved while optimizing the cutting trajectory of the target to be cut, and the problem of low segmentation accuracy caused by the cumulative error introduced by the neural network method due to the difference between the discrete system and the continuous system is solved.

[0029] In addition, the present invention recursively samples each trajectory point on the image to be processed (i.e., samples the same trajectory point a predetermined number of times), corrects the displacement from the current trajectory point to the next trajectory point (i.e., the moving direction or sampling direction) according to the current trajectory point corresponding to the current time step and the calculated corrected displacement difference, and adjusts the predetermined number of recursive samplings for different medical image applications, thereby obtaining a more accurate target trajectory to be cut and further improving the segmentation accuracy.

[0030] In addition, by adjusting the preset distance of the convergence conditions of different medical images, a more accurate trajectory of the target to be cut can be obtained while optimizing the pre-trained deep learning model, and the segmentation accuracy can be further improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a flowchart of an example of a method for generating a cutting trajectory of a target to be cut in a medical image according to the present invention;

[0032] Figure 2 It is a schematic diagram of a specific example of an image to be processed using the method for generating a cutting trajectory of a target to be cut in a medical image of the present invention;

[0033] Figure 3 It is a schematic diagram of an example of recursive sampling in the method for generating cutting trajectories of a target to be cut in a medical image of the present invention;

[0034] Figure 4 It is a specific example diagram of the relationship between the image of the sampling area and the displacement vector in the image to be processed using the method for generating the cutting trajectory of the target to be cut in the medical image of the present invention;

[0035] Figure 5 It is a schematic diagram of a specific example of determining the generation of a to-be-cut trajectory in a dermatoscope detection image using the image to-be-cut target cutting control method of the present invention;

[0036] Figure 6 It is a schematic diagram of an example of calculating the exponential moving average in the method for generating a cutting trajectory of a target to be cut in a medical image of the present invention;

[0037] Figure 7 is a schematic diagram of an example of a cutting trajectory generated by the method for generating a cutting trajectory of a target to be cut in a medical image according to the present invention;

[0038] Figure 8 It is a structural block diagram of an example of a device for generating a cutting trajectory of a target to be cut in a medical image according to the present invention;

[0039] Fig. 9 It is a structural block diagram of an example of an electronic device of the present invention. DETAILED DESCRIPTION

[0040] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0041] In view of the above problems, the present invention proposes a method for generating a cutting trajectory of a target to be cut in a medical image. The method selects an initial point of the image to be processed, and starts a sampling operation with the initial point as the starting point and an initial sampling area generated by the initial point until a cutting trajectory containing the target to be cut is generated. According to the displacement information predicted by the pre-trained deep learning model, the exponential moving average is calculated in real time to determine the deviation of each sampling to correct the direction of each sampling, so that while generating a cutting trajectory on the image to be processed, the generated cutting trajectory is optimized; the image to be processed is cut according to the optimized cutting trajectory to obtain a target image containing the target to be cut, and the segmentation accuracy is improved while optimizing the cutting trajectory of the target to be cut, and the problem of low segmentation accuracy caused by the cumulative error introduced by the neural network method due to the difference between the discrete system and the continuous system is solved.

[0042] In addition, the present invention can generate a cutting trajectory more accurately and optimize the generated cutting trajectory while generating the cutting trajectory by recursively sampling each trajectory point on the processed image (i.e., sampling the same trajectory point a predetermined number of times) and correcting the sampling direction of the sampling area corresponding to each trajectory point a predetermined number of times.

[0043] Example 1

[0044] Refer to the following Figure 1 , Figure 2 , Figure 3 , Figure 4 , Figure 5 Figure 6 and Figure 7 , the contents of the present invention will be described in detail.

[0045] Figure 1 It is a flowchart of an example of the method for generating a cutting trajectory of a target to be cut in a medical image of the present invention.

[0046] like Figure 1 As shown, in step S101, an image to be processed is acquired, wherein the image to be processed includes a chest X-ray image, a cardiac MRI image, and a dermoscopy detection image, and the image to be processed contains a target to be cut of a related medical anatomical structure.

[0047] For example, an image to be processed is obtained from an image captured in real time.

[0048] Specifically, the images to be processed include chest X-ray images (specifically including the heart to be segmented), cardiac MRI images (specifically including the left atrium to be segmented), dermoscopy images (specifically including pigmented nevi to be segmented), and the like.

[0049] It should be noted that the above is only described as an optional example and should not be understood as a limitation to the present invention.

[0050] Next, in step S102, an initial point of the image to be processed is selected, and the initial point is used as the starting point of the navigation agent, and the navigation agent is guided to generate trajectory points until a cutting trajectory including the target to be cut is generated, specifically including: performing a sampling operation on each trajectory point to obtain a sampling area, inputting the obtained sampling area into a pre-trained deep learning model to obtain a displacement from each trajectory point to the next trajectory point, so that the navigation agent generates continuous trajectory points to include the target to be cut;

[0051] Figure 2 It is a schematic diagram of an example of recursive sampling in the method for generating a cutting trajectory of a target to be cut in a medical image of the present invention. Figure 3 It is a schematic diagram of another example of recursive sampling in the method for generating a cutting trajectory of a target to be cut in a medical image of the present invention.

[0052] like Figure 2 As shown, an initial point O of a chest X-ray image, ie, an image to be processed, is selected, and a sampling operation is performed starting from the initial point O and an initial sampling area generated by the initial point O.

[0053] Specifically, a point (e.g., point O) in the chest X-ray image is randomly selected as the initial point, the initial point O is used as the starting point, and an initial sampling area (e.g., Figure 2The rectangular area ABCD in the figure, i.e., the initial sampling area ABCD), the rectangular area corresponding to the initial sampling area forms an angle with respect to the horizontal direction or the vertical direction, and the angle represents the direction in which the next sampling operation of the initial sampling area will move (i.e., the sampling direction or the moving direction).

[0054] Furthermore, the sampling operation is performed starting from the initial sampling area ABCD generated by the initial point O, and the sampling operation of the next time step is guided according to the sampling direction determined by the first sampling area ABCD, and the sampling direction is repeatedly determined in sequence to guide the navigation agent to generate a trajectory point, specifically, the next trajectory point corresponding to the sampling operation of the next time step.

[0055] In the first embodiment, recursive sampling is performed on the initial point O, and correction processing is performed a predetermined number of times to obtain the displacement corresponding to the initial point O after the correction processing, specifically including performing a predetermined number of sampling operations on the initial point O, and adjusting the sampling direction of the initial sampling area formed based on the initial point O a predetermined number of times to obtain the sampling direction of the corrected initial sampling area.

[0056] Specifically, recursive sampling is performed on each trajectory point to be formed on the processed image (including the trajectory point corresponding to the initial point O), and correction processing is performed a predetermined number of times to obtain the displacement corresponding to each trajectory point after correction.

[0057] The following will refer to Figure 2 and Figure 3 The example illustrates the specific process of recursive sampling.

[0058] Specifically, the recursive sampling comprises executing the following steps:

[0059] Step S201: When the navigation agent moves to the position of the current track point, the navigation agent first cuts out an image block corresponding to the sampling area from the image to be processed according to the displacement of the previous track point along the direction of the previous displacement.

[0060] Step S202: input the image block corresponding to the intercepted sampling area into the pre-trained deep learning model, output a temporary sampling direction, and adjust the sampling area corresponding to the current trajectory point to a sampling direction consistent with the temporary sampling direction.

[0061] Step S203: According to the number of recursive sampling, step S202 is repeatedly executed for the current trajectory point to output the temporary sampling direction of the corresponding number of times. When the predetermined number of recursive sampling is reached, the temporary displacement output for the last time, i.e., the temporary sampling direction, is used as the moving direction to the next trajectory point and the sampling direction of the next trajectory point.

[0062] For the initial point O, the image block obtained by cutting the sampling area formed by the initial point O is input into the pre-trained deep learning model, and a temporary sampling direction is output. The sampling area corresponding to the current trajectory point is adjusted to a sampling direction consistent with the temporary sampling direction (such as Figure 2 Temporary direction F1) shown.

[0063] It should be noted that at the initial point, since there is no displacement in the previous step, the sampling direction of the initial sampling area will be randomly selected from (0-360 degrees), for example, the sampling direction is F0, see Figure 2 .

[0064] For trajectory points other than the initial point, when the navigation agent moves to the position of the current trajectory point, the navigation agent first moves in the direction of the previous step P t-1 To P t ,P t-1 To P t , the displacement between them is v t-1 , that is, P t-1 +v t-1 =P t ), intercepted as Figure 3 The image in the sampling area EFGH shown is cut out to obtain an image block), that is, this point P t Image information of a square area centered on the image.

[0065] Specifically, when the image block corresponding to the cut sampling area EFGH is input into the pre-trained deep learning model, a temporary displacement ( ), that is, a temporary sampling direction f1 (such as Figure 3 As shown in FIG. 1 , the sampling direction F2 of the current trajectory point is adjusted to the temporary sampling direction f1 (i.e., adjusted to the sampling direction consistent with the temporary sampling direction f1), forming a sampling area E'F'G'H' corresponding to the temporary sampling direction f1, cutting out the sampling area E'F'G'H' from the image to be processed, and looping step S202 for the current trajectory point. When the predetermined number of recursive sampling (i.e., the preset number of loops) is reached, the loop ends, and the temporary displacement outputted last time, i.e., the temporary sampling direction, is used as the moving direction to the next trajectory point and the sampling direction of the next trajectory point. Note that the temporary displacement generated above will not be used to move the navigation agent.

[0066] For the deep learning model, a CNN network model is used to construct a deep learning model, and a training data set is used to train the deep learning model to obtain a pre-trained deep learning model. The training data set includes a cardiac MRI image with a left ventricle labeled, a chest X-ray image with a heart labeled, and a dermoscopic detection image with a pigmented nevus labeled.

[0067] In this example, the above-mentioned annotated images (specifically including chest images and dermoscopic images with the target to be segmented) are all black-and-white images with the target to be segmented marked. The black-and-white image of the annotated image is transformed into a dynamic field system (Dynamic Field) with a limit cycle, that is, a vector field, and the resolution (mesh density) of the vector field is the same as the resolution of the black-and-white image. For each position on the black-and-white image corresponding to the annotated image, the vector (unique vector) at the corresponding position in the vector field can be found. The image blocks in each sampling area (for example Figure 2 The rectangular area ABCD corresponding to the initial sampling area in ) has a one-to-one correspondence with the vector, for example, the displacement vector v x1y1 、v x2y2 、v x3y3 ......v xn-3yn-3 、v xn-2yn-2 、v xn-1yn-1 、v xnyn The corresponding sampling areas on the black and white image of the image to be processed are in a one-to-one correspondence between the image blocks and the vectors in each sampling area. For details, see Figure 4 The annotated image (specifically including a chest image and a dermoscopic detection image annotated with the target to be segmented) includes a matching correspondence between an image block and a vector. In each time step, a sampling operation is performed on the same area of ​​interest of the image to be processed (for example, the target to be cut including the left ventricle, the heart, a pigmented nevus, etc.), and the displacement vector corresponding to each sampling area can be determined according to the above matching correspondence to form a training data set for training a deep learning model.

[0068] The network structure of the deep learning model includes an input layer, five convolutional layers, five pooling layers, a fully connected layer, and an output layer. For details, see Table 1 below.

[0069] For the input layer, the input image patch (Patch), for example, the output image patch (the area obtained after sampling) is an RGB image of size 64x64x3, such as a dermatoscope detection image.

[0070] For example, the input image patch is a grayscale image of size 64x64x1 (e.g., cardiac MRI image, chest X-ray image). For the fully connected layer, it contains 4096 neurons. For the output layer, the thousandth layer is changed to contain two neurons, for example, the output displacement ( ), which is the sampling displacement direction.

[0071] Table 1

[0072]

[0073] Table 1 shows relevant parameters of an example of the network structure of the deep learning model of the present invention.

[0074] By training the deep learning model with the above training data set, the pre-trained deep learning model can learn the matching correspondence between the "navigation agent" and the displacement of the target to be cut, so as to obtain a trained pre-trained deep learning model.

[0075] It should be noted that after the above model training process, the trained model will guide the "navigation agent" to move on the above medical images. This movement is step by step, that is, the navigation gives the displacement from the current trajectory point to the next step based on the image block information of a directional square area around the current trajectory point, for example, using v t (Here, t=1, 2, 3, 4…n, n is a positive integer, and t represents the step number).

[0076] Furthermore, when the predetermined number of times is C, the sampling direction of each trajectory point on the processed image is corrected C times, and the following expression is used to calculate the corrected displacement difference between the current time step t and the time step ti in real time to determine the deviation of each sampling, which is used to correct each sampling direction:

[0077]

[0078] Among them, CDO t It represents the corrected displacement difference that the navigation agent needs to correct from the current trajectory point corresponding to the current time step t on the image to be processed to the next trajectory point, where t represents the current time step; S(C) is a sigmoid function with log, that is, It is used to characterize the weight at each time step, C represents the number of recursive sampling at the same trajectory point; v t is the displacement at time t or the current time step, v t-i represents the displacement at time ti, i is a positive integer, specifically 1, 2, ..., n, i represents the number of steps of displacement from the current time t forward, when i = n, it means that the number of steps of displacement from the current time t forward is n; Δv t-i Indicates v t-i With v t-i-1 difference.

[0079] The calculated corrected displacement difference between the current time step t and the time step ti is used to correct the displacement of the current trajectory point corresponding to the current time step t, thereby obtaining the corrected displacement of each trajectory point. The proxy navigation moves on the image to be processed to generate each trajectory point until a cutting trajectory containing the target to be cut is generated.

[0080] According to the segmentation accuracy results obtained from multiple experiments, the predetermined number of recursive sampling in different medical image application scenarios is obtained. The details are as follows:

[0081] When the image to be processed is a chest X-ray image, the predetermined number C of recursive sampling is in the range of 5 to 20 times, preferably 15 times.

[0082] When the image to be processed is a cardiac MRI image, the predetermined number C of recursive sampling is in the range of 5 to 20 times, preferably 15 times.

[0083] When the image to be processed is a dermoscopic detection image, the predetermined number C of recursive sampling is in the range of 10 to 20 times, preferably 15 times.

[0084] It should be noted that the predetermined number of recursive samplings mentioned above is obtained through multiple experiments, that is, a range of values ​​is determined, and then the optimal number or optimal range of the predetermined number of recursive samplings is obtained according to the segmentation accuracy corresponding to each parameter. In addition, in order to enable the pre-trained deep learning model to more accurately determine the displacement of the current trajectory point to the next trajectory point at each time step. Every time the navigation agent reaches a new trajectory point, the pre-trained deep learning model is used to make multiple adjustments to the predicted displacement (i.e., the model output of the pre-trained deep learning model) to correct it to obtain a more accurate displacement. Recursive sampling of the same trajectory point is a cyclic step. The number of cycles is adjusted specifically according to the application scenarios of different medical images.

[0085] In the second embodiment, the following expression is used to calculate the exponential moving average of the current time step t and the time step ti in real time, so as to be used as the correction displacement difference required to be corrected when the current trajectory point goes to the next trajectory point, to correct the correction displacement required to be corrected when the current trajectory point goes to the next trajectory point, to determine the deviation of each sampling, and to correct the direction of each sampling:

[0086]

[0087] Among them, EMA t It refers to the corrected displacement difference that the navigation agent needs to correct when it moves from the current trajectory point corresponding to the current time step t on the image to be processed to the next trajectory point. t represents the current time step; η represents the weight corresponding to different time steps; v t is the displacement at time t or the current time step, v t-i represents the displacement at time ti, i is a positive integer, i represents the number of steps of forward displacement starting from the current time t; Δv t-i Indicates v t-i With v t-i-1 difference.

[0088] Optionally, η is in the range of 0.4 to 0.9.

[0089] The second implementation method is used to calculate the correction displacement required to move from the current trajectory point to the next trajectory point. It is necessary to determine the weight η corresponding to different time steps and the number of recursive sampling corresponding to different medical image application scenarios.

[0090] It should be noted that the predetermined number of recursive sampling and the adjustment of weights corresponding to different time steps are obtained through multiple experiments, that is, a range of values ​​is determined, and then the optimal number or optimal range of the predetermined number of recursive sampling and the optimal value or optimal range of weights corresponding to different time steps are obtained according to the segmentation accuracy corresponding to each parameter. In addition, in order to enable each time step of the pre-trained deep learning model to more accurately determine the displacement of the current trajectory point to the next trajectory point. At each new trajectory point of the navigation agent, the pre-trained deep learning model is used to make multiple adjustments to the predicted displacement (i.e., the model output of the pre-trained deep learning model) to correct it to obtain a more accurate displacement. Recursive sampling of the same trajectory point is a cyclic step. The number of cycles is specifically adjusted according to the application scenarios of different medical images.

[0091] In the third embodiment, a sampling operation is performed within a time step, one time step corresponds to one sampling operation, and the agent navigation forms a trajectory point on the image to be processed.

[0092] Specifically, the image (image block) in the area corresponding to the initial sampling area in the image to be processed is intercepted and input into the pre-trained deep learning model, the direction vector F1 of the second sampling operation (i.e., the next sampling operation) is output, and the sampling direction of the initial sampling area on the image to be processed is adjusted to F1, that is, the displacement direction or moving direction F1 of the second sampling operation (i.e., the next sampling operation) corresponding to the current time step t is determined. For example, a 64x64 square area is taken as the initial sampling area, and the direction of the square area is the displacement (displacement) at the corresponding position of the corresponding vector field. The image block in this square area and the determined displacement vector are saved.

[0093] Specifically, the sampling direction F1 is subjected to displacement correction processing (also referred to as correction processing) to obtain the sampling direction F1' after correction processing, so as to guide the sampling operation of the next time step according to the determined sampling direction F1'. Specifically, the sampling direction F1' is intercepted in the area corresponding to the image to be processed, input into the pre-trained deep learning model, and the displacement direction or moving direction F2 of the third sampling operation is output. The sampling direction F2 is subjected to displacement correction processing to obtain the sampling direction F2' after correction processing to guide the sampling operation of the next time step according to the determined sampling direction F2'. Specifically, the sampling direction F2' is intercepted in the area corresponding to the image to be processed, input into the pre-trained deep learning model, and the displacement direction or moving direction F3 of the fourth sampling operation is output. The sampling direction F3 is subjected to displacement correction processing to obtain the sampling direction F3' after correction processing, so as to guide the sampling operation of the next time step according to the determined sampling direction F3'.

[0094] Next, the sampling operation is performed in sequence, so that the sampling direction of the next displacement corresponding to the next time step is determined according to the moving direction of the current time step, until a cutting trajectory including the target to be cut is generated.

[0095] In the first embodiment, the second embodiment and the third embodiment, whether the generation process of the cutting trajectory including the object to be cut is completed is determined according to the convergence limiting condition.

[0096] Specifically, the convergence limitation conditions include determining a detection line, and in the process of generating a cutting trajectory including the target to be cut, forming an interval line according to the intersection of the generated trajectory line and the detection line, and further comparing the drawn interval line with a preset distance to determine whether the process of generating the cutting trajectory including the target to be cut is completed.

[0097] Figure 5 1 is a schematic diagram of a specific example of determining the generation of a to-be-cut trajectory in a dermatoscope detection image using the image to-be-cut target cutting control method of the present invention. Figure 5 An example of the process of determining whether the generation of the cutting trajectory including the target to be cut is completed according to the convergence limitation condition is shown in FIG.

[0098] like Figure 5 As shown, on the image to be processed, starting from the initial point O, each trajectory point is generated in sequence to form a trajectory line. Specifically, the image center Z of the image to be processed is connected to any point on any boundary line of the image to be processed to form a detection line, and the detection line is used to form an interval line with the intersection of the generated trajectory line for further comparison and judgment of the preset distance.

[0099] It should be noted that since the anatomical structure of the target to be segmented in the image to be processed (the heart in the chest X-ray, the left atrium in the cardiac MRI, and the pigmented nevus in the dermoscopy detection image) is a closed, approximately elliptical area and is usually located in the central area of ​​the image to be processed, a straight line is formed by connecting the midpoint of the image to be processed and any point on any boundary line of the image to be processed to serve as the detection line.

[0100] According to the above-mentioned sampling operation process, each trajectory point and trajectory line is generated in sequence, and the intersection points of the generated trajectory line and the detection line are recorded, such as the three intersection points P1, P2, and P3. The interval line (such as interval line P2P3) formed based on the last two intersection points is used for comparison with the predetermined distance. Specifically, when the formed interval line (such as interval line P2P3) is less than the predetermined distance, it is determined that the convergence limitation condition is met, and it is determined that the generation process of the cutting trajectory containing the target to be cut is completed. When the formed interval line (such as interval line P2P3) is less than the predetermined distance, it is determined that the convergence limitation condition is not met, and it is determined that the generation process of the cutting trajectory containing the target to be cut is not completed. It is necessary to continue to perform sampling and trajectory generation operations until the convergence limitation condition is met to determine that the generation process of the cutting trajectory containing the target to be cut is completed. For example, it is determined that the generation process of the cutting trajectory of the left atrium and the heart is completed.

[0101] It should be noted that there is no particular limitation on the number of intersections. In other implementations, there may be 8, 9 or more intersections. The above is only described as an optional example and should not be construed as a limitation to the present invention.

[0102] In addition, in order to avoid an infinite loop in the algorithm of the pre-trained deep learning model, the maximum step size of the navigation agent's movement is adjusted to a specific value, such as 10,000 steps. In order to adapt to different application scenarios of medical images, the preset distance of the above-mentioned convergence limiting condition is adjusted. When the navigation agent crosses the detection line for the first time, the intersection of the "navigation agent" and the detection line on the image to be processed is observed. When the interval line formed by the two previous and subsequent intersections is less than the specified range of the predetermined distance (for example, when the specified range of pixel distance is 2-10, due to the large difference in image resolution in different application scenarios, the predetermined distance is determined according to different application scenarios, and can be directly calculated using 2D Euclidean distance). Then the navigation agent stops moving, and based on the intersection information, the trajectory line between the last two intersections of the agent to be retained on the detection line is determined as the cutting trajectory of the target to be cut.

[0103] Specifically, when the target to be segmented is the left atrium, the preset distance (specifically, preset pixels) of the above convergence limiting condition is 2.

[0104] When the target to be segmented is a heart, the preset distance (specifically, preset pixels) of the above convergence limiting condition is 5.

[0105] When the target to be segmented is a pigmented nevus, the preset distance (specifically, the preset pixels) of the above convergence limiting condition is 10.

[0106] By adjusting the preset distance of the convergence conditions of different medical images, a more accurate trajectory of the target to be cut can be obtained while optimizing the pre-trained deep learning model, and the segmentation accuracy can be further improved.

[0107] In a specific embodiment, when applying a trained deep learning model (i.e., a pre-trained deep learning model), a sampling area of ​​a dermatoscope detection image is input, and a displacement corresponding to the center point of the sampling area (the initial point, each trajectory point to be generated by the navigation agent) is output, specifically including a recursive sampling process to obtain the corrected displacement of each trajectory point, and obtain a cutting trajectory (such as Figure 5 The trajectory line shown is P2hdefgP3).

[0108] It should be noted that the above is only described as an optional example and should not be understood as a limitation to the present invention.

[0109] Next, in step S103, the deviation of each sampling is calculated in real time based on the generated trajectory points and the sampling area corresponding to each sampling operation to correct the direction of each sampling, so that the generated cutting trajectory is optimized while generating the cutting trajectory on the image to be processed.

[0110] Specifically, the exponential moving average is calculated in real time to determine the deviation of each sampling to correct the direction of each sampling.

[0111] In the first embodiment, according to the current trajectory point corresponding to the current time step t, the calculated corrected displacement difference CDO t , correct the displacement from the current trajectory point to the next trajectory point (that is, the moving direction or sampling direction):

[0112] v' t =v t +CDO t (3)

[0113] Among them, v t Indicates the displacement from the current trajectory point to the next trajectory point; v' t Indicates the corrected displacement from the current track point to the next track point; CDO t It represents the corrected displacement difference that the navigation agent needs to correct from the current trajectory point corresponding to the current time step t on the image to be processed to the next trajectory point, where t represents the current time step.

[0114] By adopting the first implementation method, the correction displacement required to move from the current trajectory point to the next trajectory point is calculated. It is only necessary to determine the number of recursive sampling corresponding to the application scenarios of different medical images, without determining the weights η corresponding to different time steps. Under the same amount of calculation, the calculation process can be further optimized and the image segmentation accuracy can be improved.

[0115] According to the current trajectory point corresponding to the current time step, the calculated corrected displacement difference CDO t , correcting the displacement from the current trajectory point to the next trajectory point (that is, the moving direction or sampling direction), and adjusting the predetermined number of recursive sampling according to the application of different medical images, a more accurate target trajectory to be cut can be obtained, which can further improve the segmentation accuracy.

[0116] Figure 6 It is a schematic diagram of an example of using the first implementation method in the method for generating a cutting trajectory of a target to be cut in a medical image of the present invention to calculate the correction displacement required for the current trajectory point to move to the next trajectory point.

[0117] like Figure 6 As shown, it is known that the displacement between p0 and p1 is v1, the displacement between p1 and p2 is v2, the displacement between p2 and p3 is v3, the displacement between p3 and p4 is v4, and the predetermined number of recursive sampling is C=10 times. When the current time step t=4 and the current trajectory point is p4, the above expression (1) is used to calculate CDO4, and the displacement v4 of the next trajectory point p5 is further calculated.

[0118]

[0119] Next, using the above expression (3), v' t =v t +CDO t , calculate v' t .

[0120] In the second embodiment, the displacement v from the current trajectory point to the next trajectory point is calculated. t , the calculated exponential moving average EMA t , calculate the corrected displacement v′ corresponding to the current trajectory point to the next trajectory point t :

[0121] v' t =v t +EMA t (4)

[0122] Among them, v t Indicates the displacement from the current trajectory to the next trajectory point; v' tIndicates the corrected displacement from the current trajectory point to the next trajectory point; EMA t It represents the corrected displacement difference that the navigation agent needs to correct from the current trajectory point corresponding to the current time step t on the image to be processed to the next trajectory point, where t represents the current time step.

[0123] The second implementation method is used to calculate the corrected displacement required to move from the current trajectory point to the next trajectory point. There is a trajectory line [p0p1p2p3p4]. It is known that the displacement between p0 and p1 is v1, the displacement between p1 and p2 is v2, the displacement between p2 and p3 is v3, and the displacement between p3 and p4 is v4. When the current time step t=4 and the current trajectory point is p4, the above expression (2) is used to calculate EMA4, and further calculate the displacement v4 of the next trajectory point p5.

[0124] It is known that Δv1=v1-v0, Δv2=v2-v1, Δv3=v3-v2, η=0.7, the current time step t=4, and the current trajectory point is p4, then the displacement v4 needs to be corrected The correction process for the displacement v4 from the current trajectory point p4 to the next trajectory point p5 is specifically embodied as: v'4=v4+EMA4.

[0125] In this example, for the image to be processed, according to the image type, the shape and size of the object to be cut, the image resolution parameter, etc., it is determined that η is in the range of 0.4 to 0.9.

[0126] When the image to be processed is a cardiac MRI image and the target to be cut is the left atrium, η is in the range of 0.7 to 0.9, and η is preferably 0.8.

[0127] When the image to be processed is a chest X-ray image and the target to be cut is a heart, η is in the range of 0.4 to 0.6, and η is preferably 0.5.

[0128] When the image to be processed is a dermatoscopic detection image and the target to be cut is a pigmented nevus, η is in the range of 0.5 to 0.7, and η is preferably 0.6.

[0129] Next, in step S104, the image to be processed is cut according to the optimized cutting trajectory to obtain a target image containing the object to be cut.

[0130] Specifically, the image to be processed is cut according to the cutting trajectory optimized in step S103 (for example Figure 7 The white area under the black background is the target to be cut, and the boundary line formed by the white area and the black background is the cutting trajectory line) to obtain a target image containing the target to be cut.

[0131] It should be noted that the above is only described as an optional example and should not be understood as a limitation to the present invention.

[0132] In addition, the drawings are only schematic illustrations of the processes included in the method according to an exemplary embodiment of the present invention, and are not intended to be limiting. It is easy to understand that the processes shown in the drawings do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0133] In order to verify the technical effect achieved by the present invention, an ablation experiment is conducted. The ablation experiment refers to gradually removing or modifying certain components in the same system (including multiple components), and then observing the impact on the model performance. The performance of the three methods, namely, the existing method (such as the DPM method), the method using the first implementation method (recursively sampling each trajectory point, calculating CDOt to correct the displacement of each trajectory point), and the method of sampling the second implementation method (recursively sampling each trajectory point, calculating EMAt to correct the displacement of each trajectory point) are compared. The following ten parameters are tested using the test data set (Test Set) in the public test data set of each application scenario using three application scenarios (chest X-ray image, cardiac MRI image, dermatoscope detection image). The public test data set is downloaded from the public data on the relevant network. For example, it also includes black and white pictures with the same resolution as the corresponding images annotated by relevant professionals, which are used as standard segmentation results (saved in the form of black and white pictures, with black as the background and white as the segmentation target area).

[0134] The segmentation accuracy of the to-be-cut track generated by the annotated image and the above three methods (for example, using average Dice) is compared. The segmentation accuracy of the to-be-cut track generated by the existing method is 0.892-0.901, the segmentation accuracy of the to-be-cut track generated by the method of the first embodiment of the invention is 0.914-0.951, and the segmentation accuracy of the to-be-cut track generated by the method of the second embodiment of the invention is 0.910-0.931. Obviously, the segmentation accuracy achieved by the two embodiments of the present invention is significantly higher than the segmentation accuracy achieved by the existing methods. Therefore, the segmentation accuracy of the to-be-cut track can be significantly improved by using the method of the present invention.

[0135] It should be noted that the segmentation target area consistency comparison standard adopted is, for example, DICE, a mainstream evaluation standard for segmentation tasks. The value range of DICE is a real number from 0 to 1. When Dice = 1, it means that the trajectory of the target to be cut estimated by this method is completely consistent with the target to be cut in the standard image provided by relevant professionals. When Dice = 0, it means that the areas provided by the two are completely inconsistent, or there is no intersection. As can be seen from Table 1, by comparing the segmentation accuracy, it can be obtained that the method of the first embodiment and the method of the second embodiment of the present invention are significantly better than the existing method.

[0136] By grid search of the two parameters, the parameter combination with the highest average Dice can be obtained as the optimal parameter combination for each medical image application scenario, where, when the target to be cut is the left atrium, η is 0.8, and the predetermined number of recursive sampling C is 15 times. When the target to be cut is the heart, η is 0.5, and the predetermined number of recursive sampling C is 15 times. When the target to be cut is a pigmented nevus, η is 0.6, and the predetermined number of recursive sampling C is 15 times.

[0137] Compared with the prior art, the present invention selects the initial point of the image to be processed, and takes the initial point as the starting point, takes the initial point as the starting point of the navigation agent, guides the navigation agent to generate trajectory points, until a cutting trajectory containing the target to be cut is generated, and determines the deviation of each sampling in real time according to the generated sampling area corresponding to each sampling operation to correct each sampling direction, so that while generating the cutting trajectory on the image to be processed, the generated cutting trajectory is optimized; the image to be processed is cut according to the optimized cutting trajectory to obtain a target image containing the target to be cut, and the segmentation accuracy is improved while optimizing the cutting trajectory of the target to be cut, and the problem of low segmentation accuracy caused by the cumulative error introduced by the neural network method due to the difference between the discrete system and the continuous system is solved.

[0138] In addition, the present invention recursively samples each trajectory point on the image to be processed (i.e., samples the same trajectory point a predetermined number of times), corrects the displacement from the current trajectory point to the next trajectory point (i.e., the moving direction or sampling direction) according to the current trajectory point corresponding to the current time step and the calculated corrected displacement difference, and adjusts the predetermined number of recursive samplings for different medical image applications, thereby obtaining a more accurate target trajectory to be cut and further improving the segmentation accuracy.

[0139] In addition, by adjusting the preset distance of the convergence conditions of different medical images, a more accurate trajectory of the target to be cut can be obtained while optimizing the pre-trained deep learning model, and the segmentation accuracy can be further improved.

[0140] Example 2

[0141] The following are embodiments of the device of the present invention, which can be used to implement the method embodiments of the present invention. For details not disclosed in the device embodiments of the present invention, please refer to the method embodiments of the present invention.

[0142] Figure 8 It is a structural schematic diagram of an example of a device for generating a cutting trajectory of a target to be cut in a medical image according to the present invention.

[0143] Reference Figure 8 The second aspect of the present disclosure provides a device 800 for generating a cutting trajectory of a target to be cut in a medical image, which adopts the method for generating a cutting trajectory of a target to be cut in a medical image described in the first aspect of the present disclosure.

[0144] The device 800 for generating a cutting trajectory of a target to be cut in a medical image includes a creation module 810 , a sampling operation module 820 , a correction module 830 , and a cutting module 840 .

[0145] In a specific embodiment, the creation module 810 obtains an image to be processed, wherein the image to be processed includes a chest X-ray image and a dermoscopic detection image, and the image to be processed includes a target to be cut of a related medical anatomical structure. The sampling operation module 820 selects the initial point of the image to be processed, and starts to perform a sampling operation with the initial point as the starting point and the initial sampling area generated by the initial point until a cutting trajectory including the target to be cut is generated, wherein the sampling area obtained by each sampling operation is input into a pre-trained deep learning model, so that the sampling direction of the next displacement corresponding to the next time step is determined according to the moving direction of the current time step to guide the sampling operation of the next time step according to the determined sampling direction. The correction module 830 calculates the exponential moving average in real time to determine the deviation of each sampling according to the generated sampling area corresponding to each sampling operation, so as to correct each sampling direction, so that the generated cutting trajectory is optimized while generating the cutting trajectory on the image to be processed. The cutting module 840 cuts the image to be processed according to the optimized cutting trajectory to obtain a target image including the target to be cut.

[0146] According to an optional implementation, an initial point of the image to be processed is selected, recursive sampling is performed on the initial point, and correction processing is performed a predetermined number of times to obtain a displacement corresponding to the initial point after the correction processing, specifically including performing a sampling operation on the initial point a predetermined number of times, and adjusting the sampling direction of the initial sampling area formed based on the initial point a predetermined number of times to obtain the sampling direction of the corrected initial sampling area; recursive sampling is performed on each trajectory point to be formed on the image to be processed, and correction processing is performed a predetermined number of times to obtain the displacement corresponding to each corrected trajectory point.

[0147] According to an optional implementation manner, when the predetermined number of times is C, the sampling direction of each trajectory point on the image to be processed is corrected C times, and the corrected displacement difference between the current time step t and the time step ti is calculated using the following expression:

[0148]

[0149] Among them, CDO t It represents the corrected displacement difference that the navigation agent needs to correct from the current trajectory point corresponding to the current time step t on the image to be processed to the next trajectory point, where t represents the current time step; S(C) is a sigmoid function with log, that is, It is used to characterize the weight at each time step, C represents the number of recursive sampling at the same trajectory point; v t is the displacement at time t or the current time step, v t-i represents the displacement at time ti, i is a positive integer, i represents the number of steps of forward displacement starting from the current time t; Δv t-i Indicates v t-i With v t-i-1 difference.

[0150] According to an optional embodiment, when the image to be processed is a chest X-ray image, the predetermined number of recursive samplings C is in the range of 5 to 20 times, preferably 15 times; when the image to be processed is a cardiac MRI image, the predetermined number of recursive samplings C is in the range of 5 to 20 times, preferably 15 times; when the image to be processed is a dermoscopy image, the predetermined number of recursive samplings C is in the range of 10 to 20 times, preferably 15 times.

[0151] According to an optional implementation, according to the current trajectory point corresponding to the current time step t, the calculated corrected displacement difference CDO t , correct the displacement from the current trajectory point to the next trajectory point:

[0152] v' t =v t +CDO t

[0153] Among them, v t Indicates the displacement from the current trajectory point to the next trajectory point; v' t Indicates the corrected displacement from the current track point to the next track point; CDO t It represents the corrected displacement difference that the navigation agent needs to correct from the current trajectory point corresponding to the current time step t on the image to be processed to the next trajectory point, where t represents the current time step.

[0154] According to an optional implementation manner, further comprising: the recursive sampling specifically comprises executing the following steps:

[0155] Step S201: when the navigation agent moves to the position of the current track point, the navigation agent first cuts out the image block corresponding to the sampling area from the image to be processed according to the displacement of the previous track point along the direction of the previous displacement;

[0156] Step S202: input the image block corresponding to the intercepted sampling area into a pre-trained deep learning model, output a temporary sampling direction, and adjust the sampling area corresponding to the current trajectory point to a sampling direction consistent with the temporary sampling direction;

[0157] Step S203: According to the number of recursive sampling, step S202 is repeatedly executed for the current trajectory point to output the temporary sampling direction of the corresponding number of times. When the predetermined number of recursive sampling is reached, the temporary displacement output for the last time, i.e., the temporary sampling direction, is used as the moving direction to the next trajectory point and the sampling direction of the next trajectory point.

[0158] According to an optional embodiment, it further comprises: determining whether the generation process of the cutting trajectory including the target to be cut is completed according to the convergence limiting condition, wherein:

[0159] The convergence limitation conditions include determining a detection line. In the process of generating a cutting trajectory including the target to be cut, an interval line is formed according to the intersection of the generated trajectory line and the detection line. The drawn interval line is further compared with a preset distance to determine whether the process of generating the cutting trajectory including the target to be cut is completed.

[0160] According to an optional implementation, the method further includes: moving to the next step according to the current trajectory point t , the calculated exponential moving average EMA t , calculate the corrected displacement v' corresponding to the current trajectory point to the next trajectory point t :

[0161] v' t =v t +EMA t

[0162] Among them, v t Indicates the displacement of the current trajectory to the next step; v' t Indicates the corrected displacement from the current trajectory point to the next trajectory point; EMA t It represents the corrected displacement difference that the navigation agent needs to correct for the current trajectory point corresponding to the current time step t on the image to be processed, where t represents the current time step.

[0163] It should be noted that in Figure 8In the example, the method for generating a cutting trajectory of a target to be cut in a medical image performed by the device for generating a cutting trajectory of a target to be cut in a medical image is the same as Figure 1 The contents of the method for generating cutting trajectories of medical image targets to be cut in the examples are roughly the same, so the description of the same parts is omitted.

[0164] Compared with the prior art, the present invention selects the initial point of the image to be processed, and takes the initial point as the starting point, takes the initial point as the starting point of the navigation agent, guides the navigation agent to generate trajectory points, until a cutting trajectory containing the target to be cut is generated, and determines the deviation of each sampling in real time according to the generated sampling area corresponding to each sampling operation to correct each sampling direction, so that while generating the cutting trajectory on the image to be processed, the generated cutting trajectory is optimized; the image to be processed is cut according to the optimized cutting trajectory to obtain a target image containing the target to be cut, and the segmentation accuracy is improved while optimizing the cutting trajectory of the target to be cut, and the problem of low segmentation accuracy caused by the cumulative error introduced by the neural network method due to the difference between the discrete system and the continuous system is solved.

[0165] In addition, the present invention recursively samples each trajectory point on the image to be processed (i.e., samples the same trajectory point a predetermined number of times), corrects the displacement from the current trajectory point to the next trajectory point (i.e., the moving direction or sampling direction) according to the current trajectory point corresponding to the current time step and the calculated corrected displacement difference, and adjusts the predetermined number of recursive samplings for different medical image applications, thereby obtaining a more accurate target trajectory to be cut and further improving the segmentation accuracy.

[0166] In addition, if Fig. 9 As shown, it also includes an electronic device corresponding to the device for generating a cutting trajectory of a target to be cut in a medical image, and the electronic device 1000 includes a processor 1200, a memory 1300, and other circuits 1400, and the memory 1300 stores a computer executable program, which is usually a machine-readable code. The computer-readable program can be executed by the processor 1200 to enable the electronic device to perform the method of the present invention, or at least some steps in the method. In addition, the electronic device 1000 also includes other modules 1500.

[0167] The memory 1300 includes a volatile memory, such as a random access memory unit (RAM) and / or a cache memory unit, and may also be a non-volatile memory, such as a read-only memory unit (ROM).

[0168] Optionally, in this embodiment, the electronic device further includes an I / O interface, which is used for the electronic device to exchange data with an external device. The I / O interface can represent one or more of several types of bus structures, including a storage unit bus or a storage unit controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of a variety of bus structures.

[0169] Those skilled in the art will appreciate that the above modules can be distributed in the device according to the description of the embodiment, or can be changed accordingly and only used in one or more devices different from the embodiment. The modules of the above embodiments can be combined into one module, or further divided into multiple sub-modules.

[0170] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the embodiment of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several commands to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the embodiment of the present invention.

[0171] The exemplary embodiments of the present invention are specifically shown and described above. It should be understood that the present invention is not limited to the detailed structure, configuration or implementation method described herein; on the contrary, the present invention is intended to cover various modifications and equivalent configurations included in the spirit and scope of the appended claims.

Claims

1. A method for generating a cutting trajectory of a target to be cut in a medical image, characterized in that: include: Acquire an image to be processed, wherein the image to be processed includes a chest X-ray image, a cardiac MRI image, and a dermoscopic detection image, and the image to be processed contains a target to be cut of a related medical anatomical structure; Selecting an initial point of the image to be processed, and using the initial point as the starting point of the navigation agent, guiding the navigation agent to generate trajectory points until a cutting trajectory including the target to be cut is generated, specifically comprising: performing a sampling operation on each trajectory point to obtain a sampling area, inputting the obtained sampling area into a pre-trained deep learning model to obtain a displacement from each trajectory point to the next trajectory point, so that the navigation agent generates continuous trajectory points to include the target to be cut; According to the generated trajectory points and the sampling area corresponding to each sampling operation, the deviation of each sampling is calculated and determined in real time to correct the direction of each sampling, so that the generated cutting trajectory is optimized while generating the cutting trajectory on the image to be processed; The image to be processed is cut according to the optimized cutting trajectory to obtain a target image containing the target to be cut.

2. The method for generating a cutting trajectory of a target to be cut in a medical image according to claim 1, characterized in that: Further including: Selecting an initial point of the image to be processed, performing recursive sampling on the initial point, and performing a correction process for a predetermined number of times to obtain a displacement corresponding to the initial point after the correction process, specifically including performing a sampling operation on the initial point for a predetermined number of times, and adjusting a sampling direction of an initial sampling area formed based on the initial point for a predetermined number of times to obtain a sampling direction of the corrected initial sampling area; Recursive sampling is performed on each track point to be formed on the processed image, and a predetermined number of correction processes are performed to obtain the displacement corresponding to each track point after correction.

3. The method for generating a cutting trajectory of a target to be cut in a medical image according to claim 2, characterized in that: Further including: When the predetermined number is C, the sampling direction of each trajectory point on the processed image is corrected C times, and the corrected displacement difference between the current time step t and the time step ti is calculated using the following expression: Among them, CDO t It represents the corrected displacement difference that the navigation agent needs to correct from the current trajectory point corresponding to the current time step t on the image to be processed to the next trajectory point, where t represents the current time step; S(C) is a sigmoid function with log, that is, It is used to characterize the weight at each time step, C represents the number of recursive sampling at the same trajectory point; v t is the displacement at time t or the current time step, v t-i represents the displacement at time ti, i is a positive integer, specifically 1, 2, ..., n, i represents the number of steps of displacement from the current time t forward, when i = n, it means that the number of steps of displacement from the current time t forward is n; Δv t-i Indicates v t-i With v t-i-1 difference.

4. The method for generating a cutting trajectory of a target to be cut in a medical image according to claim 2, characterized in that: When the image to be processed is a chest X-ray image, the predetermined number C of recursive sampling is in the range of 5 to 20 times; When the image to be processed is a cardiac MRI image, the predetermined number C of recursive sampling is in the range of 5 to 20 times; When the image to be processed is a dermoscopic detection image, the predetermined number C of recursive sampling is in the range of 10 to 20 times.

5. The method for generating a cutting trajectory of a target to be cut in a medical image according to claim 2, characterized in that: According to the current trajectory point corresponding to the current time step t, the calculated corrected displacement difference CDO t , correct the displacement from the current trajectory point to the next trajectory point: in t =in t +CDO t Among them, v t Indicates the displacement from the current trajectory point to the next trajectory point; v' t Indicates the corrected displacement from the current track point to the next track point; CDO t It represents the corrected displacement difference that the navigation agent needs to correct from the current trajectory point corresponding to the current time step t on the image to be processed to the next trajectory point, where t represents the current time step.

6. The method for generating a cutting trajectory of a target to be cut in a medical image according to claim 2, characterized in that: Further including: The recursive sampling specifically includes executing the following steps: Step S201: when the navigation agent moves to the position of the current track point, the navigation agent first cuts out the image block corresponding to the sampling area from the image to be processed according to the displacement of the previous track point along the direction of the previous displacement; Step S202: input the image block corresponding to the intercepted sampling area into a pre-trained deep learning model, output a temporary sampling direction, and adjust the sampling area corresponding to the current trajectory point to a sampling direction consistent with the temporary sampling direction; Step S203: According to the number of recursive sampling, step S202 is repeatedly executed for the current trajectory point to output the temporary sampling direction of the corresponding number of times. When the predetermined number of recursive sampling is reached, the temporary displacement output for the last time, i.e., the temporary sampling direction, is used as the moving direction to the next trajectory point and the sampling direction of the next trajectory point.

7. The method for generating a cutting trajectory of a target to be cut in a medical image according to claim 1, characterized in that: Further including: According to the convergence limit condition, it is determined whether the generation process of the cutting trajectory including the target to be cut is completed, wherein: The convergence limitation conditions include determining a detection line. In the process of generating a cutting trajectory including the target to be cut, an interval line is formed according to the intersection of the generated trajectory line and the detection line. The drawn interval line is further compared with a preset distance to determine whether the process of generating the cutting trajectory including the target to be cut is completed.

8. The method for generating a cutting trajectory of a target to be cut in a medical image according to claim 1, characterized in that: Also includes: According to the current trajectory point to the next displacement v t , the calculated exponential moving average EMA t , calculate the corrected displacement v' corresponding to the current trajectory point to the next trajectory point t : v' t =v t +EMA t Among them, v t Indicates the displacement of the current trajectory to the next step; v' t Indicates the corrected displacement from the current trajectory point to the next trajectory point; EMA t It represents the corrected displacement difference that the navigation agent needs to correct for the current trajectory point corresponding to the current time step t on the image to be processed, where t represents the current time step.

9. A device for generating a cutting trajectory of a target to be cut in a medical image, using the method for generating a cutting trajectory of a target to be cut in a medical image according to any one of claims 1 to 8, characterized in that: The device for generating a cutting trajectory of a target to be cut in a medical image comprises: Creating a module to obtain an image to be processed, wherein the image to be processed includes a chest X-ray image, a cardiac MRI image, and a dermoscopy detection image, and the image to be processed contains a target to be cut of a related medical anatomical structure; A sampling operation module selects an initial point of the image to be processed and uses the initial point as the starting point of the navigation agent to guide the navigation agent to generate trajectory points until a cutting trajectory including the target to be cut is generated, specifically including: performing a sampling operation on each trajectory point to obtain a sampling area, inputting the obtained sampling area into a pre-trained deep learning model to obtain a displacement from each trajectory point to the next trajectory point, so that the navigation agent generates continuous trajectory points to include the target to be cut; A correction module, which calculates and determines the deviation of each sampling in real time according to the generated trajectory points and the sampling area corresponding to each sampling operation, so as to correct the direction of each sampling, so as to optimize the generated cutting trajectory while generating the cutting trajectory on the image to be processed; The cutting module cuts the image to be processed according to the optimized cutting trajectory to obtain a target image containing the target to be cut.

10. The device for generating cutting trajectory of a target to be cut in a medical image according to claim 9, characterized in that: Further including: Selecting an initial point of the image to be processed, performing recursive sampling on the initial point, and performing a correction process for a predetermined number of times to obtain a displacement corresponding to the initial point after the correction process, specifically including performing a sampling operation on the initial point for a predetermined number of times, and adjusting a sampling direction of an initial sampling area formed based on the initial point for a predetermined number of times to obtain a sampling direction of the corrected initial sampling area; Recursive sampling is performed on each track point to be formed on the processed image, and a predetermined number of correction processes are performed to obtain the displacement corresponding to each track point after correction.

Images