A tumor cell diffusion target tracking method based on multi-focus fusion

Abstract

The present invention relates to the field of cell diffusion tracking technology, and discloses a tumor cell diffusion target tracking method based on multi-focus fusion, comprising the following steps: sequentially acquiring a first image set at a set of continuous time points; in the first image set at each time point, screening the first image set according to the focus evaluation result of the first image to obtain a second image; obtaining a third image through a multi-focus fusion model of the image; in the third image, connecting the local areas representing the tumor cells according to the time points to form a local diffusion path of the tumor cells, and according to the local diffusion path of the tumor cells, using a neural network to perform local path learning to obtain a global diffusion path of the tumor cells. The present invention can intuitively and statically display the dynamic diffusion of tumors, realize the predictability of the diffusion of tumor cells, grasp the future diffusion of tumor cells in advance, and reduce the limitations of diffusion tracking.

Description

Technical Field

[0001] The present invention relates to the field of cell diffusion tracking, and in particular to a tumor cell diffusion target tracking method based on multi-focus fusion. Background Art

[0002] In the field of cell tracking, the popular cell tracking algorithms can be roughly divided into two categories: generative tracking and discriminative tracking. The generative tracking method is also called the tracking method based on model update. The premise of this algorithm is to model the cell area in the current frame and obtain the mathematical model of the target. The next frame predicts the cell position by finding the area most similar to the model. The more famous algorithms include the use of mean filtering, Gaussian mixture and particle filtering methods to track cells. However, these methods take a lot of time to match the model, and it is difficult to train a representative network model in the case of low contrast and small amount of cell data, and the feasibility is not strong. The discriminative tracking method is also called the detection-based tracking method. The classic steps of this method are to extract the features of the image, and then use machine learning such as SVM to train the classifier with background learning to classify the target area. The algorithm defines the target area of ​​the current frame as a positive sample and the background area as a negative sample. The machine learning then uses the trained classifier to find the optimal area of ​​the next frame to match the target.

[0003] Although the existing technology can use images to track the spread of cancer cells, it is difficult to accurately display the spread of tumor cells on the same image during image processing, so it is difficult to intuitively display the spread of tumors. At the same time, existing cell tracking is lagging tracking, which makes it difficult to achieve predictability of tumor cell spread, and thus it is impossible to predict the future spread of tumor cells in advance. Diffusion tracking has limitations. Summary of the invention

[0004] The purpose of the present invention is to provide a tumor cell diffusion target tracking method based on multi-focus fusion to solve the technical problems that it is difficult to intuitively display the tumor diffusion situation in the prior art, and the existing cell tracking is all lagging tracking, which makes it difficult to achieve predictability of tumor cell diffusion and unable to predict the future diffusion of tumor cells in advance, and the diffusion tracking has limitations.

[0005] In order to solve the above technical problems, the present invention specifically provides the following technical solutions:

[0006] A tumor cell diffusion target tracking method based on multi-focus fusion includes the following steps:

[0007] Sequentially acquiring a first image set at a set of continuous time points, wherein the first image set at each time point includes a plurality of first images, and the first images correspond to medical imaging data of physiological tissues containing tumor cells;

[0008] In the first image set at each time point, focusing evaluation is performed on a local area representing tumor cells in each first image to obtain a focusing evaluation result of the first image;

[0009] According to the focus evaluation result of the first image, the first image set is screened to obtain a second image, wherein the second image corresponds to the first image in which the local area representing the tumor cell is a focus area;

[0010] According to the second image at each time point, a third image is obtained through a multi-focus fusion model of the image, wherein the third image corresponds to a fusion result of the focus area in the second image at each time point, and the multi-focus fusion model is a neural network;

[0011] In the third image, local areas representing tumor cells are connected according to time-sequential points to form a local diffusion path of the tumor cells, and based on the local diffusion path of the tumor cells, a neural network is used to perform local path learning to obtain a global diffusion path of the tumor cells, wherein the global diffusion path corresponds to the path of tumor cell diffusion target tracking.

[0012] As a preferred solution of the present invention, the type of medical imaging data includes at least one of CT image data, MRI image data and ultrasound image data.

[0013] As a preferred solution of the present invention, the focusing evaluation of the local area representing the tumor cells in each first image includes:

[0014] Passing the first image through a low-pass filter in the frequency domain to obtain a low-frequency filtered image of the first image;

[0015] Performing differential processing on the low-frequency filtered image of the first image and the first image to obtain a first differential image;

[0016] Inputting the first difference image and the first image into a parameter estimation network to determine guided filtering operation parameters of the first difference image;

[0017] Constructing a guided filtering operation formula according to the guided filtering operation parameters of the first difference image, and performing guided filtering on the first difference image by using the guided filtering operation formula to obtain a focus representation map of the first image;

[0018] In the focus representation image of the first image, counting the pixel area of ​​the focus region of the first image;

[0019] In the first image, counting the pixel areas representing the local area of ​​the tumor cells;

[0020] taking the ratio of the pixel area of ​​the focused area in the first image to the pixel area of ​​the local area representing the tumor cells as the focus evaluation result of the local area representing the tumor cells in the first image;

[0021] Wherein, the establishment of the parameter calculation network includes:

[0022] Randomly selecting a plurality of first differential images and first images as first sample images and second sample images respectively, and setting a set of preselected values ​​of guided filter operation parameters by using a gradient descent method;

[0023] Matching each preselected value of the guided filter parameter one by one to form a set of optional formulas of the guided filter operation formula;

[0024] Performing guided filtering on the first sample image by a set of optional formulas to obtain a focus representation map of the second sample image, quantifying the pixel difference between the focus area and the non-focus area of ​​the second sample image in the focus representation map of the second sample image, and obtaining a focus enhancement result of the second sample image;

[0025] In each second sample image, a neural network is used to perform learning and training on a set of preselected values ​​of the guided filtering operation parameters and the focus enhancement results of the second sample image, so as to obtain a focus enhancement effect evaluation network that characterizes the mapping relationship between the preselected values ​​of the guided filtering operation parameters and the focus enhancement results;

[0026] Using the focus enhancement effect evaluation network to evaluate the selected value range of the pre-selected value of the guided filter operation parameter one by one, the selected value range of the focus enhancement result of the second sample image is obtained, and the pre-selected value of the guided filter operation parameter corresponding to the highest focus enhancement result is used as the optimal guided filter operation parameter of the second sample image;

[0027] Using a neural network to learn and train the first sample image, the second sample image and the optimal guided filtering operation parameters of the second sample image to obtain the parameter measurement network;

[0028] The expression of the focus enhancement effect evaluation network is:

[0029] Q=BP(filter_parameter); where Q is the focus enhancement effect, filter_parameter is the guided filter operation parameter, and BP is the BP neural network;

[0030] The expression of the parameter calculation network is: filter_parameter =BP(S,Sc);

[0031] Where filter_parameter is the guided filtering operation parameter, S and Sc are the first sample image and the second sample image respectively, and BP is the BP neural network.

[0032] As a preferred solution of the present invention, acquiring the second image includes:

[0033] The focus evaluation results of each first image in the first image set at each sequential point are compared, and the first image corresponding to the maximum focus evaluation result in the first image set is marked as the second image at the corresponding sequential point.

[0034] As a preferred solution of the present invention, acquiring the third image includes:

[0035] Obtaining a focus representation image corresponding to the second image at each sequential point;

[0036] generating a fusion decision matrix through a decision operation according to the focus representation map corresponding to the second image;

[0037] Inputting the second image at each sequential point and the fusion decision matrix into a multi-focus fusion model to obtain a fusion weight of the second image at each sequential point;

[0038] The second images at each time point are weighted averaged according to the fusion weight to obtain the third image.

[0039] As a preferred solution of the present invention, the establishment of the multi-focus fusion model includes:

[0040] Randomly selecting a plurality of second images as third sample images, and obtaining a fusion decision matrix of the third sample image according to a focus representation map of the third sample image;

[0041] Randomly setting multiple groups of fusion weights of the third sample image;

[0042] Performing a weighted average operation on the third sample image using multiple groups of fusion weights to obtain a fusion image of the multiple third sample images;

[0043] performing a focus evaluation on a local area representing tumor cells in the fused image of the third sample image to obtain a focus evaluation result of the local area representing tumor cells in the fused image of the third sample image;

[0044] taking the difference between the focus evaluation result of the local area representing the tumor cells in the fused image of the third sample image and the sum of the focus evaluation results of the local areas representing the tumor cells corresponding to each of the second images as the fusion evaluation result of the third sample image;

[0045] Using a neural network to perform learning and training on the fusion weight of the third sample image and the fusion evaluation result of the third sample image, so as to obtain a fusion evaluation network of the third sample image;

[0046] Using the fusion evaluation network to perform evaluations one by one in the selected value range of the fusion weight, a selected value range of the fusion evaluation result of the third sample image is obtained, and the fusion weight corresponding to the highest fusion evaluation result is used as the optimal fusion evaluation result of the third sample image;

[0047] Using a neural network to perform learning and training on a fusion decision matrix of the third sample image and an optimal fusion evaluation result of the third sample image, to obtain the multi-focus fusion model;

[0048] The expression of the fusion evaluation network is: P=BP(W);

[0049] In the formula, P is the fusion evaluation result, W is the fusion weight, and BP is the BP neural network;

[0050] The expression of the multi-focus fusion model is:

[0051] W = BP(Sd);

[0052] Where W is the fusion weight, Sd is the fusion decision matrix, and BP is the BP neural network.

[0053] As a preferred solution of the present invention, the generation of the fusion decision matrix includes:

[0054] Perform pixel-by-pixel decision operations on the focus representation map corresponding to the second image at each time point, correspond the maximum value of the pixel in the focus representation map to the second image, retain the pixel in the second image corresponding to the maximum value of the pixel in the focus representation map, and obtain a decision matrix for the second image;

[0055] Performing matrix summation on the decision matrices of the second image at each time point to obtain the fusion decision matrix;

[0056] The decision operation formula is: ; ; In the formula, R t (x, y) is the value of the x row and y column in the decision matrix corresponding to the (x, y) coordinate point in the second image at the t-th time point, bl is the identifier of the pixel that retains the meaning, nl is the identifier of the pixel that does not retain the meaning, f t (x, y) is the pixel value at the (x, y) coordinate point in the corresponding focus representation map in the second image at the tth time point, f k (x, y) is the pixel value at the (x, y) coordinate point in the focus representation corresponding to the second image at the kth time point, max is the maximum value operator, is the maximum value of the pixel at the (x, y) coordinate point in the corresponding focus representation diagram in the second image at each sequential point, N is the total number of second images or the total number of sequential points, and t, k are counting variables;

[0057] R(x,y) is the value of row x and column y in the fusion decision matrix. As a preferred solution of the present invention, determining the global diffusion path of tumor cells includes:

[0058] The local area representing the tumor cells at the preceding time point in the local diffusion path of the tumor cells is used as the input item of the LSTM neural network, and the local area representing the tumor cells at the following time point in the local diffusion path of the tumor cells is used as the output item of the LSTM neural network;

[0059] Using the LSTM neural network to perform network training on the input items of the LSTM neural network and the output items of the LSTM neural network to obtain the global diffusion path;

[0060] The network expression of the global diffusion path is:

[0061] O future =LSTM(O old );

[0062] In the formula, O future is the local area representing tumor cells at the later time point, O old is a local area representing tumor cells at the preceding time point, and LSTM is a neural network. As a preferred solution of the present invention, the decision matrix is ​​consistent with the matrix specification of the pixel matrix of the second image.

[0063] As a preferred solution of the present invention, the first images at various time points have the same medical imaging settings.

[0064] Compared with the prior art, the present invention has the following beneficial effects:

[0065] The present invention sequentially focuses on identifying and fusion-focusing local areas representing tumor cells at various time points, thereby accurately displaying the diffusion of tumor cells on the same image. The dynamic diffusion of tumors can be intuitively displayed statically. At the same time, based on the static display of the dynamic diffusion of tumors, the global diffusion path is learned to achieve predictability of the diffusion of tumor cells, grasp the future diffusion of tumor cells in advance, and reduce the limitations of diffusion tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the implementation methods or the description of the prior art. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.

[0067] Figure 1 A tumor cell diffusion target tracking method based on multi-focus fusion provided in an embodiment of the present invention;

[0068] Figure 2 An image including a focused area and a non-focused area provided by an embodiment of the present invention;

[0069] Figure 3 The embodiment of the present invention provides Figure 2 An image with different focused and unfocused areas;

[0070] Figure 4 The embodiment of the present invention provides Figure 2 and Figure 3 Fusion decision matrix of fusion;

[0071] Figure 5 The embodiment of the present invention provides Figure 2 and Figure 3 Fusion results of the focused area. DETAILED DESCRIPTION

[0072] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0073] It is difficult to accurately display the diffusion of tumor cells on the same image during image processing, so it is difficult to intuitively display the diffusion of tumors. At the same time, existing cell tracking is lagging tracking, which makes it difficult to achieve predictability of tumor cell diffusion, and thus it is impossible to grasp the future diffusion of tumor cells in advance. Diffusion tracking has limitations. Therefore, the present invention provides a tumor cell diffusion target tracking method based on multi-focus fusion, which can intuitively statically display the dynamic diffusion of tumors by focusing on and fusing the local areas representing tumor cells at each time point in turn. At the same time, on the basis of statically displaying the dynamic diffusion of tumors, the global diffusion path is learned to grasp the future diffusion of tumor cells in advance.

[0074] like Figure 1 As shown, the present invention provides a tumor cell diffusion target tracking method based on multi-focus fusion, comprising the following steps:

[0075] Sequentially acquiring a first image set at a set of continuous time points, wherein the first image set at each time point includes a plurality of first images, and the first images correspond to medical imaging data of physiological tissues containing tumor cells;

[0076] In the first image set at each time point, focusing evaluation is performed on a local area representing tumor cells in each first image to obtain a focusing evaluation result of the first image;

[0077] According to the focus evaluation result of the first image, the first image set is screened to obtain a second image, where the second image corresponds to the first image in which the local area representing the tumor cell is a focus area;

[0078] According to the second image at each time point, a third image is obtained through a multi-focus fusion model of the image, the third image corresponds to the fusion result of the focus area in the second image at each time point, and the multi-focus fusion model is a neural network;

[0079] In the third image, local areas representing tumor cells are connected according to time-sequential points to form a local diffusion path of the tumor cells, and based on the local diffusion path of the tumor cells, a neural network is used to perform local path learning to obtain a global diffusion path of the tumor cells, wherein the global diffusion path corresponds to the path of tumor cell diffusion target tracking.

[0080] like Figure 2 and 3 As shown, the focused area corresponds to the image clarity of the area or the image detail display degree is high, and the non-focused area corresponds to the image clarity of the area or the image detail display degree is low, such as Figure 4 As shown, the fusion decision matrix is Figure 5 As shown, Figure 2 and Figure 3 Fusion results of the focused area.

[0081] In order to display the dynamic diffusion of tumor cells on a static image, the present invention fuses the medical imaging data of physiological tissues containing tumor cells at multiple continuous time points, thereby realizing local fusion of pixel areas representing tumor cells. The pixel areas representing tumor cells at multiple time points can be displayed intuitively and objectively in the same image, realizing static to dynamic information display, and achieving the purpose of displaying the dynamic diffusion of tumor cells on a static image.

[0082] In order to further ensure the accuracy of displaying the dynamic diffusion of tumor cells on static images, the present invention needs to ensure that the pixel areas representing tumor cells at each time point are accurately displayed in the static image. First, the pixel areas representing tumor cells at each time point are focused and identified to improve the recognition accuracy of the pixel areas representing tumor cells at each time point. Secondly, the pixel areas representing tumor cells at each time point are focused and fused to improve the fusion accuracy of the pixel areas representing tumor cells at each time point.

[0083] The present invention realizes identification of focused areas and non-focused areas of medical imaging data (first image) of physiological tissues containing tumor cells at various time points by utilizing low-pass filtering in combination with guided filtering (the focused area corresponds to a high image clarity or image detail display degree of the area, and the non-focused area corresponds to a low image clarity or image detail display degree of the area), and screens out medical imaging data (second image) that focuses on and expresses the main range of pixel areas representing tumor cells by evaluating the focused area of ​​the first image, so that the main pixel features of the pixel areas representing tumor cells are clearly displayed in the second image used for fusion, that is, focus identification can more accurately screen out the pixel areas representing tumor cells, provide the best image data for subsequent fusion, and is the basis for ensuring that the pixel areas representing tumor cells at various time points are accurately displayed in static images.

[0084] In the present invention, the evaluation of the focus area is performed by taking the ratio of the pixel area of ​​the focus area to the pixel area of ​​the local area representing the tumor cells as an evaluation index. The higher the evaluation index, the more inclusive the focus area of ​​the first image is for the pixel area representing the tumor cells, that is, the more concentrated the pixel area representing the tumor cells is in the focus area. Selecting a first image with a higher evaluation index indicates that the pixel area representing the tumor cells in the first image is mainly concentrated in the focus area, that is, the pixel area representing the tumor cells in the first image has the best clarity or detail display in the first image.

[0085] Moreover, a high evaluation index corresponds to that the pixel areas representing tumor cells in the first image are mainly concentrated in the focus area, and it cannot be completely guaranteed that all the pixel areas representing tumor cells are concentrated in the focus area. The present invention utilizes the selection of evaluation indexes to grasp the main distribution of tumor cells at each time point, and displays the main distribution of tumor cells at each time point in the same static image, showing the mainstream diffusion of tumor cells between continuous time series, grasping the mainstream diffusion trend of tumor cells, and actively ignoring the random diffusion of tumor cells, because the random diffusion of tumor cells has no positive significance for the tracking of tumor cells, and will only cause tracking interference. Therefore, the random diffusion of tumor cells is actively ignored and mapped to the distribution of tumor cells, which corresponds to the random distribution of tumor cells, that is, the pixel areas representing tumor cells that are not in the focus area.

[0086] The present invention selects the proportion of the pixel area of ​​the focused area to the pixel area of ​​the local area characterizing the tumor cells as an evaluation index, which can grasp the main distribution of tumor cells at each time point, actively ignore the random distribution of tumor cells at each time point, and realize the active neglect of the random diffusion of tumor cells, thereby avoiding the interference of the random diffusion of tumor cells on the tracking of tumor cells and ensuring the tracking accuracy of tumor cells.

[0087] Furthermore, in order to improve the accuracy of focused area recognition, the present invention uses a neural network to construct a setting network for guiding filter operation parameters, which can determine the optimal guided filter operation parameters for the first differential image at each time point, strengthen the difference between the focused area and the non-focused area, and make the recognition accuracy of the focused area stronger.

[0088] The present invention determines the optimal fusion weight of the second image at each time point through a multi-focus fusion model. The fused image (third image) obtained according to the optimal fusion weight can make the focused area (pixel area representing tumor cells) in the second image still present high clarity or high image detail display after fusion, thereby ensuring that the pixel area representing tumor cells at each time point is accurately displayed in the static image.

[0089] Furthermore, since the pixel areas representing tumor cells at each time point are accurately displayed in the static image, the local diffusion trend (local diffusion path) of the pixel areas representing tumor cells at each time point in the static image is learned by using a neural network to learn the local paths, thereby obtaining the global diffusion trend (global diffusion path) of the tumor cells. Therefore, the optimal fusion weight ensures that the focus area (pixel area representing tumor cells) in the second image still presents high clarity or high image detail display after fusion, which can make the representation of the local diffusion trend more accurate, thereby making the prediction of the global diffusion trend more accurate, and further ensuring the accuracy of tumor cell tracking.

[0090] The types of medical imaging data include at least one of CT image data, MRI image data and ultrasound image data.

[0091] The evaluation of the focus area in the present invention is based on the ratio of the pixel area of ​​the focus area to the pixel area of ​​the local area representing the tumor cells as an evaluation index, specifically as follows:

[0092] A focus evaluation is performed on a local area representing tumor cells in each first image, including:

[0093] Passing the first image through a low-pass filter in the frequency domain to obtain a low-frequency filtered image of the first image;

[0094] Performing differential processing on the low-frequency filtered image of the first image and the first image to obtain a first differential image;

[0095] Inputting the first difference image and the first image into a parameter estimation network to determine guided filtering operation parameters of the first difference image;

[0096] Constructing a guided filtering operation formula according to the guided filtering operation parameters of the first difference image, and performing guided filtering on the first difference image by using the guided filtering operation formula to obtain a focus representation map of the first image;

[0097] In the focus representation image of the first image, counting the pixel area of ​​the focus region of the first image;

[0098] In the first image, counting the pixel areas representing the local area of ​​the tumor cells;

[0099] taking the ratio of the pixel area of ​​the focused area in the first image to the pixel area of ​​the local area representing the tumor cells as the focus evaluation result of the local area representing the tumor cells in the first image;

[0100] The higher the evaluation index is, the more inclusive the focus area of ​​the first image is for the pixel area representing the tumor cells, that is, the pixel area representing the tumor cells is more concentrated in the focus area. Selecting a first image with a higher evaluation index means that the pixel area representing the tumor cells in the first image is mainly concentrated in the focus area, that is, the pixel area representing the tumor cells in the first image has the best clarity or detail display in the first image.

[0101] Moreover, a high evaluation index corresponds to that the pixel areas representing tumor cells in the first image are mainly concentrated in the focus area, and it cannot be completely guaranteed that all the pixel areas representing tumor cells are concentrated in the focus area. The present invention utilizes the selection of evaluation indexes to grasp the main distribution of tumor cells at each time point, and displays the main distribution of tumor cells at each time point in the same static image, showing the mainstream diffusion of tumor cells between continuous time series, grasping the mainstream diffusion trend of tumor cells, and actively ignoring the random diffusion of tumor cells, because the random diffusion of tumor cells has no positive significance for the tracking of tumor cells, and will only cause tracking interference. Therefore, the random diffusion of tumor cells is actively ignored and mapped to the distribution of tumor cells, which corresponds to the random distribution of tumor cells, that is, the pixel areas representing tumor cells that are not in the focus area.

[0102] The present invention selects the proportion of the pixel area of ​​the focused area to the pixel area of ​​the local area characterizing the tumor cells as an evaluation index, which can grasp the main distribution of tumor cells at each time point, actively ignore the random distribution of tumor cells at each time point, and realize the active neglect of the random diffusion of tumor cells, thereby avoiding the interference of the random diffusion of tumor cells on the tracking of tumor cells and ensuring the tracking accuracy of tumor cells.

[0103] Furthermore, in order to improve the accuracy of focusing area recognition, the present invention uses a neural network to construct a setting network for guiding filter operation parameters, which can determine the optimal guiding filter operation parameters at the first differential image at each time point, strengthen the difference between the focusing area and the non-focusing area, and make the recognition accuracy of the focusing area stronger, as follows:

[0104] Among them, the establishment of the parameter calculation network includes:

[0105] Randomly selecting a plurality of first differential images and first images as first sample images and second sample images respectively, and setting a set of preselected values ​​of guided filter operation parameters by using a gradient descent method;

[0106] Matching each preselected value of the guided filter parameter one by one to form a set of optional formulas of the guided filter operation formula;

[0107] Performing guided filtering on the first sample image by a set of optional formulas to obtain a focus representation map of the second sample image, quantifying the pixel difference between the focus area and the non-focus area of ​​the second sample image in the focus representation map of the second sample image, and obtaining a focus enhancement result of the second sample image;

[0108] In each second sample image, a neural network is used to perform learning and training on a set of preselected values ​​of the guided filtering operation parameters and the focus enhancement results of the second sample image, so as to obtain a focus enhancement effect evaluation network that characterizes the mapping relationship between the preselected values ​​of the guided filtering operation parameters and the focus enhancement results;

[0109] Using the focus enhancement effect evaluation network to evaluate the selected value range of the pre-selected value of the guided filter operation parameter one by one, the selected value range of the focus enhancement result of the second sample image is obtained, and the pre-selected value of the guided filter operation parameter corresponding to the highest focus enhancement result is used as the optimal guided filter operation parameter of the second sample image;

[0110] Using a neural network to learn and train the optimal guided filtering operation parameters of the first sample image and the second sample image and the second sample image to obtain a parameter measurement network;

[0111] The expression of the focus enhancement effect evaluation network is:

[0112] Q=BP(filter_parameter); where Q is the focus enhancement effect, filter_parameter is the guided filter operation parameter, and BP is the BP neural network;

[0113] The expression of the parameter estimation network is:

[0114] filter_parameter =BP(S,Sc); where filter_parameter is the guided filtering operation parameter, S, Sc are the first sample image and the second sample image respectively, and BP is the BP neural network.

[0115] Acquiring a second image, comprising:

[0116] The focus evaluation results of each first image in the first image set at each sequential point are compared, and the first image corresponding to the maximum focus evaluation result in the first image set is marked as the second image at the corresponding sequential point.

[0117] The present invention determines the optimal fusion weight of the second image at each time point through a multi-focus fusion model. The fused image (third image) obtained according to the optimal fusion weight can make the focus area (pixel area representing tumor cells) in the second image still present high definition or high image detail display after fusion, thereby ensuring that the pixel area representing tumor cells at each time point is accurately displayed in the static image, as follows:

[0118] Acquire a third image, including:

[0119] Obtaining a focus representation image corresponding to the second image at each sequential point;

[0120] generating a fusion decision matrix through a decision operation according to the focus representation map corresponding to the second image;

[0121] Inputting the second image at each time point and the fusion decision matrix into the multi-focus fusion model to obtain the fusion weight of the second image at each time point;

[0122] The second images at each time point are weighted averaged according to the fusion weights to obtain a third image.

[0123] The establishment of multi-focus fusion model includes:

[0124] Randomly selecting a plurality of second images as third sample images, and obtaining a fusion decision matrix of the third sample image according to a focus representation map of the third sample image;

[0125] Randomly setting multiple groups of fusion weights of the third sample image;

[0126] Performing a weighted average operation on the third sample image using multiple groups of fusion weights to obtain a fusion image of the multiple third sample images;

[0127] performing a focus evaluation on a local area representing tumor cells in the fused image of the third sample image to obtain a focus evaluation result of the local area representing tumor cells in the fused image of the third sample image;

[0128] taking the difference between the focus evaluation result of the local area representing the tumor cells in the fused image of the third sample image and the sum of the focus evaluation results of the local areas representing the tumor cells corresponding to each of the second images as the fusion evaluation result of the third sample image;

[0129] Using a neural network to perform learning and training on the fusion weight of the third sample image and the fusion evaluation result of the third sample image, so as to obtain a fusion evaluation network of the third sample image;

[0130] Using the fusion evaluation network to perform evaluations one by one in the selected value range of the fusion weight, a selected value range of the fusion evaluation result of the third sample image is obtained, and the fusion weight corresponding to the highest fusion evaluation result is used as the optimal fusion evaluation result of the third sample image;

[0131] Using a neural network to perform learning and training on a fusion decision matrix of the third sample image and an optimal fusion evaluation result of the third sample image, thereby obtaining a multi-focus fusion model;

[0132] The expression of the fusion evaluation network is:

[0133] P = BP (W);

[0134] In the formula, P is the fusion evaluation result, W is the fusion weight, and BP is the BP neural network;

[0135] The expression of the multi-focus fusion model is:

[0136] W = BP(Sd);

[0137] Where W is the fusion weight, Sd is the fusion decision matrix, and BP is the BP neural network.

[0138] The generation of the fusion decision matrix includes:

[0139] Perform pixel-by-pixel decision operations on the focus representation map corresponding to the second image at each time point, correspond the maximum value of the pixel in the focus representation map to the second image, retain the pixel in the second image corresponding to the maximum value of the pixel in the focus representation map, and obtain a decision matrix for the second image;

[0140] Perform matrix summation on the decision matrices of the second image at each time point to obtain a fusion decision matrix;

[0141] The decision operation formula is: ; ; In the formula, R t (x, y) is the value of the x row and y column in the decision matrix corresponding to the (x, y) coordinate point in the second image at the t-th time point, bl is the identifier of the pixel that retains the meaning, nl is the identifier of the pixel that does not retain the meaning, ft (x, y) is the pixel value at the (x, y) coordinate point in the corresponding focus representation map in the second image at the tth time point, f k (x, y) is the pixel value at the (x, y) coordinate point in the focus representation corresponding to the second image at the kth time point, max is the maximum value operator, is the maximum value of the pixel at the (x, y) coordinate point in the corresponding focus representation diagram in the second image at each sequential point, N is the total number of second images or the total number of sequential points, and t, k are counting variables;

[0142] R(x,y) is the value of row x and column y in the fusion decision matrix.

[0143] In order to retain the pixels representing the tumor cells in the second image at each time point, a fusion decision matrix is ​​determined by using a decision operation, so that the pixel areas representing the tumor cells at each time point can be fused into the same static image according to the fusion decision matrix.

[0144] Furthermore, since the pixel regions representing tumor cells at each time point are accurately displayed in the static image, the local diffusion trend (local diffusion path) of the pixel regions representing tumor cells at each time point in the static image is learned by using a neural network to learn the local path, and thus the global diffusion trend (global diffusion path) of the tumor cells is obtained. Therefore, the optimal fusion weight enables the focus area (pixel region representing tumor cells) in the second image to still present a high definition or high image detail display degree after fusion, which can make the representation of the local diffusion trend more accurate, and thus make the prediction of the global diffusion trend more accurate, and further ensure the accuracy of tumor cell tracking, as follows:

[0145] Determine the global spread pathways of tumor cells, including:

[0146] The local area representing the tumor cells at the preceding time point in the local diffusion path of the tumor cells is used as the input item of the LSTM neural network, and the local area representing the tumor cells at the following time point in the local diffusion path of the tumor cells is used as the output item of the LSTM neural network;

[0147] The LSTM neural network is used to train the input items and output items of the LSTM neural network to obtain the global diffusion path;

[0148] The network expression of the global diffusion path is:

[0149] O future =LSTM(O old );Wherein, O future is the local area representing tumor cells at the later time point, Oold is a local area representing tumor cells at the preceding time point, LSTM is a neural network, and the decision matrix is ​​consistent with the matrix specification of the pixel matrix of the second image.

[0150] The first images at each time point have the same medical imaging settings.

[0151] The present invention sequentially focuses on identifying and fusion-focusing local areas representing tumor cells at various time points, thereby accurately displaying the diffusion of tumor cells on the same image. The dynamic diffusion of tumors can be intuitively displayed statically. At the same time, based on the static display of the dynamic diffusion of tumors, the global diffusion path is learned to achieve predictability of the diffusion of tumor cells, grasp the future diffusion of tumor cells in advance, and reduce the limitations of diffusion tracking.

[0152] The above embodiments are only exemplary embodiments of the present application and are not intended to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and protection scope of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present application.

Claims

1. A tumor cell diffusion target tracking method based on multi-focus fusion, characterized by: The following steps are involved: Sequentially acquiring a first image set at a set of continuous time points, wherein the first image set at each time point includes a plurality of first images, and the first images correspond to medical imaging data of physiological tissues containing tumor cells; In the first image set at each time point, focusing evaluation is performed on a local area representing tumor cells in each first image to obtain a focusing evaluation result of the first image; According to the focus evaluation result of the first image, the first image set is screened to obtain a second image, wherein the second image corresponds to the first image in which the local area representing the tumor cell is a focus area; According to the second image at each time point, a third image is obtained through a multi-focus fusion model of the image, wherein the third image corresponds to a fusion result of the focus area in the second image at each time point, and the multi-focus fusion model is a neural network; In the third image, local areas representing tumor cells are connected according to time-sequential points to form local diffusion paths of tumor cells, and local path learning is performed using a neural network based on the local diffusion paths of tumor cells to obtain global diffusion paths of tumor cells, wherein the global diffusion path corresponds to the path of tumor cell diffusion target tracking; A focus evaluation is performed on a local area representing tumor cells in each first image, including: Passing the first image through a low-pass filter in the frequency domain to obtain a low-frequency filtered image of the first image; Performing differential processing on the low-frequency filtered image of the first image and the first image to obtain a first differential image; Inputting the first difference image and the first image into a parameter estimation network to determine guided filtering operation parameters of the first difference image; Constructing a guided filtering operation formula according to the guided filtering operation parameters of the first difference image, and performing guided filtering on the first difference image by using the guided filtering operation formula to obtain a focus representation map of the first image; In the focus representation image of the first image, counting the pixel area of ​​the focus region of the first image; In the first image, counting the pixel areas representing the local area of ​​the tumor cells; taking the ratio of the pixel area of ​​the focused area in the first image to the pixel area of ​​the local area representing the tumor cells as the focus evaluation result of the local area representing the tumor cells in the first image; Wherein, the establishment of the parameter calculation network includes: Randomly selecting a plurality of first differential images and first images as first sample images and second sample images respectively, and setting a set of preselected values ​​of guided filter operation parameters by using a gradient descent method; Matching each preselected value of the guided filter parameter one by one to form a set of optional formulas of the guided filter operation formula; Performing guided filtering on the first sample image by a set of optional formulas to obtain a focus representation map of the second sample image, quantifying the pixel difference between the focus area and the non-focus area of ​​the second sample image in the focus representation map of the second sample image, and obtaining a focus enhancement result of the second sample image; In each second sample image, a neural network is used to perform learning and training on a set of preselected values ​​of the guided filtering operation parameters and the focus enhancement results of the second sample image, so as to obtain a focus enhancement effect evaluation network that characterizes the mapping relationship between the preselected values ​​of the guided filtering operation parameters and the focus enhancement results; Using the focus enhancement effect evaluation network to evaluate the selected value range of the pre-selected value of the guided filter operation parameter one by one, the selected value range of the focus enhancement result of the second sample image is obtained, and the pre-selected value of the guided filter operation parameter corresponding to the highest focus enhancement result is used as the optimal guided filter operation parameter of the second sample image; Using a neural network to train the first sample image, the second sample image and the optimal guided filtering operation parameters of the second sample image to obtain the parameter measurement network; The expression of the focus enhancement effect evaluation network is: Q = BP (filter_parameter); Where, Q is the focus enhancement effect, filter_parameter is the guided filter operation parameter, and BP is the BP neural network; The expression of the parameter estimation network is: filter_parameter=BP(S,Sc); Where filter_parameter is the guided filtering operation parameter, S and Sc are the first sample image and the second sample image respectively, and BP is the BP neural network; Acquiring a second image, comprising: Comparing the focus evaluation results of each first image in the first image set at each time point, and marking the first image corresponding to the maximum focus evaluation result in the first image set as the second image at the corresponding time point; Acquire a third image, including: Obtaining a focus representation image corresponding to the second image at each sequential point; generating a fusion decision matrix through a decision operation according to the focus representation map corresponding to the second image; Inputting the second image at each sequential point and the fusion decision matrix into a multi-focus fusion model to obtain a fusion weight of the second image at each sequential point; Performing a weighted average operation on the second image at each time point according to the fusion weight to obtain the third image; The generation of the fusion decision matrix includes: Perform pixel-by-pixel decision operations on the focus representation map corresponding to the second image at each time point, correspond the maximum value of the pixel in the focus representation map to the second image, retain the pixel in the second image corresponding to the maximum value of the pixel in the focus representation map, and obtain a decision matrix for the second image; Performing matrix summation on the decision matrices of the second image at each time point to obtain the fusion decision matrix; The decision operation formula is: In the formula, R t (x, y) is the value of the x row and y column in the decision matrix corresponding to the (x, y) coordinate point in the second image at the t-th time point, bl is the identifier of the pixel that retains the meaning, nl is the identifier of the pixel that does not retain the meaning, f t (x, y) is the pixel value at the (x, y) coordinate point in the corresponding focus representation map in the second image at the tth time point, f k (x, y) is the pixel value at the (x, y) coordinate point in the focus representation corresponding to the second image at the kth time point, max is the maximum value operator, max{f k (x,y)|k∈[1,N]} is the maximum value of the pixel at the (x,y) coordinate point in the corresponding focus representation map in the second image at each time point, N is the total number of second images or the total number of time points, and t, k are counting variables; R(x,y) is the value of row x and column y in the fusion decision matrix; The establishment of the multi-focus fusion model includes: Randomly selecting a plurality of second images as third sample images, and obtaining a fusion decision matrix of the third sample image according to a focus representation map of the third sample image; Randomly setting multiple groups of fusion weights of the third sample image; Performing a weighted average operation on the third sample image using multiple groups of fusion weights to obtain a fusion image of the multiple third sample images; performing a focus evaluation on a local area representing tumor cells in the fused image of the third sample image to obtain a focus evaluation result of the local area representing tumor cells in the fused image of the third sample image; taking the difference between the focus evaluation result of the local area representing the tumor cells in the fused image of the third sample image and the sum of the focus evaluation results of the local areas representing the tumor cells corresponding to each of the second images as the fusion evaluation result of the third sample image; Using a neural network to perform learning and training on the fusion weight of the third sample image and the fusion evaluation result of the third sample image, so as to obtain a fusion evaluation network of the third sample image; Using the fusion evaluation network to perform evaluations one by one in the selected value range of the fusion weight, a selected value range of the fusion evaluation result of the third sample image is obtained, and the fusion weight corresponding to the highest fusion evaluation result is used as the optimal fusion evaluation result of the third sample image; Using a neural network to perform learning and training on a fusion decision matrix of the third sample image and an optimal fusion evaluation result of the third sample image, to obtain the multi-focus fusion model; The expression of the fusion evaluation network is: P = BP (W); In the formula, P is the fusion evaluation result, W is the fusion weight, and BP is the BP neural network; The expression of the multi-focus fusion model is: W = BP (Sd); Where W is the fusion weight, Sd is the fusion decision matrix, and BP is the BP neural network.

2. The method for tracking tumor cell diffusion targets based on multi-focus fusion according to claim 1, characterized in that: The types of medical imaging data include at least one of CT image data, MRI image data and ultrasound image data.

3. The method for tracking tumor cell diffusion targets based on multi-focus fusion according to claim 1, characterized in that: Determine the global spread pathways of tumor cells, including: The local area representing the tumor cells at the preceding time point in the local diffusion path of the tumor cells is used as the input item of the LSTM neural network, and the local area representing the tumor cells at the following time point in the local diffusion path of the tumor cells is used as the output item of the LSTM neural network; Using the LSTM neural network to perform network training on the input items of the LSTM neural network and the output items of the LSTM neural network to obtain the global diffusion path; The network expression of the global diffusion path is: O future =LSTM(O old ); In the formula, O future is the local area representing tumor cells at the later time point, O old It is the local area representing the tumor cells at the preceding time point, and LSTM is a neural network.

4. The method for tracking tumor cell diffusion targets based on multi-focus fusion according to claim 1, characterized in that: The decision matrix is ​​consistent with the matrix specifications of the pixel matrix of the second image.

5. The method for tracking tumor cell diffusion targets based on multi-focus fusion according to claim 1, characterized in that: The first images at each time point have the same medical imaging settings.

Images