A puncture navigation method and system based on image recognition

Abstract

The present invention provides a puncture navigation method and system based on image recognition, including: constructing a 3D image according to a tomographic image of an abnormal tissue to be identified; selecting surface projection or curve fitting for puncture navigation according to the part displayed in the 3D image; if surface projection is selected, multiple puncture paths are determined by the shortest straight-line distance between the centroid of the puncture area and the outer surface; the puncture area corresponding to the puncture path with the highest value in the first coefficient is determined as the selected puncture area; connecting the puncture starting point and the centroid of the selected puncture area to obtain the selected puncture path; if curve fitting is selected, the centroid of each puncture area and multiple key points are curve fitted to obtain multiple puncture paths; the second coefficient of each puncture path is obtained by weighted fitting of the number of key points, the minimum curvature of the puncture path and the safety distance of the key points; the puncture path corresponding to the highest value of the second coefficient is determined as the selected puncture path. Improve the accuracy of puncture.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a puncture navigation method and system based on image recognition. Background Art

[0002] A puncture biopsy is a multidisciplinary diagnostic procedure that uses physical examination, ultrasound, CT, magnetic resonance imaging and other means to locate the lesion, and uses a puncture needle to obtain part of the human body's diseased tissue for pathological diagnosis. However, the control of the puncture needle and the selection of the best puncture path have always troubled the puncture doctor. The commonly used puncture method is CT guidance, but even when the biopsy puncture is performed under CT guidance, there are still phenomena such as the tumor tissue being necrotic, or the tumor tissue coming into contact with normal tissue or blood during the puncture, or unnecessary damage to normal tissue during the puncture process. In summary, the puncture methods in the prior art have problems such as inaccurate collection targets and poor puncture paths. Summary of the invention

[0003] In view of the above technical problems, the present invention provides a puncture navigation method based on image recognition, comprising:

[0004] Constructing a 3D image of the abnormal tissue and tissue organs in a preset area according to the CT tomography image and the MRI tomography image of the abnormal tissue to be identified;

[0005] According to the location of the abnormal tissue displayed by the 3D image and the location of the tissue and organ in the preset area, select surface projection navigation or curve fitting navigation to perform puncture navigation on the abnormal tissue;

[0006] If surface projection navigation is selected for puncture navigation, multiple puncture areas marked on the outer surface of the abnormal tissue are obtained according to the 3D image, and multiple puncture paths are determined by the shortest straight-line distance between the centroid of each puncture area and the outer surface; a weighted fitting calculation is performed on the area of ​​each puncture area, the shortest straight-line distance between the centroid of each puncture area and the outer surface, and the volume of the functional area tissue contained in the projection of each puncture area on the outer surface to obtain the first coefficient of each puncture path;

[0007] Determine the puncture area corresponding to the puncture path with the highest value of the first coefficient among the puncture paths as the selected puncture area, and determine the centroid of the selected puncture area;

[0008] Determine the centroid of the projection area of ​​the selected puncture area to the outer surface of the abnormal tissue as the puncture starting point;

[0009] connecting the puncture starting point and the centroid of the selected puncture area to obtain a selected puncture path determined by surface projection navigation;

[0010] If curve fitting navigation is selected for puncture navigation, multiple puncture areas and multiple key points marked on the outer surface of the abnormal tissue are obtained according to the 3D image;

[0011] Performing curve fitting on the centroids of each puncture area and the plurality of key points to obtain a plurality of puncture paths;

[0012] The second coefficient of each puncture path is obtained by performing weighted fitting on the number of key points, the minimum curvature of each puncture path, and the safety distance of each key point;

[0013] The puncture path corresponding to the highest value of the second coefficient is determined as the selected puncture path determined by curve fitting navigation.

[0014] Furthermore, before the step of constructing a 3D image of the abnormal tissue and tissue organs in a preset area based on the CT tomography image and the MRI tomography image of the abnormal tissue to be identified, the method further includes:

[0015] Acquire a CT tomographic image and an MRI tomographic image of the abnormal tissue to be identified;

[0016] Obtaining a CT abnormal tissue region according to the CT tomography image and the MRI tomography image;

[0017] Performing active tissue segmentation on the abnormal tissue region of the CT to obtain multiple active tissue regions of interest;

[0018] Extracting a plurality of preset attribute features of each active tissue region of interest in the plurality of active tissue-induced regions;

[0019] Quantitatively calculating a plurality of preset attribute features of each active tissue region of interest to generate corresponding feature quantization values ​​of each active tissue region of interest;

[0020] The corresponding feature quantification value of each active tissue region of interest is input into a pre-built machine learning classifier to determine multiple puncture areas on the outer surface of the abnormal tissue in the CT tomography image and mark them in the CT tomography image.

[0021] Furthermore, obtaining the CT abnormal tissue area according to the CT tomographic image and the MRI tomographic image includes:

[0022] Performing abnormal tissue segmentation on the CT tomography image and the MRI tomography image respectively to obtain an abnormal tissue segmentation region, a CT abnormal tissue boundary, a CT abnormal tissue segmentation confidence, an MRI abnormal tissue boundary, and an MRI abnormal tissue segmentation confidence in the CT tomography image;

[0023] According to the CT abnormal tissue boundary, the CT abnormal tissue segmentation confidence, the MRI abnormal tissue boundary and the MRI abnormal tissue segmentation confidence, the abnormal tissue segmentation area in the CT tomographic image is corrected to obtain the CT abnormal tissue area.

[0024] Furthermore, based on the CT tomography image and the MRI tomography image of the abnormal tissue to be identified, a 3D image of the abnormal tissue and tissue organs in a preset area is constructed, including:

[0025] The CT tomography image and multiple puncture areas in the MRI tomography image of the abnormal tissue to be identified, as well as the tissue and organ images displayed in the preset area, are three-dimensionally reconstructed to obtain the 3D images of the abnormal tissue and the tissue and organs in the preset area.

[0026] Further, according to the location of the abnormal tissue displayed by the 3D image and the location of the tissue organ in the preset area, surface projection navigation or curve fitting navigation is selected to perform puncture navigation on the abnormal tissue, including:

[0027] Inputting the location of the abnormal tissue displayed by the 3D image and the location of the tissue and organ in the preset area into a pre-constructed location recognition model to obtain a location recognition result;

[0028] According to the site identification result, surface projection navigation or curve fitting navigation is selected to perform puncture navigation on the abnormal tissue.

[0029] Further, according to the site recognition result, surface projection navigation or curve fitting navigation is selected to perform puncture navigation on the abnormal tissue, including:

[0030] If the part recognition result is the head, then select surface projection navigation for puncture navigation;

[0031] If the part recognition result is not the head, curve fitting navigation is selected for puncture navigation.

[0032] Furthermore, a weighted fitting calculation is performed on the area of ​​each puncture region, the shortest straight-line distance between the centroid of each puncture region and the outer surface, and the volume of the functional area tissue contained in the projection of each puncture region on the outer surface to obtain the first coefficient of each puncture path, including:

[0033] Calculate the area of ​​each puncture region according to the multiple puncture regions marked in the 3D image , the shortest straight-line distance from the centroid of each puncture area to the outer surface , the functional area tissue volume contained in the projection of the outer surface of each puncture area ;

[0034] right and Perform weighted fitting calculation to obtain the first coefficient of each puncture path ,

[0035] ,

[0036] in are both greater than 0 and less than 1, , is the weight coefficient of the puncture area, is the weight coefficient of the shortest straight-line distance from the centroid of the puncture area to the outer surface, It is the weight coefficient of the functional area tissue volume contained in the outer surface projection of the puncture area.

[0037] Further, determining the puncture area corresponding to the puncture path with the highest value of the first coefficient among the puncture paths as the selected puncture area includes:

[0038] In multiple puncture areas, The puncture area corresponding to the puncture path is determined as the selected puncture area.

[0039] Furthermore, it also includes:

[0040] Perform curve fitting on multiple key points in the puncture path to obtain the minimum curvature Greater than the preset threshold and beyond the safe distance from the preset tissues and organs Multiple key point curves;

[0041] The key points on the plurality of key point curves are obtained.

[0042] Furthermore, the centroids of each puncture area and the plurality of key points are subjected to curve fitting to obtain a plurality of puncture paths, including:

[0043] The centroid of each puncture starting area , the centroid of each puncture area The curve fitting is performed with the plurality of key points to obtain a minimum curvature greater than Multiple puncture paths.

[0044] Furthermore, by performing weighted fitting on the number of key points, the minimum curvature of each puncture path and the safety distance of each key point, the second coefficient of each puncture path is obtained, including:

[0045] By the number of key points , the minimum curvature of each puncture path Safe distance from each key point Perform weighted fitting to obtain the second coefficients of multiple puncture paths ,

[0046]

[0047] in are greater than 0 and less than 1, , is the weight of the number of key points, is the weight of the minimum curvature of the puncture path, is the safety distance weight of the key point.

[0048] Further, determining the puncture path corresponding to the highest value of the second coefficient as the selected puncture path determined by curve fitting navigation includes:

[0049] In multiple puncture paths, The puncture path is determined as the selected puncture path.

[0050] The present invention also provides a puncture navigation system based on image recognition, comprising:

[0051] A 3D image construction module, used to construct a 3D image of the abnormal tissue and tissue organs in a preset area according to the CT tomography image and the MRI tomography image of the abnormal tissue to be identified;

[0052] A navigation selection module, used for selecting surface projection navigation or curve fitting navigation to perform puncture navigation on the abnormal tissue according to the location of the abnormal tissue displayed by the 3D image and the location of the tissue organ in the preset area;

[0053] A first coefficient obtaining module is used for obtaining a plurality of puncture areas marked on the outer surface of the abnormal tissue according to the 3D image when surface projection navigation is selected for puncture navigation, and determining a plurality of puncture paths by the shortest straight-line distance between the centroid of each puncture area and the outer surface; performing weighted fitting calculation on the area of ​​each puncture area, the shortest straight-line distance between the centroid of each puncture area and the outer surface, and the volume of the functional area tissue contained in the projection of each puncture area on the outer surface, to obtain a first coefficient of each puncture path;

[0054] A selected puncture area determination module, used to determine the puncture area corresponding to the puncture path with the highest value of the first coefficient among the puncture paths as the selected puncture area, and determine the centroid of the selected puncture area;

[0055] A puncture starting point determination module, used to determine the centroid of the projection area of ​​the selected puncture area to the outer surface of the abnormal tissue as the puncture starting point;

[0056] a first selected puncture path determination module, configured to connect the puncture starting point and the centroid of the selected puncture area to obtain a selected puncture path determined by surface projection navigation;

[0057] A puncture area and key point determination module, for obtaining a plurality of puncture areas and a plurality of key points marked on the outer surface of the abnormal tissue according to the 3D image if curve fitting navigation is selected for puncture navigation;

[0058] A multiple puncture path acquisition module is used to perform curve fitting on the centroid of each puncture area and the multiple key points to obtain multiple puncture paths;

[0059] A second coefficient obtaining module, used for obtaining a second coefficient of each puncture path by performing weighted fitting on the number of key points, the minimum curvature of each puncture path and the safety distance of each key point;

[0060] The second selected puncture path determination module is used to determine the puncture path corresponding to the second coefficient maximum value as the selected puncture path determined by curve fitting navigation.

[0061] The present invention also provides a computer-readable storage medium on which a computer program is stored. The computer program is loaded by a processor to execute the steps in any of the above-mentioned puncture navigation methods based on image recognition.

[0062] The present invention provides a puncture navigation method and system based on image recognition, which identifies the location of abnormal tissue by constructing a 3D image of abnormal tissue and tissue organs in a preset area, and uses different puncture navigation methods to determine the puncture path for abnormal tissues in different locations, thereby improving the accuracy and safety of puncture. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 It is a flowchart of a puncture navigation method based on image recognition provided by an embodiment of the present invention;

[0064] Figure 2 is a structural schematic diagram of a puncture navigation system based on image recognition provided by an embodiment of the present invention;

[0065] Figure 3 It is a structural block diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0066] Many specific details are set forth in the following description to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present invention, so the present invention is not limited to the specific implementation disclosed below.

[0067] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more features. In the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0068] In the present invention, the word "exemplary" is used to mean "used as an example, illustration or description". Any embodiment described as "exemplary" in the present invention is not necessarily to be construed as being preferred or advantageous over other embodiments. The following description is given to enable any technician in the field to implement and use the present invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other instances, well-known structures and processes will not be elaborated in detail to avoid obscuring the description of the present invention with unnecessary details. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present invention.

[0069] An image recognition-based puncture navigation method and system provided by the embodiments of the present invention are described in detail below.

[0070] See also Figure 1 , Figure 1 A schematic flow chart of a puncture navigation method based on image recognition provided by the present invention, the method comprising the following steps:

[0071] Step S101, constructing a 3D image of the abnormal tissue and tissue organs in a preset area based on the CT tomography image and the MRI tomography image of the abnormal tissue to be identified.

[0072] In the embodiment of the present invention, the CT tomographic scan image and the MRI tomographic scan image should correspond to each other in the same way. The abnormal tissue may be a tumor or other uncertain abnormal proliferation.

[0073] First, a CT tomographic image and an MRI tomographic image of the abnormal tissue to be identified are obtained, and then the images are processed accordingly. Specifically, a CT tomographic image and an MRI tomographic image of the abnormal tissue to be identified are obtained; a CT abnormal tissue region is obtained according to the CT tomographic image and the MRI tomographic image; active tissue segmentation is performed on the CT abnormal tissue region to obtain multiple active tissue regions of interest; multiple preset attribute features of each active tissue region of interest in the multiple active tissue regions of interest are extracted; multiple preset attribute features of each active tissue region of interest are quantified and calculated to generate corresponding feature quantization values ​​of each active tissue region of interest; the corresponding feature quantization values ​​of each active tissue region of interest are input into a pre-constructed machine learning classifier, multiple puncture areas on the outer surface of the abnormal tissue in the CT tomographic image are determined, and marked in the CT tomographic image.

[0074] Acquisition of CT abnormal tissue area, by performing abnormal tissue segmentation on the CT tomographic image and the MRI tomographic image respectively, to obtain the abnormal tissue segmentation area, CT abnormal tissue boundary, CT abnormal tissue segmentation confidence, MRI abnormal tissue boundary and MRI abnormal tissue segmentation confidence in the CT tomographic image; according to the CT abnormal tissue boundary, CT abnormal tissue segmentation confidence, MRI abnormal tissue boundary and MRI abnormal tissue segmentation confidence, the abnormal tissue segmentation area in the CT tomographic image is corrected to obtain the CT abnormal tissue area.

[0075] Among them, CT abnormal tissue segmentation refers to segmenting the abnormal tissue area image according to the area where the abnormal tissue is located in the CT tomographic scan image.

[0076] Specifically, the CT abnormal tissue area is used as a sample image, and a segmentation model is pre-trained. For example, a Unet++ network model is selected. In a specific embodiment, the target area image is used as the input of the trained segmentation model, and the output result of the segmentation model is the CT abnormal tissue boundary and the CT abnormal tissue segmentation confidence.

[0077] The MRI abnormal tissue segmentation refers to segmenting the abnormal tissue region image according to the region where the abnormal tissue is located in the MRI tomographic image.

[0078] Specifically, the MRI abnormal tissue area is used as a sample image, and a segmentation model is pre-trained. For example, a Unet++ network model is selected. In a specific embodiment, the target area image is used as the input of the trained segmentation model, and the output result of the segmentation model is the MRI abnormal tissue boundary and the MRI abnormal tissue segmentation confidence.

[0079] Specifically, the CT abnormal tissue boundaries and CT abnormal tissue segmentation confidences obtained from the one-to-one corresponding CT tomography images and MRI tomography images and the MRI abnormal tissue boundaries and MRI abnormal tissue segmentation confidences are paired and stored.

[0080] According to the CT abnormal tissue boundary, the CT segmentation confidence, the MRI abnormal tissue boundary, and the MRI segmentation confidence, the abnormal tissue segmentation area in the CT tomographic image is corrected to obtain the CT abnormal tissue area.

[0081] Each CT tomographic scan image has a corresponding CT abnormal tissue boundary, CT abnormal tissue segmentation confidence and MRI abnormal tissue boundary, MRI abnormal tissue segmentation confidence. Based on the corresponding CT abnormal tissue boundary, CT abnormal tissue segmentation confidence and MRI abnormal tissue boundary, MRI abnormal tissue segmentation confidence, it is determined whether there is abnormal tissue in the CT tomographic scan image. If there is abnormal tissue, the abnormal tissue area is precisely processed.

[0082] Active tissue segmentation is performed on the CT abnormal tissue region to obtain multiple active tissue regions of interest, wherein active tissue region of interest segmentation refers to segmenting the active tissue region of interest image according to the active tissue regions of interest in the CT abnormal tissue region segmentation image.

[0083] Specifically, the active tissue region of interest is used as a sample image, and a segmentation model is pre-trained. For example, a Unet++ network model is selected. In a specific embodiment, the target area image is used as the input of the trained segmentation model, and the output result of the segmentation model is a segmentation image of the active tissue region of interest.

[0084] It should be noted here that there may be no active tissue region of interest in an abnormal tissue region, which means that the abnormal tissue is dead. There may be one active tissue region of interest or multiple discrete active tissue regions of interest.

[0085] The feature extraction of multiple preset attributes is performed on the active tissue region of interest segmentation image to obtain feature quantization values ​​corresponding to the multiple preset attributes.

[0086] Among them, the preset attributes refer to multiple attributes of the active tissue region of interest segmentation image, such as color attributes, texture attributes, regional morphology attributes, area attributes, boundary coincidence attributes, etc. of the active tissue region of interest segmentation image, and the feature quantization value refers to the quantization value corresponding to the feature of each preset attribute.

[0087] Specifically, a feature extraction method is used to extract features from the segmented image of the region of interest of the active tissue to obtain a feature quantization value, wherein the feature extraction method can be a combination of an artificial feature extraction method and an algorithm based on image feature analysis, such as pixel neighborhood mean calculation, maximum pixel value extraction, etc., to calculate the feature quantization value, or it can be a feature extraction method of deep learning, such as convolutional neural network CNN, UNet++, etc., which can be selected according to the characteristics of the preset attributes, which are not limited here. In this embodiment, by extracting features from the segmented image of the region of interest of the active tissue and obtaining the corresponding feature quantization value, the quantitative calculation of the features of each preset attribute of the segmented image of the region of interest of the active tissue is realized, so that the feature quantization value is more comprehensive and rich, so that accurate and intuitive image analysis and recognition can be performed based on the multiple feature quantization values ​​in the subsequent process, thereby improving the accuracy of puncture target positioning.

[0088] Multiple feature quantization values ​​are input into the trained machine learning classifier to determine the optimal puncture area in the CT tomography image and mark it in the CT tomography image.

[0089] In one embodiment, multiple first preset attributes include color attributes, texture attributes, regional morphology attributes, area attributes, and boundary coincidence attributes; the step of obtaining the characteristic quantization value corresponding to each preset attribute includes: using a preset color quantization method to determine the color quantization result of the active tissue region of interest segmentation image; using a preset texture quantization method to determine the texture quantization result of the active tissue region of interest segmentation image; using a preset regional morphology quantization method to determine the regional morphology quantization result of the active tissue region of interest segmentation image; using a preset area quantization method to determine the area quantization result of the active tissue region of interest segmentation image; using a preset boundary coincidence quantization method to determine the boundary coincidence quantization result of the active tissue region of interest segmentation image.

[0090] Specifically, the HOG feature extraction class in the preset skimage toolkit is called to quantize the color directional gradient histogram to obtain the one-dimensional feature vector of the color feature , and calculate the one-dimensional feature vector of the color feature The variance is obtained , then the color quantization value of the segmented image of the active tissue region of interest is .

[0091] Specifically, the segmented image of the active tissue region of interest is grayed out and the LBP feature extraction class in the preset skimage toolkit is called to perform grayscale texture quantization to obtain a one-dimensional feature vector of the texture feature. , and calculate the one-dimensional feature vector of the texture feature The average value , then the texture quantization value of the segmentation image of the active tissue region of interest is .

[0092] Specifically, based on the connected domain, the maximum inscribed circle of the image segmentation region of interest is organized, and the radius of the maximum inscribed circle is , calculate the distance from the segmentation image boundary of the active tissue region of interest to the maximum inscribed circle as , then the morphological quantification value of the active tissue region of interest segmentation image area is .

[0093] Specifically, based on the connected domain, the segmented image area of ​​each active tissue region of interest in the current abnormal tissue tomographic image is obtained. and abnormal segmentation image area , then the quantified value of the segmented image area of ​​the active tissue region of interest is .

[0094] Specifically, based on the connected domain, the number of boundary pixel points of the segmented image of each active tissue region of interest is obtained as , the number of pixels where the boundary of each active tissue region of interest segmentation image overlaps with the boundary of abnormal tissue is , then the quantization value of the boundary coincidence of the segmented image of the active tissue region of interest is .

[0095] In one embodiment, the trained machine learning classifier includes a feature fitting subnetwork and a classification subnetwork; multiple feature quantization values ​​are input into the trained machine learning classifier for classification to obtain the classification result of the optimal puncture area, including: using the feature fitting subnetwork to fit each feature quantization value to obtain a determination coefficient; based on the determination coefficient, using the classification subnetwork for analysis to obtain a classification result.

[0096] Specifically, the feature quantization values ​​of the feature fitting subnetwork are fitted, and the corresponding weights of the feature quantization values ​​for fitting are obtained according to the fitting results. For example, we use decision trees, random forests, etc. to determine The corresponding weights are , then the fusion feature value is: ,when When the active tissue region of interest is selected, it is the puncture area in the current tomographic layer.

[0097] Among them, the trained machine learning classifier can be realized by learning a machine learning algorithm model with classification ability through samples. The machine learning classifier of this embodiment is used to classify different feature value sets into one of the best puncture area or non-best puncture area results. Specifically, a classifier that can be classified using at least one machine learning model. The machine learning model can be one or more of the following: neural network (for example, convolutional neural network, BP neural network, etc.), logistic regression model, support vector machine, decision tree, random forest, perceptron and other machine learning models. As part of the training of such a machine learning model, the training input is each feature quantization value, for example, color attribute, texture attribute, regional morphology attribute, area attribute, boundary coincidence attribute, etc. Through training, a classifier of the corresponding relationship between the feature value set and the best puncture area or the non-best puncture area is established, so that the preset classifier has the ability to judge and identify the best puncture area result or the non-best puncture area result. In this embodiment, the classifier is a binary classifier, that is, two classification results are obtained, that is, the best puncture area result or the non-best puncture area result. It can be understood that in this embodiment, the influence of the feature quantization values ​​of multiple different attributes of the segmented image of the active tissue region of interest on the accuracy and intuitiveness of image recognition is fully considered. By extracting features with richer information and quantifying and comprehensively processing the features of multiple different attributes, the rationality of the feature value quantization is improved. Compared with the traditional method of only considering a single feature information and a single statistical comparison method, the recognition efficiency of the active tissue region of interest suitable for puncture is greatly improved.

[0098] Specifically, according to the machine learning recognition result, the boundary of the active tissue region of interest corresponding to the maximum confidence value in the machine learning output result is marked in the CT tomography image.

[0099] The CT tomography image and the multiple puncture areas in the MRI tomography image of the abnormal tissue to be identified, as well as the tissue and organ images displayed in the preset area, are reconstructed in three dimensions to obtain the 3D images of the abnormal tissue and the tissue and organ in the preset area. The VTK three-dimensional construction software can be used to superimpose along the Z-axis direction to obtain the 3D images of the abnormal tissue and the tissue and organ around the preset area.

[0100] It should be noted here that the 3D image of abnormal head tissue and surrounding tissues and organs in the preset area should be a 3D image of the entire head, and the 3D image of abnormal non-head tissue and surrounding tissues and organs in the preset area should involve the human body surface.

[0101] Step S102, according to the location of the abnormal tissue displayed by the 3D image and the location of the tissue organ in the preset area, select surface projection navigation or curve fitting navigation to perform puncture navigation on the abnormal tissue.

[0102] The location of the abnormal tissue displayed in the 3D image and the location of the tissue and organ in the preset area are input into a pre-constructed location recognition model to obtain a location recognition result; based on the location recognition result, surface projection navigation or curve fitting navigation is selected to perform puncture navigation on the abnormal tissue.

[0103] In the present invention, the implementation methods of head puncture and non-head puncture are different. First, the location of the abnormal tissue needs to be clarified. Specifically, a head / non-head 2-classification recognition model of the 3D image of the abnormal tissue and the surrounding tissues and organs in the preset area is trained. The model is a 3D convolutional neural network, and the labels are head part and non-head part.

[0104] If the part recognition result is the head, then surface projection navigation is selected for puncture navigation; if the part recognition result is the torso, then curve pseudo-navigation is selected for puncture navigation.

[0105] Step S103, if surface projection navigation is selected for puncture navigation, multiple puncture areas marked on the outer surface of the abnormal tissue are obtained according to the 3D image, and multiple puncture paths are determined by the shortest straight-line distance between the centroid of each puncture area and the outer surface; a weighted fitting calculation is performed on the area of ​​each puncture area, the shortest straight-line distance between the centroid of each puncture area and the outer surface, and the volume of the functional area tissue contained in the projection of each puncture area on the outer surface to obtain a first coefficient of each puncture path.

[0106] When surface projection navigation is used for puncture navigation, it is usually targeted at head puncture. The head has functional tissues, and the outer surface of the head closest to the abnormal tissue is usually selected as the puncture entry area in clinical practice, taking the shortest straight line path to avoid causing additional damage to more normal functional tissues; when puncturing abnormal tissues in non-head parts, in order to avoid damage to blood vessels, nerves, etc. caused by straight line puncture, the present invention adopts the best curved path for puncture, bypassing blood vessels, nerves and other structures.

[0107] When performing puncture navigation on the head using surface projection navigation, the area of ​​each puncture area is calculated according to the multiple puncture areas marked in the 3D image. , the shortest straight-line distance from the centroid of each puncture area to the outer surface , the functional area tissue volume contained in the projection of the outer surface of each puncture area ;

[0108] right Perform weighted fitting calculation to obtain the first coefficient of each puncture path ,

[0109] ,

[0110] in are both greater than 0 and less than 1, , is the weight coefficient of the puncture area, is the weight coefficient of the shortest straight-line distance from the centroid of the puncture area to the outer surface, It is the weight coefficient of the functional area tissue volume contained in the outer surface projection of the puncture area.

[0111] Step S104, determining the puncture area corresponding to the puncture path with the highest value of the first coefficient among the puncture paths as the selected puncture area, and determining the centroid of the selected puncture area.

[0112] In multiple puncture areas, The puncture area corresponding to the puncture path is determined as the selected puncture area.

[0113] Step S105, determining the centroid of the projection area of ​​the selected puncture area to the outer surface of the abnormal tissue as the puncture starting point.

[0114] Step S106, connecting the puncture starting point and the centroid of the selected puncture area to obtain a selected puncture path determined by surface projection navigation.

[0115] Step S107: if curve fitting navigation is selected for puncture navigation, a plurality of puncture areas and a plurality of key points marked on the outer surface of the abnormal tissue are acquired according to the 3D image.

[0116] When curve fitting navigation is chosen for puncture navigation, it is generally used for non-head punctures, such as the gallbladder and liver in the abdomen.

[0117] First, determine the area of ​​each puncture area marked on the outer surface of the abnormal tissue in the 3D image of the non-head abnormal tissue and the surrounding tissue organs of the preset area and the centroid of each puncture area ;

[0118] Evaluate the area on the human body surface that is most suitable for the puncture needle to enter in the 3D image of the abnormal tissue and the surrounding tissue organs of the preset area, and calculate the centroid of the puncture starting area that is most suitable for the puncture needle to enter ;

[0119] Then, the key points in the puncture path are selected from the 3D images of the non-abnormal tissues of the head and the surrounding tissues and organs in the preset area. The number of key points is The determination of the key points requires curve fitting of multiple key points in the puncture path to obtain the minimum curvature. Greater than the preset threshold and beyond the safe distance from the preset tissues and organs The key points avoid preset tissues and organs, such as blood vessels, bile ducts, etc.; curve fitting is performed on the key points to ensure the minimum curvature on the entire puncture path. Greater than the preset threshold and beyond the safe distance from the preset tissues and organs The safety distance is determined based on the importance of the tissue or organ, the degree of susceptibility to damage, the speed and force of the puncture needle setting; and each key point on the multiple key point curves is obtained.

[0120] Step S108, performing curve fitting on the centroids of each puncture area and the plurality of key points to obtain a plurality of puncture paths.

[0121] The centroid of each puncture starting area , the centroid of each puncture area The curve fitting is performed with the plurality of key points to obtain a minimum curvature greater than Multiple puncture paths.

[0122] Step S109, obtaining a second coefficient of each puncture path by performing weighted fitting on the number of key points, the minimum curvature of each puncture path and the safety distance of each key point.

[0123] By the number of key points , the minimum curvature of each puncture path Safe distance from each key point Perform weighted fitting to obtain the second coefficients of multiple puncture paths ,

[0124]

[0125] in, are greater than 0 and less than 1, , is the weight of the number of key points, is the weight of the minimum curvature of the puncture path, is the safety distance weight of the key point.

[0126] Step S110, determining the puncture path corresponding to the highest value of the second coefficient as the selected puncture path determined by curve fitting navigation.

[0127] In multiple puncture paths, The puncture path is determined as the selected puncture path.

[0128] Based on the same inventive concept, the present invention also provides a puncture navigation system based on image recognition, such as Figure 2 As shown, including:

[0129] A 3D image construction module 210, configured to construct a 3D image of the abnormal tissue and tissue organs in a preset area according to the CT tomography image and the MRI tomography image of the abnormal tissue to be identified;

[0130] A navigation selection module 220, for selecting surface projection navigation or curve fitting navigation to perform puncture navigation on the abnormal tissue according to the location of the abnormal tissue displayed in the 3D image and the location of the tissue organ in the preset area;

[0131] The first coefficient obtaining module 230 is used for obtaining a plurality of puncture areas marked on the outer surface of the abnormal tissue according to the 3D image if surface projection navigation is selected for puncture navigation, and determining a plurality of puncture paths by the shortest straight-line distance between the centroid of each puncture area and the outer surface; performing weighted fitting calculation on the area of ​​each puncture area, the shortest straight-line distance between the centroid of each puncture area and the outer surface, and the volume of the functional area tissue contained in the projection of each puncture area on the outer surface, to obtain the first coefficient of each puncture path;

[0132] A selected puncture area determination module 240 is used to determine the puncture area corresponding to the puncture path with the highest value of the first coefficient among the puncture paths as the selected puncture area, and determine the centroid of the selected puncture area;

[0133] A puncture starting point determination module 250 is used to determine the centroid of the projection area of ​​the selected puncture area to the outer surface of the abnormal tissue as the puncture starting point;

[0134] A first selected puncture path determination module 260, used to connect the puncture starting point and the centroid of the selected puncture area to obtain a selected puncture path determined by surface projection navigation;

[0135] A puncture area and key point determination module 270, for obtaining a plurality of puncture areas and a plurality of key points marked on the outer surface of the abnormal tissue according to the 3D image if curve fitting navigation is selected for puncture navigation;

[0136] A multiple puncture path acquisition module 280, used for performing curve fitting on the centroid of each puncture area and the multiple key points to obtain multiple puncture paths;

[0137] A second coefficient obtaining module 290, for obtaining a second coefficient of each puncture path by performing weighted fitting on the number of key points, the minimum curvature of each puncture path and the safety distance of each key point;

[0138] The second selected puncture path determination module 2100 is used to determine the puncture path corresponding to the second coefficient maximum value as the selected puncture path determined by curve fitting navigation.

[0139] In one embodiment, the puncture navigation method based on image recognition provided by the present invention can be implemented in the form of a computer program. The computer program can be used in Figure 3 The computer device is run on the computer device shown. The memory of the computer device can store various system modules that constitute the puncture navigation system based on image recognition. A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps in the above-mentioned automatic puncture navigation method when executing the computer program.

[0140] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned automatic puncture navigation method are implemented.

[0141] The present invention provides a puncture navigation method and system based on image recognition, which can be used in conjunction with corresponding automatic puncture equipment to complete automated puncture.

[0142] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0143] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0144] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for a person of ordinary skill in the art, multiple variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A puncture path planning method based on image recognition, characterized in that: include: Constructing a 3D image of the abnormal tissue and tissue organs in a preset area according to the CT tomography image and the MRI tomography image of the abnormal tissue to be identified; Selecting surface projection navigation or curve fitting navigation according to the location of abnormal tissue displayed in the 3D image and the location of tissue organs in a preset area; If surface projection navigation is selected for puncture navigation, multiple puncture areas marked on the outer surface of the abnormal tissue are obtained according to the 3D image, and multiple puncture paths are determined by the shortest straight-line distance between the centroid of each puncture area and the outer surface; a weighted fitting calculation is performed on the area of ​​each puncture area, the shortest straight-line distance between the centroid of each puncture area and the outer surface, and the volume of the functional area tissue contained in the projection of each puncture area on the outer surface to obtain the first coefficient of each puncture path; Determine the puncture area corresponding to the puncture path with the highest value of the first coefficient among the puncture paths as the selected puncture area, and determine the centroid of the selected puncture area; Determine the centroid of the projection area of ​​the selected puncture area to the outer surface of the abnormal tissue as the puncture starting point; connecting the puncture starting point and the centroid of the selected puncture area to obtain a selected puncture path determined by surface projection navigation; If curve fitting navigation is selected for puncture navigation, multiple puncture areas and multiple key points marked on the outer surface of the abnormal tissue are obtained according to the 3D image; Performing curve fitting on the centroids of each puncture area and the plurality of key points to obtain a plurality of puncture paths; The second coefficient of each puncture path is obtained by performing weighted fitting on the number of key points, the minimum curvature of each puncture path, and the safety distance of each key point; The puncture path corresponding to the highest value of the second coefficient is determined as the selected puncture path determined by curve fitting navigation.

2. The method according to claim 1, characterized in that Before the step of constructing a 3D image of the abnormal tissue and tissue organs in a preset area according to the CT tomography image and the MRI tomography image of the abnormal tissue to be identified, the method further includes: Acquire a CT tomographic image and an MRI tomographic image of the abnormal tissue to be identified; Obtaining a CT abnormal tissue region according to the CT tomography image and the MRI tomography image; Performing active tissue segmentation on the abnormal tissue region of the CT to obtain multiple active tissue regions of interest; Extracting a plurality of preset attribute features of each active tissue region of interest in the plurality of active tissue-induced regions; Quantitatively calculating a plurality of preset attribute features of each active tissue region of interest to generate corresponding feature quantization values ​​of each active tissue region of interest; The corresponding feature quantification value of each active tissue region of interest is input into a pre-built machine learning classifier to determine multiple puncture areas on the outer surface of the abnormal tissue in the CT tomography image and mark them in the CT tomography image.

3. The method according to claim 2, characterized in that Obtaining a CT abnormal tissue region according to the CT tomography image and the MRI tomography image includes: Performing abnormal tissue segmentation on the CT tomography image and the MRI tomography image respectively to obtain an abnormal tissue segmentation region, a CT abnormal tissue boundary, a CT abnormal tissue segmentation confidence, an MRI abnormal tissue boundary, and an MRI abnormal tissue segmentation confidence in the CT tomography image; According to the CT abnormal tissue boundary, the CT abnormal tissue segmentation confidence, the MRI abnormal tissue boundary and the MRI abnormal tissue segmentation confidence, the abnormal tissue segmentation area in the CT tomographic image is corrected to obtain the CT abnormal tissue area.

4. The method according to claim 1 or 2, characterized in that: According to the CT tomography image and the MRI tomography image of the abnormal tissue to be identified, a 3D image of the abnormal tissue and tissue organs in a preset area is constructed, including: The CT tomography image and multiple puncture areas in the MRI tomography image of the abnormal tissue to be identified, as well as the tissue and organ images displayed in the preset area, are three-dimensionally reconstructed to obtain the 3D images of the abnormal tissue and the tissue and organs in the preset area.

5. The method according to claim 1, characterized in that According to the location of the abnormal tissue displayed by the 3D image and the location of the tissue organ in the preset area, surface projection navigation or curve fitting navigation is selected to perform puncture navigation on the abnormal tissue, including: Inputting the location of the abnormal tissue displayed by the 3D image and the location of the tissue and organ in the preset area into a pre-constructed location recognition model to obtain a location recognition result; According to the site identification result, surface projection navigation or curve fitting navigation is selected to perform puncture navigation on the abnormal tissue.

6. The method according to claim 5, characterized in that According to the site identification result, surface projection navigation or curve fitting navigation is selected to perform puncture navigation on the abnormal tissue, including: If the part recognition result is the head, then select surface projection navigation for puncture navigation; If the part recognition result is not the head, curve fitting navigation is selected for puncture navigation.

7. The method according to claim 1 or 2, characterized in that: The first coefficient of each puncture path is obtained by performing weighted fitting calculation on the area of ​​each puncture region, the shortest straight-line distance between the centroid of each puncture region and the outer surface, and the volume of the functional area tissue contained in the projection of each puncture region on the outer surface, including: Calculate the area of ​​each puncture region according to the multiple puncture regions marked in the 3D image , the shortest straight-line distance from the centroid of each puncture area to the outer surface , the functional area tissue volume contained in the projection of the outer surface of each puncture area ; right Perform weighted fitting calculation to obtain the first coefficient of each puncture path , ， in are greater than 0 and less than 1, , is the weight coefficient of the puncture area, is the weight coefficient of the shortest straight-line distance from the centroid of the puncture area to the outer surface, It is the weight coefficient of the functional area tissue volume contained in the outer surface projection of the puncture area.

8. The method according to claim 1, characterized in that Determining the puncture area corresponding to the puncture path having the highest value of the first coefficient among the puncture paths as the selected puncture area includes: In multiple puncture areas, The puncture area corresponding to the puncture path is determined as the selected puncture area.

9. The method according to claim 1, characterized in that: Also includes: Perform curve fitting on multiple key points in the puncture path to obtain the minimum curvature Greater than the preset threshold and beyond the safe distance from the preset tissues and organs Multiple key point curves; The key points on the plurality of key point curves are obtained.

10. The method according to claim 1, characterized in that Performing curve fitting on the centroids of each puncture area and the plurality of key points to obtain a plurality of puncture paths, including: The centroid of each puncture starting area , the centroid of each puncture area The curve fitting is performed with the plurality of key points to obtain a minimum curvature greater than Multiple puncture paths.

11. The method according to claim 1, characterized in that: By performing weighted fitting on the number of key points, the minimum curvature of each puncture path and the safety distance of each key point, the second coefficient of each puncture path is obtained, including: By the number of key points , the minimum curvature of each puncture path Safe distance from each key point Perform weighted fitting to obtain the second coefficients of multiple puncture paths , in are greater than 0 and less than 1, , is the weight of the number of key points, is the weight of the minimum curvature of the puncture path, is the safety distance weight of the key point.

12. The method according to claim 1, characterized in that Determining the puncture path corresponding to the highest value of the second coefficient as the selected puncture path determined by curve fitting navigation includes: In multiple puncture paths, The puncture path is determined as the selected puncture path.

13. A puncture navigation system based on image recognition, characterized in that: include: A 3D image construction module, used to construct a 3D image of the abnormal tissue and tissue organs in a preset area according to the CT tomography image and the MRI tomography image of the abnormal tissue to be identified; A navigation selection module, used for selecting surface projection navigation or curve fitting navigation to perform puncture navigation on the abnormal tissue according to the location of the abnormal tissue displayed by the 3D image and the location of the tissue organ in the preset area; A first coefficient obtaining module is used for obtaining a plurality of puncture areas marked on the outer surface of the abnormal tissue according to the 3D image when surface projection navigation is selected for puncture navigation, and determining a plurality of puncture paths by the shortest straight-line distance between the centroid of each puncture area and the outer surface; performing weighted fitting calculation on the area of ​​each puncture area, the shortest straight-line distance between the centroid of each puncture area and the outer surface, and the volume of the functional area tissue contained in the projection of each puncture area on the outer surface, to obtain a first coefficient of each puncture path; A selected puncture area determination module, used to determine the puncture area corresponding to the puncture path with the highest value of the first coefficient among the puncture paths as the selected puncture area, and determine the centroid of the selected puncture area; A puncture starting point determination module, used to determine the centroid of the projection area of ​​the selected puncture area to the outer surface of the abnormal tissue as the puncture starting point; a first selected puncture path determination module, configured to connect the puncture starting point and the centroid of the selected puncture area to obtain a selected puncture path determined by surface projection navigation; A puncture area and key point determination module, for obtaining a plurality of puncture areas and a plurality of key points marked on the outer surface of the abnormal tissue according to the 3D image if curve fitting navigation is selected for puncture navigation; A multiple puncture path acquisition module is used to perform curve fitting on the centroid of each puncture area and the multiple key points to obtain multiple puncture paths; A second coefficient obtaining module, used for obtaining a second coefficient of each puncture path by performing weighted fitting on the number of key points, the minimum curvature of each puncture path and the safety distance of each key point; The second selected puncture path determination module is used to determine the puncture path corresponding to the second coefficient maximum value as the selected puncture path determined by curve fitting navigation.

14. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in the puncture path planning method based on image recognition as described in any one of claims 1 to 12.

Images