Computer-assisted surgery system for high-precision foreign body positioning and removal

Through the needle detection method based on feature extraction and the contrast enhancement technology of generating adversarial networks, the problems in needle recognition and CT image contrast enhancement after surgery are solved, and the accuracy of high-precision recognition and contrast enhancement is achieved, reducing false positives and adverse reactions .

CN118415755BActive Publication Date: 2024-11-26ZHUHAI PEOPLES HOSPITAL GUANGDONG PROVINCE
View PDF 9 Cites 0 Cited by

Patent Information

Application Number
CN202410553712.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-05-07
Publication Date
2024-11-26
Estimated Expiration
2044-05-07

AI Technical Summary

Technical Problem

The prior art is difficult to accurately identify and count surgical needles after surgery, resulting in false positive problems. Traditional contrast enhancement methods have limitations on adverse reactions in patients with renal dysfunction, and the CT image contrast enhancement process is affected by respiratory and digestive peristalsis.

Method used

The needle detection method based on feature extraction is adopted to screen the needle candidates through area, density, contrast noise ratio, and curve fitting, and use the generative adversarial network to automatically synthesize contrast-enhanced CT images from the original CT images, introducing perceptual loss for dual-path training. Accurately synthesize the intensity details of the CT image and preserve the texture structure.

Benefits of technology

High-precision surgical needle recognition is achieved, false positives are reduced, adverse reactions of contrast agents are avoided, and the accuracy and stability of enhanced contrast of CT images are improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118415755B_ABST
    Figure CN118415755B_ABST
Patent Text Reader

Abstract

The present invention belongs to the field of medical foreign body identification, specifically refers to a computer-assisted surgery system for high-precision foreign body positioning and removal, the system includes a CT image acquisition device, a contrast enhancement subsystem, a needle candidate object segmentation module, a needle candidate object screening module and a needle foreign body removal module. This scheme proposes a more general needle detection method based on feature extraction to complete the task of high-precision surgical needle identification; adopts a method based on setting rule thresholds to eliminate a large number of false positives; proposes a training framework based on generative adversarial networks to automatically synthesize contrast-enhanced CT images directly from original CT images to avoid adverse reactions that may be caused by contrast agents; introduces perceptual loss to compare the high-level semantic differences between virtual enhanced CT images and actual enhanced CT images, and proposes a dual-path training mode to more accurately synthesize the intensity details of CT images and retain their texture structure.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the field of medical foreign body identification, and in particular is a computer-assisted surgery system for high-precision foreign body positioning and removal. Background Art

[0002] Foreign objects inadvertently left in a patient's body after surgery, such as surgical sponges, surgical needles, sutures, and other surgical instruments, can cause serious medical problems, with retained foreign bodies (RFO) likely to occur at least once in every 7,000 surgeries, leading to sepsis, fistulas, visceral perforation, and even death. At present, the typical solution to prevent accidents of foreign body residue is to manually count 50 surgical instruments (surgical sponges, surgical needles, sutures, etc.) before and after surgery to check for differences. There is now a 3D sponge micro-label attached to the surgical sponge to improve the visibility of the sponge on X-rays, but this method relies heavily on the accuracy of radiologists' identification and counting from X-rays, and is susceptible to human error. The existing technology uses CAD systems to assist in calculations, but traditional image analysis methods are specifically designed for detecting sponge micro-labels, and lack technical means for using CAD systems to achieve high-precision foreign body detection for surgical needles; there is a problem that due to the large number of high-contrast anatomical structures in the human body, it is difficult to distinguish surgical needles from bone tissue only through traditional deep learning methods, which will lead to a large number of false positives; in magnetic resonance imaging, contrast agents are usually used to highlight blood vessels and organ structures, but these drugs may cause allergic reactions and nephrotoxicity, and there is a problem that traditional contrast enhancement methods are limited in use in patients with renal dysfunction; in the process of contrast enhancement of CT images, there is a problem that respiratory and digestive peristalsis will affect the training and learning of generative adversarial networks; Summary of the invention

[0003] In view of the above situation, in order to overcome the defects of the prior art, the present invention provides a computer-assisted surgery system for high-precision foreign body positioning and removal. The typical solution to prevent accidents of foreign body residue is to manually count 50 surgical instruments (surgical sponges, surgical needles, sutures, etc.) before and after surgery to check for differences. There is now a 3D sponge micro-label attached to the surgical sponge, which improves the visibility of the sponge on X-rays, but this method relies heavily on the accuracy of radiologists' identification and counting from X-rays, and is susceptible to human error. The prior art uses CAD systems to assist in calculations, but traditional image analysis methods are specially designed for detecting sponge micro-labels, and lack the technical means to achieve high-precision foreign body detection for surgical needles using CAD systems. This solution proposes a more general needle detection method based on feature extraction to complete the task of high-precision surgical needle recognition; there are a large number of high-contrast anatomical structures in the human body, and it is difficult to distinguish between surgical needles and bone tissue using traditional deep learning methods alone, which will lead to a large number of false positives. This scheme adopts a method based on setting rule thresholds, imposes multiple constraints, and screens needle candidate objects by area, density, contrast-to-noise ratio, and curve fitting to eliminate a large number of false positives; in magnetic resonance imaging, contrast agents are usually used to highlight blood vessels and organ structures, but these drugs may cause allergic reactions or nephrotoxicity, and there is a problem that traditional contrast enhancement methods are limited in use in patients with renal dysfunction. This scheme proposes a training framework based on a generative adversarial network to automatically synthesize contrast-enhanced CT images directly from original CT images to avoid adverse reactions that may be caused by contrast agents; in the process of contrast enhancement of CT images, there is a problem that respiratory and digestive motility affect the training and learning of the generative adversarial network. This scheme introduces perceptual loss to compare the high-level semantic differences between virtual enhanced CT images and actual enhanced CT images, and proposes a dual-path training mode to more accurately synthesize the intensity details of CT images and retain their texture structure.

[0004] The technical solution adopted by the present invention is as follows: the present invention provides a computer-assisted surgery system for high-precision foreign body positioning and removal, the system comprising a CT image acquisition device, a contrast enhancement subsystem, a needle candidate object segmentation module, a needle candidate object screening module and a needle foreign body removal module;

[0005] The CT image acquisition device acquires the original CT image of the patient;

[0006] The contrast enhancement subsystem uses a generative adversarial network to enhance the contrast of the original CT image and output a virtual enhanced image;

[0007] The needle candidate object segmentation module segments the virtual enhanced image to obtain the needle candidate object;

[0008] The needle candidate screening module screens needle candidate objects by area, density, contrast-to-noise ratio, and curve fitting to identify needle foreign bodies;

[0009] The needle foreign body removal module informs the number and location of the needle foreign bodies, and the medical staff takes corresponding surgical operations and interventional treatments to remove the needle foreign bodies;

[0010] The high-precision foreign body positioning and removal computer-assisted surgery system uses a high-precision foreign body positioning and removal computer-assisted surgery method to detect needle foreign bodies remaining in the patient's body;

[0011] The computer-assisted surgery method for high-precision foreign body positioning and removal comprises the following steps:

[0012] Step S1: The CT image acquisition device acquires the original CT image of the patient;

[0013] Step S2: contrast enhancement; the contrast enhancement subsystem uses a generative adversarial network to learn a mapping from an original CT image to an actual contrast-enhanced CT image, and outputs a virtual contrast-enhanced CT image; the actual contrast-enhanced CT image is recorded as an actual enhanced image, and the virtual contrast-enhanced CT image is recorded as a virtual enhanced image;

[0014] Step S3: segmenting the virtual enhanced image to obtain a candidate needle object;

[0015] Step S4: Screening needle candidate objects by area;

[0016] Step S5: Screening needle candidate objects by density;

[0017] Step S6: Screening needle candidate objects by contrast-to-noise ratio;

[0018] Step S7: Screening needle candidate objects by geometric parameters;

[0019] Step S8: Screening needle candidate objects by curve fitting;

[0020] Step S9: Medical personnel take appropriate surgical and interventional treatments to remove the needle foreign bodies according to the number and location of the detected needle foreign bodies.

[0021] Furthermore, in step S2, contrast enhancement specifically includes the following steps:

[0022] Step S231: Global path, the generator receives the original CT image, extracts the global structural features, and generates a global virtual enhanced image; calculates the global loss and optimizes the generator. The formula used is as follows:

[0023] ;

[0024] In the formula, represents the global loss, Indicates that the optimal parameters of the generator are obtained by gradient descent method. It means that given the generator, we need to find the discriminator that maximizes the overall loss function. represents the overall loss function, represents a hyperparameter used to balance the weights between feature loss terms, which represent the difference between the feature representations of generated images and real images at different levels. represents a generator, represents the mean square error, represents the sum of feature loss terms, represents a virtual enhanced image, represents the actual enhanced image, is a pre-trained model for extracting features. Indicates different levels of feature representation;

[0025] Step S232: local path, dividing the original CT image into four sub-blocks, multiplying the sub-blocks by four different intensity coefficients respectively, extracting more detailed local texture features, and generating four corresponding local virtual enhanced images after the four sub-blocks are processed by the generator, and the four local virtual enhanced images are integrated into a complete image as the local virtual enhanced image;

[0026] Step S233: Perceptual loss is a loss function that compares the differences between feature representations. During the training phase, the local regression loss and the local perceptual loss between the local virtual enhanced image and the actual enhanced image are calculated respectively. The local regression loss and the local perceptual loss are combined and recorded as the local loss.

[0027] Step S234: The sum of the global loss and the local loss is recorded as the objective function. By minimizing the objective function, the generative adversarial network is optimized. The objective function calculation formula is as follows:

[0028] ;

[0029] In the formula, and is a hyperparameter used to adjust the weights of dual-channel training, represents the objective function, represents the global loss, Indicates local loss.

[0030] Furthermore, in step S3, the virtual enhanced image is segmented to obtain a needle candidate object, which specifically includes the following steps:

[0031] Step S31: Select three linear frame filters that meet the following conditions:

[0032] ;

[0033] Wherein, M1 represents the kernel size of the first linear frame filter, M2 represents the kernel size of the second linear frame filter, and M3 represents the kernel size of the third linear frame filter;

[0034] Step S32: enhancing the contrast of the needle tip in the virtual enhanced image, the formula used is as follows:

[0035] ;

[0036] In the formula, It represents the pixel value of the virtual enhanced image after contrast enhancement using a linear frame filter. , and Respectively represent the results of filtering the virtual enhanced image using the first, second and third box filters;

[0037] Step S33: Record the filtered virtual enhanced image as a CNR enhanced image, and output the CNR enhanced image;

[0038] Step S34: remove the background, perform bandpass filtering on the CNR enhanced image, and add a constant gray value to all pixels of the CNR enhanced image to prevent the pixel value from being lower than 0;

[0039] Step S35: Image segmentation: The CNR enhanced image is segmented into background and salient objects by a preset grayscale threshold. Saliency refers to the property of attracting human eye attention. Salient objects refer to the object areas in the image that are visually significant. The region growing technology is used to identify the needles in the salient objects to obtain needle candidate objects.

[0040] Furthermore, in step S4, the needle candidate objects are screened by area, which specifically includes the following steps:

[0041] Step S41: determining the area of ​​each needle candidate object, that is, the total number of pixels;

[0042] Step S42: setting the upper and lower limits of the area of ​​the needle candidate objects. Only the needle candidate objects whose areas meet the following conditions are retained, otherwise they are removed:

[0043] ;

[0044] In the formula, represents the area of ​​the needle candidate object, Indicates the lower limit of area, Indicates the upper limit of area.

[0045] Further, in step S7, the needle candidate objects are screened by geometric parameters, specifically including: constraining the geometric parameters, the virtual box is a rectangular frame surrounding the entire needle candidate object, the geometric parameters are composed of the virtual box height and the virtual box width, and constraints are imposed on the geometric parameters. Needle candidate objects that satisfy the virtual box height being between the upper and lower limits of the height and the virtual box width being between the upper and lower limits of the width are retained, otherwise they are removed:

[0046] ;

[0047] ;

[0048] In the formula, Indicates the virtual frame height, Indicates the lower limit of the virtual frame height. Indicates the upper limit of the virtual frame height. Indicates the virtual frame width, Indicates the lower limit of the virtual frame width. Indicates the lower limit of the virtual frame width.

[0049] Furthermore, in step S5, the needle candidate objects are screened by density, which specifically includes the following steps:

[0050] Step S51: Determine the density of the needle candidate object, and the calculation formula is as follows:

[0051] ;

[0052] In the formula, represents the density of needle candidate objects, represents the pixel index, Represents the total number of pixels, Indicates The gray value of a pixel, represents the sum of the grayscale values ​​within the needle candidate object, represents the threshold of the contrast-to-noise ratio, Indicates when Greater than hour Only then will they participate in the calculation;

[0053] Step S52: The needle candidate objects whose density meets the following conditions are retained, otherwise they are removed:

[0054] ;

[0055] In the formula, Indicates the lower density limit.

[0056] Furthermore, in step S6, the needle candidate objects are screened by contrast-to-noise ratio, which specifically includes the following steps:

[0057] Step S61: It is an indicator that measures the ratio of the contrast between the target object and the background to the noise. This means that the contrast between the needle candidate and the background area is high, and the needle candidate is easier to distinguish. The calculation formula is as follows:

[0058] ;

[0059] In the formula, represents the contrast-to-noise ratio, represents the average pixel value of the needle candidate object, represents the average pixel value of the background area, represents the standard deviation of background noise;

[0060] Step S62: Setting Threshold, yes Apply constraints to exclude bones and muscles from being present in the needle candidates:

[0061] ;

[0062] In the formula, express Threshold.

[0063] Furthermore, in step S8, the needle candidate objects are screened by curve fitting, which specifically includes the following steps:

[0064] Step S81: Perform curve fitting based on the uniqueness of the needle shape, and use the least squares method to fit the needle candidate object to the quadratic curve equation:

[0065] ;

[0066] In the formula, represents the equation of a quadratic curve, and The curve in and The first-order deviation in the direction, represents the rotation and tilt cross term that affects the curve, , represents the quadratic term that determines the shape of the curve, and F represents the constant term that determines the position of the curve;

[0067] Step S82: Calculate the normal distance, the formula is as follows:

[0068] ;

[0069] In the formula, represents the normal distance, that is, the vertical distance from each pixel of the needle candidate object to the fitting curve, represents the equation of a quadratic curve, represents the gradient size, represents the total number of pixels in the needle candidate;

[0070] Step S83: constrain the normal distance, mark the needle candidate objects whose normal distance is less than or equal to the normal distance threshold as detected needle foreign bodies, and remove the needle candidate objects whose normal distance is greater than the normal distance threshold. The formula used is as follows:

[0071] ;

[0072] In the formula, Indicates the normal distance threshold.

[0073] The beneficial effects achieved by the present invention using the above scheme are as follows:

[0074] (1) The typical solution to prevent accidents of foreign body residue is to manually count 50 surgical instruments (surgical sponges, surgical needles, sutures, etc.) before and after surgery to check for differences. There is now a 3D sponge micro-label attached to the surgical sponge, which improves the visibility of the sponge on X-rays, but this method relies heavily on the accuracy of radiologists' identification and counting from X-rays, and is susceptible to human error. The existing technology uses CAD systems to assist in calculations, but traditional image analysis methods are specifically designed for detecting sponge micro-labels, and lack the technical means to use CAD systems to achieve high-precision foreign body detection for surgical needles. This solution proposes a more general needle detection method based on feature extraction to complete the task of high-precision surgical needle recognition;

[0075] (2) There are a large number of high-contrast anatomical structures in the human body. It is difficult to distinguish surgical needles from bone tissue using traditional deep learning methods alone, which will lead to a large number of false positives. This scheme adopts a method based on setting rule thresholds, imposes multiple constraints, and screens needle candidates by area, density, contrast-to-noise ratio, and curve fitting, eliminating a large number of false positives.

[0076] (3) In magnetic resonance imaging, contrast agents are often used to highlight blood vessels and organ structures. However, these drugs may cause allergic reactions or nephrotoxicity. Traditional contrast enhancement methods are limited in patients with renal dysfunction. This paper proposes a training framework based on generative adversarial networks to automatically synthesize contrast-enhanced CT images directly from raw CT images, avoiding the adverse reactions that may be caused by contrast agents.

[0077] (4) To address the problem that the training and learning of the generative adversarial network is affected by respiratory and digestive motility during the contrast enhancement process of CT images, this scheme introduces perceptual loss to compare the high-level semantic differences between virtual enhanced CT images and actual enhanced CT images, and proposes a dual-path training model to more accurately synthesize the intensity details of CT images and preserve their texture structure. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 A schematic diagram of a computer-assisted surgery system for high-precision foreign body positioning and removal provided by the present invention;

[0079] Figure 2 A schematic diagram of a flow chart of a computer-assisted surgical method for high-precision foreign body positioning and removal provided by the present invention;

[0080] Figure 3 It is a schematic diagram of step S232.

[0081] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention. DETAILED DESCRIPTION

[0082] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0083] Example 1, see Figure 1 and Figure 2 , the present invention provides a computer-assisted surgery system for high-precision foreign body positioning and removal, the system comprising a CT image acquisition device, a contrast enhancement subsystem, a needle candidate object segmentation module, a needle candidate object screening module and a needle foreign body removal module;

[0084] The CT image acquisition device acquires the original CT image of the patient;

[0085] The contrast enhancement subsystem uses a generative adversarial network to enhance the contrast of the original CT image and output a virtual enhanced image;

[0086] The needle candidate object segmentation module segments the virtual enhanced image to obtain the needle candidate object;

[0087] The needle candidate screening module screens needle candidate objects by area, density, contrast-to-noise ratio, and curve fitting to identify needle foreign bodies;

[0088] The needle foreign body removal module informs the number and location of the needle foreign bodies, and the medical staff takes corresponding surgical operations and interventional treatments to remove the needle foreign bodies;

[0089] The high-precision foreign body positioning and removal computer-assisted surgery system uses a high-precision foreign body positioning and removal computer-assisted surgery method to detect needle foreign bodies remaining in the patient's body;

[0090] The computer-assisted surgery method for high-precision foreign body positioning and removal comprises the following steps:

[0091] Step S1: The CT image acquisition device acquires the original CT image of the patient;

[0092] Step S2: contrast enhancement; the contrast enhancement subsystem uses a generative adversarial network to learn a mapping from an original CT image to an actual contrast-enhanced CT image, and outputs a virtual contrast-enhanced CT image; the actual contrast-enhanced CT image is recorded as an actual enhanced image, and the virtual contrast-enhanced CT image is recorded as a virtual enhanced image;

[0093] Step S3: segmenting the virtual enhanced image to obtain a candidate needle object;

[0094] Step S4: Screening needle candidate objects by area;

[0095] Step S5: Screening needle candidate objects by density;

[0096] Step S6: Screening needle candidate objects by contrast-to-noise ratio;

[0097] Step S7: Screening needle candidate objects by geometric parameters;

[0098] Step S8: Screening needle candidate objects by curve fitting;

[0099] Step S9: Medical personnel take appropriate surgical and interventional treatments to remove the needle foreign bodies according to the number and location of the detected needle foreign bodies.

[0100] Example 2, see Figure 1 and Figure 2 This embodiment is based on the above embodiment. In step S2, contrast enhancement specifically includes the following steps:

[0101] Step S21: The generative adversarial network includes a generator and a discriminator. The generator learns the mapping from the original CT image to the actual enhanced image, predicts and synthesizes the virtual enhanced image, and the discriminator is responsible for distinguishing the actual enhanced image from the virtual enhanced image.

[0102] Step S22: construct a pre-training model to learn feature representations of different levels of CT images;

[0103] Step S23: Dual-channel training.

[0104] Example 3, see Figure 1 , Figure 2 and Figure 3 This embodiment is based on the above embodiment. In step S23, dual-channel training specifically includes the following steps:

[0105] Step S231: Global path, the generator receives the original CT image, extracts the global structural features, and generates a global virtual enhanced image; calculates the global loss and optimizes the generator. The formula used is as follows:

[0106] ;

[0107] In the formula, represents the global loss, Indicates that the optimal parameters of the generator are obtained by gradient descent method. It means that given the generator, we need to find the discriminator that maximizes the overall loss function. represents the overall loss function, represents a hyperparameter used to balance the weights between feature loss terms, which represent the difference between the feature representations of generated images and real images at different levels. represents a generator, represents the mean square error, represents the sum of feature loss terms, represents a virtual enhanced image, represents the actual enhanced image, is a pre-trained model for extracting features. Indicates different levels of feature representation;

[0108] Step S232: local path, dividing the original CT image into four sub-blocks, multiplying the sub-blocks by four different intensity coefficients respectively, extracting more detailed local texture features, and generating four corresponding local virtual enhanced images after the four sub-blocks are processed by the generator, and the four local virtual enhanced images are integrated into a complete image as the local virtual enhanced image;

[0109] Step S233: Perceptual loss is a loss function that compares the differences between feature representations. During the training phase, the local regression loss and the local perceptual loss between the local virtual enhanced image and the actual enhanced image are calculated respectively. The local regression loss and the local perceptual loss are combined and recorded as the local loss.

[0110] Step S234: The sum of the global loss and the local loss is recorded as the objective function. By minimizing the objective function, the generative adversarial network is optimized. The objective function calculation formula is as follows:

[0111] ;

[0112] In the formula, and is a hyperparameter used to adjust the weights of dual-channel training, represents the objective function, represents the global loss, Indicates local loss.

[0113] Example 4, see Figure 1 and Figure 2 This embodiment is based on the above embodiment. In step S3, the virtual enhanced image is segmented to obtain a needle candidate object, which specifically includes the following steps:

[0114] Step S31: Select three linear frame filters that meet the following conditions:

[0115] ;

[0116] Wherein, M1 represents the kernel size of the first linear frame filter, M2 represents the kernel size of the second linear frame filter, and M3 represents the kernel size of the third linear frame filter;

[0117] Step S32: enhancing the contrast of the needle tip in the virtual enhanced image, the formula used is as follows:

[0118] ;

[0119] In the formula, It represents the pixel value of the virtual enhanced image after contrast enhancement using a linear frame filter. , and Respectively represent the results of filtering the virtual enhanced image using the first, second and third box filters;

[0120] Step S33: Record the filtered virtual enhanced image as Enhanced image, output Enhance images;

[0121] Step S34: Remove the background. The enhanced image is bandpass filtered to give Enhance all pixels of the image by adding a constant grayscale value to avoid pixel values ​​below 0;

[0122] Step S35: Image segmentation, by presetting the grayscale threshold The enhanced image is segmented into background and salient objects. Saliency refers to the property of attracting human eye attention. Salient objects refer to the object regions in the image that are visually significant. The region growing technology is used to identify the needle among the salient objects and obtain the needle candidate objects.

[0123] Example 5, see Figure 1 and Figure 2 This embodiment is based on the above embodiment. In step S4, the needle candidate objects are screened by area, which specifically includes the following steps:

[0124] Step S41: determining the area of ​​each needle candidate object, that is, the total number of pixels;

[0125] Step S42: setting the upper and lower limits of the area of ​​the needle candidate objects. Only the needle candidate objects whose areas meet the following conditions are retained, otherwise they are removed:

[0126] ;

[0127] In the formula, represents the area of ​​the needle candidate object, Indicates the lower limit of area, Indicates the upper limit of area.

[0128] Example 6, see Figure 1 and Figure 2 This embodiment is based on the above embodiment. In step S7, the needle candidate objects are screened by geometric parameters, specifically including: constraining the geometric parameters. The virtual box is a rectangular frame surrounding the entire needle candidate object. The geometric parameters are composed of the virtual box height and the virtual box width. Constraints are imposed on the geometric parameters. Needle candidate objects that meet the virtual box height between the upper and lower height limits and the virtual box width between the upper and lower width limits are retained, otherwise they are removed:

[0129] ;

[0130] ;

[0131] In the formula, Indicates the virtual frame height, Indicates the lower limit of the virtual frame height. Indicates the upper limit of the virtual frame height. Indicates the virtual frame width, Indicates the lower limit of the virtual frame width. Indicates the lower limit of the virtual frame width.

[0132] Embodiment 7, see Figure 1 and Figure 2This embodiment is based on the above embodiment. In step S5, the needle candidate objects are screened by density, which specifically includes the following steps:

[0133] Step S51: Determine the density of the needle candidate object, and the calculation formula is as follows:

[0134] ;

[0135] In the formula, represents the density of needle candidate objects, represents the pixel index, Represents the total number of pixels, Indicates The gray value of a pixel, represents the sum of the grayscale values ​​within the needle candidate object, represents the threshold of the contrast-to-noise ratio, Indicates when Greater than hour Only then will they participate in the calculation;

[0136] Step S52: The needle candidate objects whose density meets the following conditions are retained, otherwise they are removed:

[0137] ;

[0138] In the formula, Indicates the lower density limit.

[0139] Embodiment 9, see Figure 1 and Figure 2 This embodiment is based on the above embodiment. In step S6, the needle candidate objects are screened by contrast-to-noise ratio, which specifically includes the following steps:

[0140] Step S61: It is an indicator that measures the ratio of the contrast between the target object and the background to the noise. This means that the contrast between the needle candidate and the background area is high, and the needle candidate is easier to distinguish. The calculation formula is as follows:

[0141] ;

[0142] In the formula, represents the contrast-to-noise ratio, represents the average pixel value of the needle candidate object, represents the average pixel value of the background area, represents the standard deviation of background noise;

[0143] Step S62: Setting Threshold, yes Apply constraints to exclude bones and muscles from being present in the needle candidates:

[0144] ;

[0145] In the formula, express Threshold.

[0146] Embodiment 10, see Figure 1 and Figure 2 This embodiment is based on the above embodiment. In step S8, the needle candidate objects are screened by curve fitting, which specifically includes the following steps:

[0147] Step S81: Perform curve fitting based on the uniqueness of the needle shape, and use the least squares method to fit the needle candidate object to the quadratic curve equation:

[0148] ;

[0149] In the formula, represents the equation of a quadratic curve, and The curve in and The first-order deviation in the direction, represents the rotation and tilt cross term that affects the curve, , represents the quadratic term that determines the shape of the curve, and F represents the constant term that determines the position of the curve;

[0150] Step S82: Calculate the normal distance, the formula is as follows:

[0151] ;

[0152] In the formula, represents the normal distance, that is, the vertical distance from each pixel of the needle candidate object to the fitting curve, represents the equation of a quadratic curve, represents the gradient size, represents the total number of pixels in the needle candidate;

[0153] Step S83: constrain the normal distance, mark the needle candidate objects whose normal distance is less than or equal to the normal distance threshold as detected needle foreign bodies, and remove the needle candidate objects whose normal distance is greater than the normal distance threshold. The formula used is as follows:

[0154] ;

[0155] In the formula, Indicates the normal distance threshold.

[0156] Example 11, see Figure 1 and Figure 2 This embodiment is based on the above embodiment. In step S22, the feature representations at different levels specifically refer to low-level features (including edges, textures), mid-level features (including shapes, partial objects) and high-level features (including object categories, scenes).

[0157] Example 12, see Figure 1 and Figure 2 This embodiment is based on the above embodiment. In step S232, the intensity coefficient is a parameter used to simulate different contrast enhancement levels. The four different intensity coefficients are 0.5, 1.0, 1.5, and 2.0 respectively.

[0158] Example 13, see Figure 1 and Figure 2 This embodiment is based on the above embodiment. In step S234, Set to 0.6, Set to 0.4.

[0159] Example 14, see Figure 1 and Figure 2 This embodiment is based on the above embodiment. In step S31, the kernel size of the first linear frame filter is set to 13 pixels (1.625 mm), the kernel size of the second linear frame filter is set to 9 pixels (1.125 mm), and the kernel size of the third linear frame filter is set to 5 pixels (0.625 mm).

[0160] Example 15, see Figure 1 and Figure 2 This embodiment is based on the above embodiment. In step S34, after bandpass filtering, the average grayscale value of the CNR enhanced image is 0, and the constant grayscale value is set to 2183.

[0161] Example 16, see Figure 1 and Figure 2 This embodiment is based on the above embodiment. In step S35, the region growing technique is used to determine whether adjacent pixels belong to the same object through 8-connectivity. The python code is implemented as follows:

[0162] import cv2

[0163] import numpy as np

[0164] class Point(object):

[0165] def __init__(self, x, y):

[0166] self.x = x

[0167] self.y = y

[0168] def getX(self):

[0169] return self.x

[0170] def getY(self):

[0171] return self.y

[0172] connects = [Point(-1, -1), Point(0, -1), Point(1, -1), Point(1, 0),

[0173] Point(1, 1), Point(0, 1), Point(-1, 1), Point(-1, 0)]

[0174] # Calculate the Euclidean distance between two points

[0175] def get_dist(seed_location1, seed_location2):

[0176] l1 = im[seed_location1.x, seed_location1.y]

[0177] l2 = im[seed_location2.x, seed_location2.y]

[0178] count = np.sqrt(np.sum(np.square(l1 - l2)))

[0179] return count

[0180] # import Image

[0181] im = cv2.imread('. / image / 222.jpg')

[0182] cv2.imshow('src', im)

[0183] cv2.waitKey(0)

[0184] cv2.destroyAllWindows()

[0185] im_shape = im.shape

[0186] height = im_shape[0]

[0187] width = im_shape[1]

[0188] print('the shape of image :', im_shape)

[0189] # Mark to determine whether the seed has grown

[0190] img_mark = np.zeros([height, width])

[0191] cv2.imshow('img_mark', img_mark)

[0192] cv2.waitKey(0)

[0193] cv2.destroyAllWindows()

[0194] # Create an empty image array as a class

[0195] img_re = im.copy()

[0196] for i in range(height):

[0197] for j in range(width):

[0198] img_re[i, j][0] = 0

[0199] img_re[i, j][1] = 0

[0200] img_re[i, j][2] = 0

[0201] cv2.imshow('img_re', img_re)

[0202] cv2.waitKey(0)

[0203] cv2.destroyAllWindows()

[0204] # Take a point as the seed point

[0205] seed_list = []

[0206] seed_list.append(Point(15, 15))

[0207] T = 7 # threshold

[0208] class_k = 1 # Category

[0209] #Grow a class

[0210] while (len(seed_list) > 0):

[0211] seed_tmp = seed_list[0]

[0212] # Remove the growing points from the seed point list of a class

[0213] seed_list.pop(0)

[0214] img_mark[seed_tmp.x, seed_tmp.y] = class_k

[0215] # Traverse 8 neighborhoods

[0216] for i in range(8):

[0217] tmpX = seed_tmp.x + connects[i].x

[0218] tmpY = seed_tmp.y + connects[i].y

[0219] if (tmpX < 0 or tmpY < 0 or tmpX >= height or tmpY >= width):

[0220] continue

[0221] dist = get_dist(seed_tmp, Point(tmpX, tmpY))

[0222] # Grow the points that meet the conditions in the seed set

[0223] if (dist < T and img_mark[tmpX, tmpY] == 0):

[0224] img_re[tmpX, tmpY][0] = im[tmpX, tmpY][0]

[0225] img_re[tmpX, tmpY][1] = im[tmpX, tmpY][1]

[0226] img_re[tmpX, tmpY][2] = im[tmpX, tmpY][2]

[0227] img_mark[tmpX, tmpY] = class_k

[0228] seed_list.append(Point(tmpX, tmpY))

[0229] # Output image

[0230] cv2.imshow('OUTIMAGE', img_re)

[0231] cv2.waitKey(0)

[0232] cv2.destroyAllWindows().

[0233] Embodiment 17, see Figure 1 and Figure 2 This embodiment is based on the above embodiment. In step S4, = 1300 pixels (20.31 mm2), = 6500 pixels (101.56 square millimeters).

[0234] Embodiment 18, see Figure 1 and Figure 2 This embodiment is based on the above embodiment. In step S51, =2185.

[0235] Embodiment 19, see Figure 1 and Figure 2 This embodiment is based on the above embodiment. In step S52, =2250.

[0236] Embodiment 20, see Figure 1 and Figure 2 This embodiment is based on the above embodiment. In step S62, =3.45.

[0237] Example 21, see Figure 1 and Figure 2 This embodiment is based on the above embodiment. In step S7, and The value was chosen to be 70 pixels (8.75 mm), and The best value was chosen to be 260 pixels (32.50 mm).

[0238] Example 22, see Figure 1 and Figure 2 This embodiment is based on the above embodiment. In step S81, an ellipse or a hyperbola is fitted to a needle candidate object, and a fitting degree of the needle candidate object to the ellipse or the hyperbola is estimated to determine whether the detected object is a needle.

[0239] Embodiment 23, see Figure 1 and Figure 2 This embodiment is based on the above embodiment. In step S83, = 8.0 pixels (0.16 mm2).

[0240] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0241] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

[0242] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.

Claims

1. A computer-assisted surgical system for high-precision foreign body positioning and removal, characterized in that: The system includes a CT image acquisition device, a contrast enhancement subsystem, a needle candidate object segmentation module, a needle candidate object screening module and a needle foreign body removal module; The CT image acquisition device acquires the original CT image of the patient; The contrast enhancement subsystem uses a generative adversarial network to enhance the contrast of the original CT image and output a virtual enhanced image; The needle candidate object segmentation module segments the virtual enhanced image to obtain the needle candidate object; The needle candidate screening module screens needle candidate objects by area, density, contrast-to-noise ratio, and curve fitting to identify needle foreign bodies; The needle foreign body removal module informs the number and location of the needle foreign bodies, and the medical staff takes corresponding surgical operations and interventional treatments to remove the needle foreign bodies; The high-precision foreign body positioning and removal computer-assisted surgery system uses a high-precision foreign body positioning and removal computer-assisted surgery method to detect needle foreign bodies remaining in the patient's body. The high-precision foreign body positioning and removal computer-assisted surgery method includes the following steps: Step S1: The CT image acquisition device acquires the original CT image of the patient; Step S2: contrast enhancement; the contrast enhancement subsystem uses a generative adversarial network to learn a mapping from an original CT image to an actual contrast-enhanced CT image, and outputs a virtual contrast-enhanced CT image; the actual contrast-enhanced CT image is recorded as an actual enhanced image, and the virtual contrast-enhanced CT image is recorded as a virtual enhanced image; Step S3: segmenting the virtual enhanced image to obtain a candidate needle object; Step S4: Screening needle candidate objects by area; Step S5: Screening needle candidate objects by density; Step S6: Screening needle candidate objects by contrast-to-noise ratio; Step S7: Screening needle candidate objects by geometric parameters; Step S8: Screening needle candidate objects by curve fitting; Step S9: The medical staff takes appropriate surgical and interventional treatments to remove the needle foreign bodies according to the number and location of the detected needle foreign bodies; In step S2, contrast enhancement is performed, which specifically includes the following steps: Step S21: The generative adversarial network includes a generator and a discriminator. The generator learns the mapping from the original CT image to the actual enhanced image, predicts and synthesizes the virtual enhanced image, and the discriminator is responsible for distinguishing the actual enhanced image from the virtual enhanced image. Step S22: construct a pre-training model to learn feature representations of different levels of CT images; Step S23: dual channel training; In step S23, dual-channel training specifically includes the following steps: Step S231: Global path, the generator receives the original CT image, extracts the global structural features, and generates a global virtual enhanced image; calculates the global loss and optimizes the generator. The formula used is as follows: ; In the formula, represents the global loss, Indicates that the optimal parameters of the generator are obtained by gradient descent method. It means that given the generator, we need to find the discriminator that maximizes the overall loss function. represents the overall loss function, represents a hyperparameter used to balance the weights between feature loss terms, which represent the difference between the feature representations of generated images and real images at different levels. represents a generator, represents the mean square error, represents the sum of feature loss terms, represents a virtual enhanced image, represents the actual enhanced image, is a pre-trained model for extracting features. Indicates different levels of feature representation; Step S232: local path, dividing the original CT image into four sub-blocks, multiplying the sub-blocks by four different intensity coefficients respectively, extracting more detailed local texture features, and generating four corresponding local virtual enhanced images after the four sub-blocks are processed by the generator, and the four local virtual enhanced images are integrated into a complete image as the local virtual enhanced image; Step S233: Perceptual loss is a loss function that compares the differences between feature representations. During the training phase, the local regression loss and the local perceptual loss between the local virtual enhanced image and the actual enhanced image are calculated respectively. The local regression loss and the local perceptual loss are combined and recorded as the local loss. Step S234: The sum of the global loss and the local loss is recorded as the objective function. By minimizing the objective function, the generative adversarial network is optimized. The objective function calculation formula is as follows: ; In the formula, and is a hyperparameter used to adjust the weights of dual-channel training, represents the objective function, represents the global loss, Indicates local loss; In step S3, the virtual enhanced image is segmented to obtain a candidate needle object, which specifically includes the following steps: Step S31: Select three linear frame filters that meet the following conditions: ; Wherein, M1 represents the kernel size of the first linear frame filter, M2 represents the kernel size of the second linear frame filter, and M3 represents the kernel size of the third linear frame filter; Step S32: enhancing the contrast of the needle tip in the virtual enhanced image, the formula used is as follows: ; In the formula, It represents the pixel value of the virtual enhanced image after contrast enhancement using a linear frame filter. , and Respectively represent the results of filtering the virtual enhanced image using the first, second and third box filters; Step S33: Record the filtered virtual enhanced image as a CNR enhanced image, and output the CNR enhanced image; Step S34: remove the background, perform bandpass filtering on the CNR enhanced image, and add a constant gray value to all pixels of the CNR enhanced image to prevent the pixel value from being lower than 0; Step S35: Image segmentation: The CNR enhanced image is segmented into background and salient objects by a preset grayscale threshold. Saliency refers to the property of attracting human eye attention. Salient objects refer to the object areas in the image that are visually significant. The region growing technology is used to identify the needles in the salient objects to obtain needle candidate objects.

2. The computer-assisted surgery system for high-precision foreign body positioning and removal according to claim 1, characterized in that: In step S6, the needle candidate objects are screened by contrast-to-noise ratio, which specifically includes the following steps: Step S61: Represents the contrast-to-noise ratio, which is an indicator to measure the ratio of contrast to noise between the target object and the background. ; Step S62: Setting Threshold, yes Imposing constraints to exclude Less than or equal to threshold needle candidates; In step S8, the needle candidate objects are screened by curve fitting, which specifically includes the following steps: Step S81: performing curve fitting according to the uniqueness of the needle shape, using the least square method to fit the needle candidate object to the quadratic curve equation; Step S82: Calculate the normal distance, the formula is as follows: ; In the formula, represents the normal distance, that is, the vertical distance from each pixel of the needle candidate object to the fitting curve, represents the equation of a quadratic curve, represents the gradient size, represents the total number of pixels in the needle candidate; Step S83: constraining the normal distance, marking the needle candidate objects whose normal distance is less than or equal to the normal distance threshold as detected needle foreign bodies, and removing the needle candidate objects whose normal distance is greater than the normal distance threshold.

Citation Information

Patent Citations

  • Brain tumor segmentation data enhancement method based on generative adversarial network

    CN111833359A

  • Method for automatically detecting foreign matters on chest DR images

    CN113362289A

  • Aortic dissection detection system based on non-enhanced CT image

    CN114723726A

  • Liver tumor image enhancement and segmentation method based on deep learning

    CN115578404A

  • Method for supporting detection of clustered micro-calcified product of digital x-ray image of mamma

    JP1994154201A