Kidney puncture AI auxiliary data processing method and system based on artificial intelligence

Abstract

The invention discloses a renal puncture AI auxiliary data processing method and system based on artificial intelligence, and the method comprises the steps: obtaining an ultrasonic image containing a kidney and internal blood vessels, and carrying out the noise suppression and contrast enhancement processing of the ultrasonic image; performing blood vessel region segmentation on blood vessels in the kidney in the ultrasonic image, and placing the segmented blood vessel region in a predefined rectangular frame; fuzzy motion caused by probe shaking in the predefined rectangular frame is corrected; inputting the ultrasonic image in the predefined rectangular frame into a trained kidney blood vessel segmentation model to obtain an ultrasonic image sequence containing an upper boundary line and a lower boundary line of the kidney; wherein the upper boundary line is positioned above the blood vessel and below the upper contour of the kidney; the lower boundary separates the lower contour of the kidney from the blood vessel; and outputting the ultrasonic image sequence to a kidney puncture auxiliary system for a doctor to use in a kidney puncture process. The upper boundary line and the lower boundary line of the kidney can be accurately recognized and extracted, and puncture accuracy and safety are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing and artificial intelligence-assisted diagnosis, and particularly relates to a method and system for renal puncture AI-assisted data processing based on artificial intelligence, and a computing device. Background Art

[0002] Renal puncture is a commonly used operative technique in the diagnosis of kidney diseases. Usually, under the guidance of ultrasound, a biopsy needle is used to take a small amount of kidney tissue for detailed pathological examination and diagnosis. However, especially when determining the puncture path and avoiding important vascular structures, it highly depends on the experience and skills of doctors, and any carelessness may lead to serious complications such as bleeding and hematoma formation.

[0003] Currently, renal puncture usually relies on doctors to manually select the puncture path by holding an ultrasound probe in real time to observe the kidney and its internal vascular structures. Since the selection of the puncture path highly depends on the experience and judgment of doctors, there may be significant differences among different doctors. Moreover, it is difficult to completely avoid hand tremors when holding the ultrasound probe, which affects the stability and clarity of the image. In addition, doctors also need to repeatedly adjust the puncture path in the real-time ultrasound image to ensure avoiding important structures such as blood vessels, increasing the operation time and the risk to patients.

[0004] Although some image processing techniques (such as image filtering, enhancement, etc.) are also applied in renal puncture to improve the visual effect of ultrasound images. However, it is often limited to the improvement of image quality and cannot identify and label the upper and lower boundaries of the kidney. In addition, the problem of image blurring or offset caused by ultrasound probe tremors is not solved, which affects the selection of the puncture path.

[0005] To solve the above problems, the present invention proposes a method and system for renal puncture AI-assisted data processing based on artificial intelligence to accurately identify and extract the upper and lower boundaries of the kidney, thereby improving the accuracy and safety of puncture. Summary of the Invention

[0006] In view of the above problems, the present invention provides a method and system for renal puncture AI-assisted data processing based on artificial intelligence, and a computing device.

[0007] According to one aspect of the present invention, there is provided a method for renal puncture AI-assisted data processing based on artificial intelligence, including: Obtaining an ultrasound image including a kidney and its internal blood vessels, and performing noise suppression and contrast enhancement processing on the ultrasound image to improve the clarity of the puncture needle and blood vessel regions; Segmenting the blood vessels inside the kidney in the ultrasound image, and placing the segmented blood vessel regions in a predefined rectangular frame; Correct the blurred motion caused by probe jitter within the predefined rectangular frame; Input the ultrasonic image within the predefined rectangular frame into a pre-trained renal vascular segmentation model to obtain a sequence of ultrasonic images including the upper and lower boundaries of the kidney; wherein, the upper boundary is located above the blood vessel and below the upper contour of the kidney; the lower boundary separates the lower contour of the kidney from the blood vessel; Output the sequence of ultrasonic images to a renal puncture assistance system for use by a doctor during renal puncture.

[0008] In an alternative manner, the noise suppression of the ultrasonic image further includes: Remove speckle and granular noise in the ultrasonic image through a bilateral filtering algorithm and output the weighted average pixel value by neighborhood pixel values; the weighted spatial value affinity factor is calculated based on the spatial proximity and pixel value difference between pixels; Wherein, the calculation formula of the spatial value affinity factor is: ; Wherein, and are pixel coordinates; and are and corresponding pixel values; is the standard deviation of spatial proximity; is the standard deviation of pixel value difference.

[0009] In an alternative manner, the vascular region segmentation of the internal blood vessels of the kidney in the ultrasonic image further includes: Select one or more pixel points located in the blood vessel as initial seed points and check the neighborhood pixels of the initial seed points; If the similarity between the neighborhood pixels and the initial seed points meets the preset threshold condition, add the neighborhood pixels to the current vascular region and use them as new seed points to continue expanding; repeat until no new neighborhood pixels are added to the vascular region; Fill the internal holes of the blood vessel according to the closing operation morphological algorithm, and smooth the boundary of the blood vessel according to the opening operation morphological algorithm.

[0010] In an alternative manner, the correction of the blurred motion caused by probe jitter within the predefined rectangular frame further includes: Detect stable feature points in consecutive frames through the ORB feature point detection algorithm; Track the feature point motion of the stable feature points between consecutive frames through the Lucas-Kanade optical flow method; Estimate the global motion parameters of the predefined rectangular box based on the motion of the feature points; Apply an inverse transformation to the region within the predefined rectangular box according to the estimated global motion parameters to cancel out the blurred motion caused by probe jitter.

[0011] In an alternative manner, the renal vascular segmentation model includes an input layer, a feature extraction layer, a deformable convolution layer, an RPN region layer, a position regression layer, an RoI region selection layer, a SENet attention layer, a multi-scale feature fusion layer, and an output layer; Among them, the deformable convolution layer is used to insert one or more layers in the feature extraction layer to capture the complex shape features of blood vessels; The RPN region layer is used to generate candidate regions containing renal blood vessels and their bounding boxes; The position regression layer is used to perform position regression on the candidate regions to adjust their bounding boxes; The RoI region selection layer is used to extract the corresponding RoI regions according to the bounding boxes adjusted by the position regression layer.

[0012] In an alternative manner, the feature extraction layer includes a ResNet50 residual network layer and an FPN feature network layer; The deformable convolution layer inserts at least one deformable convolution layer in the feature extraction layer. The deformable convolution layer includes an offset prediction layer and a convolution operation layer with offsets. Among them, the offset prediction layer is used to generate an offset map; the convolution operation layer with offsets is used to generate a sampling network according to the offset map; The RoI region selection layer includes a mask layer, a coordinate extraction layer, and a classification layer; among them, the mask layer is used to generate a mask to filter out pixels containing blood vessels; the coordinate extraction layer is used to extract the bounding box coordinates of blood vessels; the classification layer is used to determine whether each RoI region contains blood vessels.

[0013] In an alternative manner, the method further includes: Generate a puncture path according to the upper contour, blood vessels, upper boundary, and lower boundary of the kidney in the ultrasound image sequence to avoid key blood vessel regions.

[0014] In an alternative manner, the generating of the puncture path further includes: Predict the motion trajectory of the puncture needle under different puncture paths; Evaluate the safety of the puncture path through the relationship between the motion trajectory and the blood vessel region; Obtain the maximum straightness of the puncture path according to minimizing the contact probability between the puncture needle and the key blood vessel region; The maximum puncture path is superimposed on the ultrasound image sequence in a visual manner for the doctor to use during the renal puncture process.

[0015] According to another aspect of the present invention, an AI-assisted data processing system for renal puncture based on artificial intelligence is provided, including: A preprocessing module for acquiring ultrasound images containing the kidney and its internal blood vessels, and performing noise suppression and contrast enhancement processing on the ultrasound images to improve the clarity of the puncture needle and blood vessel regions; A blood vessel region segmentation module for segmenting the internal blood vessels of the kidney in the ultrasound image and placing the segmented blood vessel regions in predefined rectangular frames; A jitter elimination module for correcting the blurred motion caused by probe jitter in the predefined rectangular frames; A prediction module for inputting the ultrasound images in the predefined rectangular frames into a trained kidney blood vessel segmentation model to obtain an ultrasound image sequence containing the upper and lower boundaries of the kidney; wherein, the upper boundary is located above the blood vessel and below the upper contour of the kidney; the lower boundary separates the lower contour of the kidney from the blood vessel; An output module for outputting the ultrasound image sequence to a renal puncture assistance system for the doctor to use during the renal puncture process.

[0016] According to yet another aspect of the present invention, a computing device is provided, including: a processor, a memory, a communication interface, and a communication bus, and the processor, the memory, and the communication interface complete communication with each other through the communication bus.

[0017] According to the solution provided by the present invention, an ultrasonic image including the kidney and its internal blood vessels is obtained, and noise suppression and contrast enhancement processing are performed on the ultrasonic image to improve the clarity of the puncture needle and blood vessel regions; the internal blood vessels of the kidney in the ultrasonic image are segmented into blood vessel regions, and the segmented blood vessel regions are placed within a predefined rectangular frame; the blurred motion caused by probe jitter within the predefined rectangular frame is corrected; the ultrasonic image within the predefined rectangular frame is input into a trained kidney blood vessel segmentation model to obtain an ultrasonic image sequence including the upper and lower boundaries of the kidney; wherein, the upper boundary is located above the blood vessel and below the upper contour of the kidney; the lower boundary separates the lower contour of the kidney from the blood vessel; the ultrasonic image sequence is output to a renal puncture assistance system for use by a doctor during renal puncture. The present invention segments the internal blood vessel regions of the kidney and places the segmentation results within a predefined rectangular frame, which not only reduces the cumbersome manual operation but also improves the segmentation accuracy. By correcting the blurred motion within the predefined rectangular frame, the problem of image blurring caused by probe jitter is solved. By identifying and marking the upper and lower boundaries of the kidney through the kidney blood vessel segmentation model, it not only helps the doctor quickly locate the puncture point but also guides the puncture depth and direction, improving the accuracy and safety of the puncture.

[0018] The above description is only an overview of the technical solution of the present invention. In order to be able to understand the technical means of the present invention more clearly, it can be implemented according to the content of the description. And in order to make the above and other objects, features, and advantages of the present invention more obvious and understandable, the following specifically enumerates the specific implementation manners of the present invention. Brief Description of the Drawings

[0019] By reading the detailed description of the preferred embodiments below, various other advantages and benefits will become clear to those of ordinary skill in the art. The drawings are only for the purpose of showing the preferred embodiments and are not considered to be a limitation of the present invention. Moreover, throughout the drawings, the same reference numerals are used to represent the same components. In the drawings: Figure 1 Shows a schematic flowchart of an artificial intelligence-based renal puncture AI-assisted data processing method according to an embodiment of the present invention; Figure 2 Shows a schematic diagram of a kidney blood vessel segmentation model according to an embodiment of the present invention; Figure 3 Shows a schematic diagram of the segmentation of the kidney blood vessels according to an embodiment of the present invention; Figure 4 Shows a schematic diagram of the ideal degree of needle insertion according to an embodiment of the present invention; Figure 5 Shows a schematic framework diagram of an artificial intelligence-based renal puncture AI-assisted data processing system according to an embodiment of the present invention; Figure 6The structural schematic diagram of the computing device according to an embodiment of the present invention is shown. Detailed implementation manners

[0020] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present invention can be more thoroughly understood and the scope of the present invention can be completely conveyed to those skilled in the art.

[0021] Figure 1 The flowchart of the AI-assisted data processing method for kidney puncture based on artificial intelligence according to an embodiment of the present invention is shown. Specifically, as Figure 1 shown, it includes the following steps: Step S101, obtain an ultrasonic image including a kidney and its internal blood vessels, and perform noise suppression and contrast enhancement processing on the ultrasonic image to improve the clarity of the puncture needle and blood vessel regions.

[0022] In this embodiment, through noise suppression and contrast enhancement, the speckle noise and blurred edges in the image can be significantly reduced, especially the clarity of the puncture needle and blood vessel regions. The contrast enhancement processing can increase the gray-scale difference between different tissues in the image, making the blood vessels and puncture needle more prominent in the image. Furthermore, it helps the doctor accurately identify the lesion area and provide real-time image guidance during the puncture operation.

[0023] Specifically, for the noise suppression processing, the binary wavelet transform is used to perform multi-scale decomposition on the ultrasonic image. By setting local thresholds and combining the soft threshold filtering method and the hard threshold filtering method, the wavelet coefficients at different scales are shrunk to suppress the speckle noise. Optionally, in combination with a morphological algorithm (such as the hit-or-miss transform), the detailed information in the image is further extracted and retained, while the high-frequency noise below the threshold is removed.

[0024] For the contrast enhancement processing, the extreme value stretching method of wavelet coefficients is used to enhance the contrast of the image. For example, by adjusting the dynamic range of the wavelet coefficients, the gray-scale distribution of the image is made more uniform, thereby improving the visual effect. Optionally, the image is divided into high-frequency and low-frequency components by the adaptive neighborhood histogram equalization method. The low-frequency component is processed by the adaptive neighborhood histogram equalization algorithm to improve the overall contrast of the image; at the same time, the high-frequency component is weighted to retain the detailed information of the image.

[0025] In an optional manner, the further noise suppression of the ultrasonic image includes: Remove the speckle and granular noise in the ultrasonic image through the bilateral filtering algorithm, and output the weighted average pixel value by the neighborhood pixel values; the weighted spatial value affinity factor is calculated according to the spatial proximity and pixel value difference between pixels; In this embodiment, the bilateral filtering algorithm is used to suppress the noise of the ultrasonic image. Compared with traditional filtering methods (such as Gaussian filtering, median filtering, etc.), when calculating the new value of each pixel, not only the spatial proximity (i.e., distance) between pixels is considered, but also the pixel value difference (i.e., brightness similarity) is considered. This makes the pixels on both sides of the edge not easily mixed during the filtering process, thus maintaining the clear edge of the image. Moreover, for the common speckle and granular noise in the ultrasonic image, the bilateral filtering can effectively remove these noises while avoiding excessive blurring of image details.

[0026] Specifically, select a square filtering window, and this window slides around each pixel to be processed. For each pixel within the window, calculate its spatial proximity weight to the central pixel (the weight can be obtained by calculating the Euclidean distance between the two and mapping it to a weight range), the closer the distance, the greater the weight.

[0027] Similarly, for each pixel within the window, calculate its pixel value difference weight to the central pixel (the weight can be obtained by calculating the difference between the two in the brightness space and mapping it to a weight range), the smaller the difference, the greater the weight.

[0028] Multiply the spatial proximity weight and the pixel value difference weight to obtain the comprehensive weight of each neighborhood pixel. Then, perform a weighted average on the values of all pixels within the window through the comprehensive weight to obtain the new value of the central pixel. Traverse each pixel in the image until the entire image is processed.

[0029] In this embodiment, the calculation formula of the spatial value affinity factor is: ; Wherein, and are pixel coordinates; and are and corresponding pixel values; is the standard deviation of spatial proximity; is the standard deviation of pixel value difference.

[0030] Through the above formula, the relative importance of spatial proximity and color difference in the calculation of the spatial value affinity factor can be flexibly controlled, and it can be applied to image processing tasks in different scenarios. Since the exponential function has strong robustness to noise and outliers, even if there are a small amount of noise or abnormal pixel values, the influence can be weakened by adjusting the standard deviation parameter, so as to maintain the stability of the overall calculation of the spatial value affinity factor.

[0031] Step S102, segment the blood vessels inside the kidney in the ultrasound image, and place the segmented blood vessel area in a predefined rectangular box.

[0032] By accurately segmenting the blood vessel area, doctors can more easily identify the blood vessel morphology, size and position, thereby improving the diagnostic accuracy of kidney diseases and reducing the surgical risk.

[0033] In this embodiment, algorithms such as threshold-based, region-growing or active contour model are used to segment the blood vessels inside the kidney to extract the blood vessel boundaries. Optionally, morphological operations (erosion, dilation, opening operation, closing operation) are performed on the segmentation result to fill the holes.

[0034] According to the size and position of the segmented blood vessel area, one or more predefined rectangular boxes are placed on the image, and the size and position of the rectangular boxes can be adjusted according to actual needs to ensure that the blood vessel area is completely contained.

[0035] In an optional manner, the segmenting the blood vessels inside the kidney in the ultrasound image further includes: Select one or more pixel points located in the blood vessels as initial seed points, and check the neighboring pixels of the initial seed points; If the similarity between the neighboring pixels and the initial seed points meets the preset threshold condition, add the neighboring pixels to the current blood vessel area and use them as new seed points to continue expanding; repeat until no new neighboring pixels are added to the blood vessel area; Fill the internal holes of the blood vessels according to the closing operation morphological algorithm, and smooth the boundaries of the blood vessels according to the opening operation morphological algorithm.

[0036] In this embodiment, by selecting initial seed points and gradually expanding, the blood vessel area inside the kidney can be more accurately identified and segmented, especially in the case of blurred blood vessel boundaries. The automatic execution of the algorithm for checking neighboring pixels, similarity comparison and region expansion reduces manual intervention. It is not only applicable to the blood vessel segmentation of kidney ultrasound images, but also applicable to medical imaging fields such as CT and MRI.

[0037] Specifically, one or more pixel points on the internal blood vessels of the kidney are automatically selected as the initial seed points. For example, possible blood vessel regions can be identified based on local features of the image such as brightness and texture, and pixel points are automatically selected.

[0038] Check the neighborhood pixels (such as 8-neighborhood) of each initial seed point, and calculate the similarity between the neighborhood pixels and the seed point in terms of features such as brightness or gradient. If the similarity is greater than the preset threshold, add the neighborhood pixel to the current blood vessel region and use it as a new seed point to continue checking its neighborhood. Repeat the above process until no new neighborhood pixels meet the conditions to be added to the blood vessel region.

[0039] Then, use closing operation to fill the possible holes inside the blood vessels. Among them, the closing operation is a process of dilation followed by erosion, which can fill small holes without significantly increasing the boundary size.

[0040] Then, use opening operation to smooth the blood vessel boundary and remove small protrusions or irregular boundaries caused by noise or other factors. The opening operation is a process of erosion followed by dilation, which can remove small objects smaller than the structuring element.

[0041] Step S103, correct the blurred motion caused by probe jitter within the predefined rectangular frame.

[0042] By eliminating the blur caused by probe jitter, the quality of the ultrasonic image can be significantly improved, ensuring that the puncture needle accurately reaches the predetermined target point.

[0043] In an alternative manner, the correction of the blurred motion caused by probe jitter within the predefined rectangular frame further includes: Detect stable feature points in consecutive frames through the ORB feature point detection algorithm; Track the feature point motion of the stable feature points between consecutive frames through the Lucas-Kanade optical flow method; Estimate the global motion parameters of the predefined rectangular frame based on the feature point motion; According to the estimated global motion parameters, apply an inverse transformation to the region within the predefined rectangular frame to cancel the blurred motion caused by probe jitter.

[0044] In this embodiment, the ORB (Oriented FAST and Rotated BRIEF) feature point detection algorithm combines the FAST key point detector and the BRIEF descriptor, and can quickly find the key points in the image. Among them, FAST and BRIEF are the feature detection algorithm and the vector creation algorithm respectively. ORB first looks for special regions in the image, called key points. Key points are small prominent regions in the image, such as those with the characteristic of a sharp change in pixel values from light to dark. Then ORB calculates the corresponding feature vector for each key point. The characteristics of ORB are very fast speed and being to a certain extent unaffected by noise and image transformations, such as rotation and scaling transformations, etc.

[0045] After detecting the feature points (key points), the Lucas-Kanade optical flow method is used to track the movement of the feature points between consecutive frames. The Lucas-Kanade optical flow method can calculate the displacement vector of the feature points between consecutive images, so as to track the movement vector of the feature points.

[0046] Traditional optical flow equations usually assume brightness constancy, that is, it is assumed that the brightness of pixel points does not change between consecutive frames. The optical flow equation in this embodiment allows brightness changes because it is based on the comparison of pixel values within a window rather than the assumption of brightness constancy of a single pixel point. Even in the case of large illumination changes, a relatively accurate optical flow estimate can be obtained. The expression of the optical flow equation is: ; Among them, is a preset size window; is the time interval between two frames of images; are the components of the movement vector of the pixel in the x and y directions; is a pixel position within the preset size window; is the image intensity function; is at the position and time the image intensity at. By using the weighted squared difference minimization strategy within the window, not only the limitation of the traditional optical flow equation assuming brightness constancy is overcome, but also the accuracy of the optical flow estimate is improved, especially in dealing with local noise and brightness changes, and it performs more excellently.

[0047] Based on the movement vector of the feature points, the global movement parameters of the entire predefined rectangular box can be estimated. According to the estimated global movement parameters, an inverse transformation is applied to the region within the predefined rectangular box, that is, a movement opposite to the probe jitter is performed to cancel the blurred movement caused by the probe jitter.

[0048] Step S104: Input the ultrasonic image within the predefined rectangular box into the trained kidney blood vessel segmentation model to obtain an ultrasonic image sequence including the upper and lower boundaries of the kidney; wherein, the upper boundary is located above the blood vessel and below the upper contour of the kidney; the lower boundary separates the lower contour of the kidney from the blood vessel.

[0049] In this embodiment, the kidney blood vessel segmentation model is as Figure 2 shown, and includes an input layer, a feature extraction layer, a deformable convolution layer, an RPN region layer, a position regression layer, an RoI region selection layer, a SENet attention layer, a multi-scale feature fusion layer, and an output layer.

[0050] Among them, the deformable convolution layer is used to insert one or more layers in the feature extraction layer to capture the complex shape features of the blood vessels. When traditional convolutional neural networks process images, due to their fixed convolution kernel size, it is difficult to effectively capture the complex and variable shape features in the images, especially for slender and curved structures like kidney blood vessels. The deformable convolution layer can more flexibly adapt to the changes in the blood vessel shape by allowing the convolution kernel to perform dynamic sampling point offsets on the input feature map.

[0051] The RPN region layer is used to generate candidate regions containing kidney blood vessels and their bounding boxes. Traditional detection methods for extracting candidate regions are very time-consuming. For example, OpenCV adaboost uses a sliding window + image pyramid, or R-CNN uses SS (Selective Search). While RPN (Region Proposal Network) is implemented with a series of CNN fully convolutional networks (the specific structure is not defined herein), which can share the convolutional features of the entire image with the detection network, thereby generating region recommendations with almost no cost and being able to quickly generate candidate regions containing kidney blood vessels and their bounding boxes.

[0052] The position regression layer is used to perform position regression on the candidate regions to adjust their bounding boxes to be closer to the actual kidney blood vessel regions.

[0053] The RoI region selection layer is used to extract the corresponding RoI regions according to the bounding boxes adjusted by the position regression layer. After the RoI (Region of Interest) passes through the convolutional network to obtain feature maps, multiple target candidate boxes are obtained using the selective search or RPN algorithm.

[0054] In an optional manner, the feature extraction layer includes a ResNet50 residual network layer and an FPN feature network layer; The deformable convolution layer inserts at least one deformable convolution layer into the feature extraction layer. The deformable convolution layer includes an offset prediction layer and a convolution operation layer with offsets. Among them, the offset prediction layer is used to generate an offset map; the convolution operation layer with offsets is used to generate a sampling network according to the offset map. The RoI region selection layer includes a mask layer, a coordinate extraction layer, and a classification layer. Among them, the mask layer is used to generate a mask to filter out pixels containing blood vessels; the coordinate extraction layer is used to extract the bounding box coordinates of blood vessels; the classification layer is used to determine whether each RoI region contains blood vessels.

[0055] In this embodiment, by combining the ResNet50 residual network layer and the FPN (Feature Pyramid Network) feature network layer, rich feature information can be extracted at different scales, and blood vessels of different sizes in the blood vessel image can be effectively detected. The deformable convolution layer (Deformable Convolution) enables the network to learn adaptive offsets during the sampling process, so as to be able to more accurately locate the blood vessel boundary, especially for blood vessels with complex shapes and high curvatures.

[0056] As Figure 2 shown, the RoI (Region of Interest) region selection layer filters out the blood vessel region through the mask layer, further accurately extracts the blood vessel bounding box through the coordinate extraction layer, and finally the classification layer judges whether the RoI contains blood vessels, so as to realize the coarse-to-fine blood vessel detection. Among them, the mask layer uses a preset threshold or obtains a mask generator through learning to generate a mask of the blood vessel region from the feature map. The coordinate extraction layer is based on the mask region and uses an edge detection algorithm such as the Canny algorithm to extract the bounding box coordinates of blood vessels. After feature extraction is performed on each RoI region by the classification layer, it is judged whether it contains blood vessels through a classifier (such as a fully connected layer, etc.).

[0057] Finally, a segmentation schematic diagram of the renal blood vessels as Figure 3 shown is obtained. Compared with the schematic diagram of the ideal degree of needle insertion as Figure 4 shown, the blood vessel regions are all placed within the specified rectangular frame, and the image is clear. The upper boundary line is above the blood vessel and below the upper contour of the kidney. The lower boundary line separates the lower contour of the kidney from the blood vessel (it should be noted that it is not required that the blood vessel is completely located above). In addition, the brightness information (i.e., clarity) of the needle is improved, and the influence caused by human factors such as probe jitter is reduced.

[0058] In an optional manner, the method further includes: Generating a puncture path according to the upper contour of the kidney, blood vessels, upper boundary line, and lower boundary line in the ultrasound image sequence to avoid key blood vessel regions.

[0059] In an alternative approach, the generation of the puncture path further includes: Predicting the movement trajectory of the puncture needle under different puncture paths; Evaluating the safety of the puncture path based on the relationship between the movement trajectory and the blood vessel region; Maximizing the straightness of the puncture path by minimizing the contact probability between the puncture needle and the critical blood vessel region; Overlaying the maximized puncture path on the ultrasound image sequence in a visual manner for use by the doctor during the renal puncture procedure.

[0060] In this embodiment, the trajectory equation of the puncture needle is: ; wherein, is the puncture starting point coordinate, is the unit vector of the puncture direction, is the time parameter.

[0061] Safety assessment requires calculating the overlapping degree between the puncture trajectory and the blood vessel region. In this embodiment, it is approximately solved by calculating the minimum distance between the trajectory points and the blood vessel boundary, and the safety assessment is expressed as: ; wherein, is the time when the puncture ends; is the point on the blood vessel boundary, defined by the parameter .

[0062] To minimize the contact probability between the puncture needle and the critical blood vessel region while maximizing the straightness of the puncture path, it is achieved by solving the following objective function. Among them, the straightness is measured by the ratio of the length of the puncture path to the length of its straight-line segment projection, and the contact probability is estimated by the volume or area of the overlap between the path and the blood vessel region. In this embodiment, a weighted sum is used as the objective function, and the expression of this objective function is: ; wherein, , is the weight coefficient, used to balance the straightness and safety; Path Length refers to the total length of the path from the starting point to the ending point; Straight Line Projection refers to the length of the straight-line segment from the starting point to the ending point, and using it as the denominator means that the first part in the whole expression attempts to minimize the deviation of the path from the straight-line distance, that is, to find a path as close to a straight line as possible; is the indicator function, when It takes the value of 1 when located within the blood vessel region and 0 otherwise; the integral term calculates the total contact time (or distance) between the puncture needle and the blood vessel region to measure the contact probability.

[0063] Finally, the calculated maximum puncture path (the path that meets the safety and straightness requirements) is superimposed on the ultrasound image sequence in a visual manner.

[0064] Step S105: Output the ultrasound image sequence to the renal puncture assistance system for the doctor to use during renal puncture.

[0065] According to the solution provided by the present invention, an ultrasound image including the kidney and its internal blood vessels is obtained, and noise suppression and contrast enhancement processing are performed on the ultrasound image to improve the clarity of the puncture needle and the blood vessel region; the internal blood vessels of the kidney in the ultrasound image are segmented into blood vessel regions, and the segmented blood vessel regions are placed in a predefined rectangular frame; the blurred motion caused by probe jitter in the predefined rectangular frame is corrected; the ultrasound image within the predefined rectangular frame is input into a trained kidney blood vessel segmentation model to obtain an ultrasound image sequence including the upper and lower boundaries of the kidney; wherein, the upper boundary is located above the blood vessel and below the upper contour of the kidney; the lower boundary separates the lower contour of the kidney from the blood vessel; the ultrasound image sequence is output to the renal puncture assistance system for the doctor to use during renal puncture. The present invention segments the internal blood vessels of the kidney and places the segmentation result in a predefined rectangular frame, which not only reduces the complexity of manual operations but also improves the segmentation accuracy. By correcting the blurred motion within the predefined rectangular frame, the problem of image blurring caused by probe jitter is solved. By identifying and marking the upper and lower boundaries of the kidney through the kidney blood vessel segmentation model, it not only helps the doctor quickly locate the puncture point but also guides the puncture depth and direction, improving the accuracy and safety of the puncture.

[0066] Figure 5 The structural schematic diagram of the artificial intelligence-based renal puncture AI assistance data processing system according to an embodiment of the present invention is shown. The artificial intelligence-based renal puncture AI assistance data processing system includes: A preprocessing module 410, configured to obtain an ultrasound image including the kidney and its internal blood vessels, and perform noise suppression and contrast enhancement processing on the ultrasound image to improve the clarity of the puncture needle and the blood vessel region; A blood vessel region segmentation module 420, configured to segment the internal blood vessels of the kidney in the ultrasound image into blood vessel regions, and place the segmented blood vessel regions in a predefined rectangular frame; A jitter elimination module 430, configured to correct the blurred motion caused by probe jitter in the predefined rectangular frame; A prediction module 440 is configured to input the ultrasound image within the predefined rectangular box into a trained kidney blood vessel segmentation model to obtain an ultrasound image sequence including the upper and lower boundaries of the kidney; wherein, the upper boundary is located above the blood vessel and below the upper contour of the kidney; the lower boundary separates the lower contour of the kidney from the blood vessel. An output module 450 is configured to output the ultrasound image sequence to a renal puncture assistance system for a doctor to use during renal puncture.

[0067] Figure 5 FIG. shows a schematic structural diagram of an embodiment of a computing device of the present invention. The specific embodiments of the present invention do not limit the specific implementation of the computing device.

[0068] As Figure 5 shown, the computing device may include: a processor 502, a communications interface 504, a memory 506, and a communication bus 508.

[0069] Wherein: the processor 502, the communications interface 504, and the memory 506 communicate with each other through the communication bus 508. The communications interface 504 is configured to communicate with network elements of other devices such as a client or other servers. The processor 502 is configured to execute a program 510, and specifically may execute relevant steps in the above-mentioned embodiment of the renal puncture AI assistance data processing method based on artificial intelligence.

[0070] Specifically, the program 510 may include program code, and the program code includes computer operation instructions.

[0071] The processor 502 may be a central processing unit CPU, or a specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present invention. One or more processors included in the computing device may be of the same type of processor, such as one or more CPUs; or may be of different types of processors, such as one or more CPUs and one or more ASICs.

[0072] The memory 506 is configured to store the program 510. The memory 506 may include a high-speed RAM memory, and may also include non-volatile memory, such as at least one disk memory.

[0073] According to the solution provided by the present invention, an ultrasonic image including a kidney and its internal blood vessels is obtained, and noise suppression and contrast enhancement processing are performed on the ultrasonic image to improve the clarity of the puncture needle and blood vessel regions; the internal blood vessels of the kidney in the ultrasonic image are segmented into blood vessel regions, and the segmented blood vessel regions are placed within a predefined rectangular frame; the blurred motion caused by probe jitter within the predefined rectangular frame is corrected; the ultrasonic image within the predefined rectangular frame is input into a trained kidney blood vessel segmentation model to obtain an ultrasonic image sequence including the upper and lower boundaries of the kidney; wherein, the upper boundary is located above the blood vessel and below the upper contour of the kidney; the lower boundary separates the lower contour of the kidney from the blood vessel; the ultrasonic image sequence is output to a renal puncture assistance system for use by a doctor during renal puncture. The present invention segments the internal blood vessel regions of the kidney and places the segmentation results within a predefined rectangular frame, which not only reduces the cumbersome manual operation but also improves the segmentation accuracy. By correcting the blurred motion within the predefined rectangular frame, the problem of image blurring caused by probe jitter is solved. By identifying and marking the upper and lower boundaries of the kidney through the kidney blood vessel segmentation model, it not only helps the doctor quickly locate the puncture point but also guides the puncture depth and direction, improving the accuracy and safety of the puncture.

[0074] Those skilled in the art can understand that the modules in the devices in the embodiments can be adaptively changed and arranged in one or more devices different from this embodiment. The modules or units or components in the embodiments can be combined into one module or unit or component, and in addition, they can be divided into multiple sub-modules or sub-units or sub-components. Except that at least some of such features and / or processes or units are mutually exclusive, any combination can be used for all the features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all the processes or units of any method or device so disclosed. Unless otherwise clearly stated, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) can be replaced by an alternative feature providing the same, equivalent, or similar purpose. In addition, those skilled in the art can understand that although some of the embodiments herein include certain features included in other embodiments rather than other features, the combination of the features of different embodiments means that it is within the scope of the present invention and forms different embodiments. For example, in the following claims, any one of the claimed embodiments can be used in any combination. The present invention can be implemented by means of hardware including several different elements and by means of a properly programmed computer. In the unit claims listing several devices, several of these devices can be embodied by the same hardware item. The steps in the above embodiments, unless otherwise specified, should not be construed as limiting the execution order.

Claims

1. An AI-assisted data processing method for kidney puncture based on artificial intelligence, characterized in that, Including: Obtain an ultrasonic image including the kidney and its internal blood vessels, and perform noise suppression and contrast enhancement processing on the ultrasonic image to improve the clarity for the puncture needle and blood vessel areas; Segment the blood vessel areas of the internal blood vessels in the ultrasonic image, and place the segmented blood vessel areas within a predefined rectangular frame; Correct the blurred motion caused by probe jitter within the predefined rectangular frame; Input the ultrasonic image within the predefined rectangular frame into a trained kidney blood vessel segmentation model to obtain a sequence of ultrasonic images including the upper and lower boundaries of the kidney; wherein, the upper boundary is located above the blood vessel and below the upper contour of the kidney; the lower boundary separates the lower contour of the kidney from the blood vessel; Output the sequence of ultrasonic images to a renal puncture assistance system for use by a doctor during renal puncture.

2. The AI-assisted data processing method for kidney puncture based on artificial intelligence according to claim 1, wherein The noise suppression of the ultrasonic image further includes: Removing speckle and granular noise in the ultrasonic image through a bilateral filtering algorithm and outputting a weighted average pixel value based on neighborhood pixel values; the weighted spatial value affinity factor is calculated based on the spatial proximity and pixel value difference between pixels; Wherein, the calculation formula of the spatial value affinity factor is: ; Wherein, and are pixel coordinates; and are and corresponding pixel values; is the standard deviation of spatial proximity; is the standard deviation of pixel value difference.

3. The AI-assisted data processing method for renal puncture based on artificial intelligence according to claim 1 or 2, characterized in that The segmentation of the blood vessel areas of the internal blood vessels in the ultrasonic image further includes: Select one or more pixel points located on the blood vessel as initial seed points and check the neighborhood pixels of the initial seed points; If the similarity between the neighborhood pixels and the initial seed points meets the preset threshold condition, add the neighborhood pixels to the current blood vessel area and use them as new seed points to continue expanding; repeat until no new neighborhood pixels are added to the blood vessel area; Fill the internal holes of the blood vessel according to the closing operation morphological algorithm, and smooth the boundary of the blood vessel according to the opening operation morphological algorithm.

4. The AI-assisted data processing method for kidney puncture based on artificial intelligence according to claim 1, wherein, The correction of the blurred motion caused by probe jitter within the predefined rectangular frame further includes: Detect stable feature points in consecutive frames through the ORB feature point detection algorithm; Track the feature point motion of the stable feature points between consecutive frames through the Lucas-Kanade optical flow method; Estimate the global motion parameters of the predefined rectangular frame based on the feature point motion; Apply an inverse transformation to the area within the predefined rectangular frame according to the estimated global motion parameters to cancel the blurred motion caused by probe jitter.

5. The AI-assisted data processing method for renal puncture based on artificial intelligence according to claim 1, wherein: The kidney blood vessel segmentation model includes an input layer, a feature extraction layer, a deformable convolution layer, an RPN region layer, a position regression layer, a RoI region selection layer, a SENet attention layer, a multi-scale feature fusion layer, and an output layer; Wherein, the deformable convolution layer is used to insert one or more layers in the feature extraction layer to capture the complex shape features of blood vessels; The RPN region layer is used to generate candidate regions including kidney blood vessels and their bounding boxes; The position regression layer is used to perform position regression on the candidate regions to adjust their bounding boxes; The RoI region selection layer is used to extract the corresponding RoI region according to the bounding box adjusted by the position regression layer.

6. The AI-assisted data processing method for kidney puncture based on artificial intelligence according to claim 5, characterized in that: The feature extraction layer includes a ResNet50 residual network layer and an FPN feature network layer; At least one deformable convolutional layer is inserted into the feature extraction layer. The deformable convolutional layer includes an offset prediction layer and a convolutional operation layer with offsets. Among them, the offset prediction layer is used to generate an offset map; the convolutional operation layer with offsets is used to generate a sampling network according to the offset map; The RoI region selection layer includes a mask layer, a coordinate extraction layer, and a classification layer; wherein, the mask layer is used to generate a mask to filter out pixels containing blood vessels; the coordinate extraction layer is used to extract the bounding box coordinates of blood vessels; the classification layer is used to judge whether each RoI region contains blood vessels.

7. The AI-assisted data processing method for kidney puncture based on artificial intelligence according to claim 1, wherein, The method further includes: Generating a puncture path according to the upper contour, blood vessels, upper boundary line, and lower boundary line of the kidney in the ultrasonic image sequence to avoid key blood vessel regions.

8. The AI-assisted data processing method for kidney puncture based on artificial intelligence according to claim 7, wherein The generating of the puncture path further includes: Predicting the movement trajectory of the puncture needle under different puncture paths; Evaluating the safety of the puncture path through the relationship between the movement trajectory and the blood vessel region; Obtaining the maximum straightness of the puncture path according to minimizing the contact probability between the puncture needle and the key blood vessel region; Overlaying the puncture path with the maximum straightness on the ultrasonic image sequence in a visual way for the doctor to use during kidney puncture.

9. An AI-assisted data processing system for kidney puncture based on artificial intelligence, characterized in that, Including: A preprocessing module for obtaining an ultrasonic image containing the kidney and its internal blood vessels, and performing noise suppression and contrast enhancement processing on the ultrasonic image to improve the clarity of the puncture needle and blood vessel regions; A blood vessel region segmentation module for segmenting the internal blood vessels of the kidney in the ultrasonic image and placing the segmented blood vessel regions in a predefined rectangular box; A jitter elimination module for correcting the blurred movement caused by probe jitter in the predefined rectangular box; A prediction module for inputting the ultrasonic image in the predefined rectangular box into a trained kidney blood vessel segmentation model to obtain an ultrasonic image sequence containing the upper and lower boundary lines of the kidney; wherein, the upper boundary line is located above the blood vessels and below the upper contour of the kidney; the lower boundary line separates the lower contour of the kidney from the blood vessels; An output module for outputting the ultrasonic image sequence to a kidney puncture assistance system for the doctor to use during kidney puncture.

10. A computing device, comprising: A processor, a memory, a communication interface, and a communication bus. The processor, the memory, and the communication interface complete communication with each other through the communication bus; The memory is used to store at least one executable instruction, and the executable instruction causes the processor to execute the operations corresponding to the above-mentioned AI-assisted data processing method for kidney puncture based on artificial intelligence.

Images