Real-time compensation method and system for kidney stone coordinates based on respiratory motion of ultrasound images

Through the improved GFTT method and Lucas-Kanade optical flow method combined with Kalman filtering technology, precise tracking and coordinate compensation of kidney stones in respiratory movements are achieved, which solves the problem of respiratory movement affecting kidney stone positioning and improves external lithotripsy Accuracy and safety of the surgery.

CN118762008BActive Publication Date: 2024-11-12TONGJI HOSPITAL ATTACHED TO TONGJI MEDICAL COLLEGE HUAZHONG SCI TECH +1
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202411242172.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-05
Publication Date
2024-11-12
Estimated Expiration
2044-09-05

AI Technical Summary

Technical Problem

Respiratory movement leads to unstable position of kidney stones in ultrasound images, affecting image clarity and accurate positioning and precise control of lithotripsy surgery. The existing technology cannot achieve automated identification and tracking, which increases the work burden and surgical risks of doctors.

Method used

The improved GFTT method based on ultrasound images and Lucas-Kanade optical flow method combined with Kalman filtering technology are used to detect and track the characteristic points of kidney stones in real time, calculate their displacement, and realize accurate tracking and coordinate compensation of kidney stones in respiratory movement.

Benefits of technology

It effectively reduces the impact of respiratory exercise on stone positioning, improves the accuracy and safety of in vitro lithotripsy surgery, realizes fully automated kidney stone detection and tracking, and reduces the requirements for doctors' professionalism.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118762008B_ABST
    Figure CN118762008B_ABST
Patent Text Reader

Abstract

The present invention relates to a method and system for real-time compensation of coordinates of kidney stones based on respiratory motion of ultrasonic images. The method comprises: extracting video frame images from real-time ultrasonic video of a patient's kidney region, and determining the corresponding relationship between each video frame image and the patient's respiratory motion; performing pixel value standardization processing and grayscale conversion on the video frame images; constructing an autocorrelation matrix whose coefficients are associated with characteristic thresholds of kidney stones, obtaining an improved GFTT method based on the autocorrelation matrix, and detecting kidney stone feature points in the video frame images based on the improved GFTT method and volume constraints; tracking kidney stone feature points based on a Lucas-Kanade optical flow method to obtain kidney stone trajectories; calculating the displacement of a kidney stone region between adjacent frames, and outputting the current position of the kidney stone; and realizing accurate tracking and real-time coordinate compensation of kidney stones in respiratory motion, effectively reducing the influence of respiratory motion on stone positioning during surgery, and effectively improving the accuracy and safety of extracorporeal lithotripsy.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image technology processing, and in particular to a method and system for real-time compensation of kidney stone coordinates based on ultrasonic image respiratory motion. Background Art

[0002] In extracorporeal lithotripsy, respiratory motion has a significant impact on ultrasound image analysis. Due to the patient's respiratory motion, the acquired ultrasound sequence images have motion offsets in the target area between frames. These motion offsets cause artifacts in the ultrasound image and cause the position of the kidney stones to change continuously, seriously affecting the stability and clarity of the image.

[0003] At present, in traditional surgical scenarios, doctors need to observe and control the entire process, and the movement of stones needs to be manually counted by professional doctors, which cannot achieve automatic identification and tracking. Image offset and artifacts caused by respiratory movement hinder the accurate positioning of kidney stones and precise control of lithotripsy. In addition, two-dimensional ultrasound images have low resolution and narrow field of view, which largely rely on the doctor's experience and require a high level of professionalism. This not only increases the workload of doctors, but also affects the clinical application and efficacy of extracorporeal lithotripsy. Summary of the invention

[0004] In response to the technical problems existing in the prior art, the present invention provides a method and system for real-time compensation of kidney stone coordinates during respiratory movement based on ultrasonic images, which can achieve accurate tracking and real-time coordinate compensation of kidney stones during respiratory movement, effectively reduce the impact of intraoperative respiratory movement on stone positioning, and effectively improve the accuracy and safety of extracorporeal lithotripsy, providing reliable technical support for intelligent extracorporeal lithotripsy.

[0005] According to a first aspect of the present invention, a method for real-time compensation of kidney stone coordinates based on ultrasonic image respiratory motion is provided, comprising:

[0006] Step 1, extracting video frame images from the real-time ultrasound video of the patient's kidney area, and determining the corresponding relationship between each video frame image and the patient's respiratory movement;

[0007] Step 2, performing pixel value normalization processing and grayscale conversion on the video frame image;

[0008] Step 3, constructing an autocorrelation matrix whose coefficients are associated with characteristic thresholds of kidney stones, obtaining an improved GFTT method based on the autocorrelation matrix, and detecting and obtaining characteristic points of kidney stones in the video frame image based on the improved GFTT method and volume constraints;

[0009] Step 4, tracking the kidney stone feature points based on the Lucas-Kanade optical flow method to obtain the kidney stone trajectory;

[0010] Step 5, calculate the displacement of the kidney stone area between adjacent frames and output the current position of the kidney stone.

[0011] Based on the above technical solution, the present invention can also make the following improvements.

[0012] Optionally, the pixel value normalization processing performed on the video frame image in step 2 is Z-score normalization processing, comprising:

[0013] Each video frame image is mean-variance normalized so that the pixel value of the video frame image has the characteristics of mean 0 and standard deviation 1.

[0014] Optionally, the process of converting the video frame image into a grayscale image in step 2 includes:

[0015] When the video frame image is a color image, the video frame image is converted into a grayscale image, and the conversion formula is:

[0016] ;

[0017] in, , and Respectively represent the pixel values ​​of the red, green and blue channels in the video frame image, Represents the converted grayscale image.

[0018] Optionally, the process of constructing an autocorrelation matrix in which coefficients are associated with characteristic thresholds of kidney stones in step 3 includes:

[0019] Step 301: construct a weight function based on the strong echo characteristics of kidney stones :

[0020] ;

[0021] in, represents the weight at the pixel point (u,v), Represents the gray value at the pixel (u, v), Represents the weight of the kidney stone strong echo area, The threshold value of the gray value representing the strong echo group area;

[0022] Step 302, constructing the autocorrelation matrix M of the kidney stone echo cluster at each pixel point in the video frame image:

[0023] ;

[0024] in, and They represent the gradients of the pixel point (u, v) in the video frame image in the x direction and the y direction respectively, and W represents the video frame image.

[0025] Optionally, the process of detecting and obtaining kidney stone feature points in the video frame image in step 3 includes:

[0026] Step 303: Calculate pixel response value , the response value Defined as the minimum eigenvalue of the autocorrelation matrix M;

[0027] Step 304: Screen and obtain the response value Feature points whose grayscale values ​​are greater than the set threshold of the strong echo group area;

[0028] Step 305: Eliminate the feature points whose volumes are outside a set range to obtain the kidney stone feature points.

[0029] Optionally, step 4 includes: setting initial values ​​of adaptive parameters of the optical flow method, wherein the adaptive parameters include: optical flow window size, maximum number of pyramid layers, and number of optical flow calculation iterations.

[0030] Optionally, after step 4, the step further includes: using a Kalman filter model to smooth the kidney stone trajectory, the process of which includes:

[0031] Initialize the custom parameters of the Kalman filter, the custom parameters include: state vector and the state transition matrix ; The state vector Defined as:

[0032] ;

[0033] in, represents the position of kidney stones in the kth frame image, The speed of kidney stones;

[0034] The state transfer matrix F is defined as:

[0035] ;

[0036] in, is the time interval between adjacent video frame images;

[0037] Through the Kalman filter recursive model, the basic state transfer equation and observation equation are substituted to update the trajectory of kidney stones.

[0038] According to a second aspect of the present invention, there is provided a real-time compensation system for kidney stone coordinates based on ultrasonic image respiratory motion, comprising: a video acquisition and analysis module, an image preprocessing module, a feature point detection and tracking module, a trajectory analysis module and a displacement estimation output module;

[0039] The video acquisition and analysis module is used to extract video frame images from the real-time ultrasound video of the patient's kidney area, and determine the corresponding relationship between each video frame image and the patient's respiratory movement;

[0040] The image preprocessing module is used to perform pixel value standardization processing and grayscale conversion on the video frame image;

[0041] The feature point detection and tracking module is used to construct an autocorrelation matrix whose coefficients are associated with characteristic thresholds of kidney stones, obtain an improved GFTT method based on the autocorrelation matrix, and detect and obtain kidney stone feature points in the video frame image based on the improved GFTT method and volume constraints;

[0042] The trajectory analysis module is used to track the kidney stone feature points based on the Lucas-Kanade optical flow method to obtain the kidney stone trajectory;

[0043] The displacement estimation output module is used to calculate the displacement of the kidney stone area between adjacent frames and output the current position of the kidney stone.

[0044] According to a third aspect of the present invention, there is provided an electronic device comprising a memory and a processor, wherein the processor is used to implement the steps of a method for real-time compensation of kidney stone coordinates based on respiratory motion in ultrasonic images when executing a computer management program stored in the memory.

[0045] According to a fourth aspect of the present invention, there is provided a computer-readable storage medium having a computer management program stored thereon, which, when executed by a processor, implements the steps of a method for real-time compensation of kidney stone coordinates based on respiratory motion in ultrasound images.

[0046] The present invention provides a method, system, electronic device and storage medium for real-time compensation of kidney stone coordinates based on respiratory movement in ultrasonic images. By combining the improved GFTT algorithm, LK optical flow method and Kalman filter smoothing processing, it is possible to obtain accurate tracking and real-time coordinate compensation of kidney stones in respiratory movement, and realize fully automated respiratory movement detection during kidney stone surgery based on ultrasonic images. It can effectively reduce the impact of intraoperative respiratory movement on stone positioning, enhance the degree of automation of lithotripsy, effectively improve the accuracy and safety of extracorporeal lithotripsy, and provide reliable technical support for intelligent extracorporeal lithotripsy. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1A flow chart of a method for real-time compensation of kidney stone coordinates based on respiratory motion in ultrasonic images provided by the present invention;

[0048] Figure 2 A flowchart of an embodiment of a method for real-time compensation of kidney stone coordinates based on ultrasonic image respiratory motion provided by the present invention;

[0049] Figure 3 A structural block diagram of a system for real-time compensation of kidney stone coordinates based on ultrasonic image respiratory motion provided by the present invention;

[0050] Figure 4 A schematic diagram of the hardware structure of a possible electronic device provided by the present invention;

[0051] Figure 5 A schematic diagram of the hardware structure of a possible computer-readable storage medium provided by the present invention. DETAILED DESCRIPTION

[0052] The principles and features of the present invention are described below in conjunction with the accompanying drawings. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.

[0053] Figure 1 The present invention provides a flow chart of a method for real-time compensation of kidney stone coordinates based on ultrasonic image respiratory motion, such as Figure 1 As shown, the method includes:

[0054] Step 1: extract video frame images from the real-time ultrasound video of the patient's kidney area, and determine the corresponding relationship between each video frame image and the patient's respiratory movement.

[0055] Step 2: normalize the pixel values ​​of the video frame images and convert them into grayscale images.

[0056] Step 3, construct an autocorrelation matrix whose coefficients are associated with characteristic thresholds of kidney stones, obtain an improved GFTT method based on the autocorrelation matrix, and detect characteristic points of kidney stones in the video frame image based on the improved GFTT method and volume constraints.

[0057] Step 4: Track the kidney stone feature points based on the Lucas-Kanade optical flow method to obtain the kidney stone trajectory.

[0058] Step 5, calculate the displacement of the kidney stone area between adjacent frames and output the current position of the kidney stone.

[0059] The present invention provides a real-time compensation method for kidney stone coordinates based on ultrasonic image respiratory motion, which can realize accurate tracking and real-time coordinate compensation of kidney stones during respiratory motion, effectively reduce the influence of intraoperative respiratory motion on stone positioning, and effectively improve the accuracy and safety of extracorporeal lithotripsy, providing reliable technical support for intelligent extracorporeal lithotripsy.

[0060] Example 1

[0061] Embodiment 1 provided by the present invention is an embodiment of a method for real-time compensation of kidney stone coordinates based on ultrasonic image respiratory motion provided by the present invention. Figure 2 A flowchart of an embodiment of a method for real-time compensation of kidney stone coordinates based on ultrasonic image respiratory motion provided by the present invention, combined with Figure 1 and Figure 2 It can be seen that the embodiment of the method includes:

[0062] Step 1: extract video frame images from the real-time ultrasound video of the patient's kidney area, and determine the corresponding relationship between each video frame image and the patient's respiratory movement.

[0063] In the specific implementation, ultrasound images of the patient's kidney area are collected in real time through ultrasound equipment to record the patient's respiratory movement. The ultrasound video data includes multiple continuous frames, each of which is a two-dimensional grayscale image, representing an ultrasound image at a specific time point. Video parsing uses the opencv library to parse the AVI file and extract video data frame by frame, including opening the AVI file, reading the frame rate and resolution of the video frame, and storing it for subsequent respiratory interval statistics.

[0064] Step 2: normalize the pixel values ​​of the video frame images and convert them into grayscale images.

[0065] In a possible embodiment, the pixel value normalization processing of the video frame image in step 2 is Z-score normalization processing for subsequent image processing and feature point detection, including:

[0066] Each video frame image is mean-variance normalized so that the pixel value of the video frame image has the characteristics of mean 0 and standard deviation 1.

[0067] In specific implementation, the mean-variance standardization is calculated as:

[0068] .

[0069] in, represents the pixel value of the pixel position (i, j) in the original ultrasound image, is the mean value of ultrasound pixel data, is the standard deviation of ultrasound data. The mean and standard deviation are calculated as follows:

[0070] .

[0071] Where M and N represent the height and width of the ultrasound image, respectively. After normalization, each pixel value in the image is transformed to a distribution with mean 0 and standard deviation 1.

[0072] Since ultrasound images are usually grayscale images, in order to unify the processing flow, it is necessary to ensure that each frame of the image exists in the form of a grayscale image. In a possible embodiment, the process of converting the video frame image into a grayscale image in step 2 includes:

[0073] When the video frame image is a color image, the optimal color conversion parameters are used to convert the video frame image into a grayscale image. The conversion formula is:

[0074] .

[0075] in, , and Respectively represent the pixel values ​​of the red, green and blue channels in the video frame image, Represents the converted grayscale image. The converted grayscale image contains only one channel, which simplifies subsequent processing.

[0076] Step 3, construct an autocorrelation matrix whose coefficients are associated with characteristic thresholds of kidney stones, obtain an improved GFTT method based on the autocorrelation matrix, and detect characteristic points of kidney stones in the video frame image based on the improved GFTT method and volume constraints.

[0077] The GFTT (Good Features to Track) method is a feature point detection method for detecting significant corners in an image. In the present invention, significant corners refer to the strong echo clusters shown in the stone area in the ultrasound image. In order to adapt to the particularity of ultrasound images and the detection of kidney stones, the traditional GFTT algorithm is improved so that its autocorrelation matrix coefficient is associated with the characteristic threshold of kidney stones.

[0078] During the specific implementation process, the processed ultrasound grayscale image is first input, then the image gradient is calculated, the autocorrelation matrix for kidney stone assessment is constructed, the response value of each pixel is calculated, and combined with the volume constraint, the strong echo group area is screened out as the stone feature point according to the size of the response value.

[0079] In a possible embodiment, the process of constructing an autocorrelation matrix in step 3 in which coefficients are associated with characteristic thresholds of kidney stones includes:

[0080] Step 301: construct a weight function based on the strong echo characteristics of kidney stones :

[0081] ;

[0082] in, represents the weight at the pixel point (u,v), Represents the gray value at the pixel (u, v), Represents the weight of the kidney stone strong echo area, The threshold value of the gray value representing the strong echo group area; specifically, Can be set to 1.75. Can be set to 274.

[0083] Introducing weight coefficients in the definition of the autocorrelation matrix , the weight coefficient Threshold of strong echo characteristics of kidney stones The correlation is adjusted based on the strong echo characteristics of kidney stones, so that a greater weight is given to the strong echo area, thereby emphasizing the influence of the region when calculating the autocorrelation matrix.

[0084] Step 302: construct an autocorrelation matrix M of the kidney stone echo cluster at each pixel point in the video frame image:

[0085] ;

[0086] in, and Respectively represent the gradient of the pixel point (u, v) in the video frame image in the x direction and the y direction, and W represents the video frame image. Through the optimization of the autocorrelation matrix, the kidney stone feature points in the ultrasound image can be more effectively detected, and these feature points can be combined for subsequent motion tracking and coordinate compensation.

[0087] Among them, the ultrasound image gradient calculation formula is:

[0088] .

[0089] in , They refer to the gradient of the image in the x and y directions respectively.

[0090] In a possible embodiment, the process of detecting and obtaining kidney stone feature points in the video frame image in step 3 includes:

[0091] Step 303: Calculate pixel response value , response value Defined as the minimum eigenvalue of the autocorrelation matrix M.

[0092] Specifically, .

[0093] in, and is the eigenvalue of the autocorrelation matrix M.

[0094] Step 304: Screening to obtain response value The feature points whose grayscale values ​​are greater than the set threshold of the strong echo group area.

[0095] The characteristic points are screened based on the characteristics of kidney stones, and the response value R is thresholded. Based on the strong echo characteristics of kidney stones, the characteristic points with significant strong echo characteristics are screened. The characteristic points need to meet the following conditions: ; Where T is the gray value threshold of the strong echo group.

[0096] Step 305: Eliminate feature points whose volumes are outside a set range to obtain kidney stone feature points.

[0097] Combined with the volume constraint, feature points are further screened to exclude feature points that are too small or too large, ensuring that the detected feature points are within the volume range of kidney stones. According to the basic volume information of kidney stones, the default value of the stone diameter is set to 2 mm.

[0098] Specifically, the diameter of a kidney stone is generally between 2mm and 2cm, which occupies about 5 to 500 pixels when converted to an ultrasound frame image. Based on the volume information of the kidney stone itself, the strong echo group area can be screened to select the feature point area that meets the volume constraint conditions, eliminate noise and false detection, avoid the influence of misidentified areas, and finally obtain stable and real-time coordinates of the kidney stone.

[0099] Step 4: Track the kidney stone feature points based on the Lucas-Kanade optical flow method to obtain the kidney stone trajectory.

[0100] After obtaining the kidney stone echo point cloud data, the Lucas-Kanade optical flow method is used to track the detected kidney stone feature points. The LK optical flow method is a feature optical flow tracking method, which refers to calculating the feature optical flow of an image through feature matching of a sequence of images and achieving separation of the target from the background through clustering.

[0101] In a possible implementation manner, step 4 includes: setting initial values ​​of adaptive parameters of the optical flow method, the adaptive parameters including: optical flow window size, maximum number of pyramid layers and number of optical flow calculation iterations.

[0102] The specific implementation process is summarized as follows: in order to consider sufficient local image grayscale value information and improve the ability to handle large-scale respiratory motion changes, the optical flow window size is set to 21×21, the maximum number of pyramid levels is set to 3, and the optical flow calculation termination condition is set to 10.

[0103] In a possible embodiment, after step 4, the method further includes: using a Kalman filter model to smooth the trajectory of the kidney stone. Kalman filtering is a recursive optimal estimation method. By predicting and updating the system state, it can effectively reduce the influence of noise and obtain a smooth trajectory of the kidney stone. Specifically, the process includes:

[0104] Initialize the custom parameters of the Kalman filter, including: state vector and the state transition matrix ; The state vector Defined as:

[0105] .

[0106] in, represents the position of kidney stones in the kth frame image, The speed of kidney stones;

[0107] The state transfer matrix F is defined as:

[0108] .

[0109] in, is the time interval between adjacent video frame images.

[0110] Through the Kalman filter recursive model, the basic state transfer equation and observation equation are substituted to update the trajectory of kidney stones.

[0111] After smoothing by the Kalman filter model, the motion trajectory of kidney stones can be accurately smoothed to reduce the impact of noise on stone movement.

[0112] Step 5, calculate the displacement of the kidney stone area between adjacent frames and output the current position of the kidney stone.

[0113] Example 2

[0114] Embodiment 2 provided by the present invention is an embodiment of a real-time compensation system for kidney stone coordinates based on ultrasonic image respiratory motion provided by the present invention. Figure 3 A structure diagram of a real-time compensation system for kidney stone coordinates based on ultrasonic image respiratory motion provided by an embodiment of the present invention, combined with Figure 3It can be seen that the embodiment of the system includes: a video acquisition and analysis module, an image preprocessing module, a feature point detection and tracking module, a trajectory analysis module and a displacement estimation output module.

[0115] The video acquisition and analysis module is used to extract video frame images from the real-time ultrasound video of the patient's kidney area and determine the corresponding relationship between each video frame image and the patient's respiratory movement.

[0116] The image preprocessing module is used to perform pixel value standardization and grayscale conversion on video frame images.

[0117] The feature point detection and tracking module is used to construct an autocorrelation matrix whose coefficients are associated with the characteristic thresholds of kidney stones, obtain an improved GFTT method based on the autocorrelation matrix, and detect the kidney stone feature points in the video frame image based on the improved GFTT method and volume constraints.

[0118] The trajectory analysis module is used to track the characteristic points of kidney stones based on the Lucas-Kanade optical flow method to obtain the trajectory of kidney stones.

[0119] The displacement estimation output module is used to calculate the displacement of the kidney stone area between adjacent frames and output the current position of the kidney stone.

[0120] It can be understood that the real-time compensation system for kidney stone coordinates based on ultrasonic image and respiratory movement provided by the present invention corresponds to the real-time compensation method for kidney stone coordinates based on ultrasonic image and respiratory movement provided by the aforementioned embodiments. The relevant technical features of the real-time compensation system for kidney stone coordinates based on ultrasonic image and respiratory movement can refer to the relevant technical features of the real-time compensation method for kidney stone coordinates based on ultrasonic image and respiratory movement, which will not be repeated here.

[0121] See also Figure 4 , Figure 4 Schematic diagram of an electronic device provided by an embodiment of the present invention. Figure 4As shown, an embodiment of the present invention provides an electronic device, including a memory 1310, a processor 1320, and a computer program 1311 stored in the memory 1310 and executable on the processor 1320. When the processor 1320 executes the computer program 1311, the following steps are implemented: extracting video frame images from real-time ultrasound video of the patient's kidney area, and determining the correspondence between each video frame image and the patient's respiratory movement; performing pixel value standardization processing and grayscale image conversion on the video frame images; constructing an autocorrelation matrix whose coefficients are associated with characteristic thresholds of kidney stones, obtaining an improved GFTT method based on the autocorrelation matrix, and detecting kidney stone feature points in the video frame images based on the improved GFTT method and volume constraints; tracking kidney stone feature points based on the Lucas-Kanade optical flow method to obtain kidney stone trajectories; calculating the displacement of the kidney stone area between adjacent frames, and outputting the current position of the kidney stone.

[0122] See also Figure 5 , Figure 5 Schematic diagram of an embodiment of a computer-readable storage medium provided by the present invention. Figure 5 As shown, this embodiment provides a computer-readable storage medium 1400, on which a computer program 1411 is stored. When the computer program 1411 is executed by a processor, the following steps are implemented: extracting video frame images from real-time ultrasound video of the patient's kidney area, and determining the correspondence between each video frame image and the patient's respiratory movement; performing pixel value standardization processing and grayscale conversion on the video frame images; constructing an autocorrelation matrix whose coefficients are associated with characteristic thresholds of kidney stones, obtaining an improved GFTT method based on the autocorrelation matrix, and detecting kidney stone feature points in the video frame images based on the improved GFTT method and volume constraints; tracking kidney stone feature points based on the Lucas-Kanade optical flow method to obtain kidney stone trajectories; calculating the displacement of the kidney stone area between adjacent frames, and outputting the current position of the kidney stone.

[0123] The embodiments of the present invention provide a method, system, electronic device and storage medium for real-time compensation of kidney stone coordinates based on respiratory movement in ultrasound images. By combining the improved GFTT algorithm, LK optical flow method and Kalman filter smoothing processing, it is possible to obtain accurate tracking and real-time coordinate compensation of kidney stones in respiratory movement, and realize fully automated respiratory movement detection during kidney stone surgery based on ultrasound images. It can effectively reduce the impact of intraoperative respiratory movement on stone positioning, enhance the degree of automation of lithotripsy, effectively improve the accuracy and safety of extracorporeal lithotripsy, and provide reliable technical support for intelligent extracorporeal lithotripsy.

[0124] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0125] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0126] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0127] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0128] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0129] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0130] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A real-time compensation method for kidney stone coordinates based on respiratory motion in ultrasound images, characterized in that: The method comprises: Step 1, extracting video frame images from the real-time ultrasound video of the patient's kidney area, and determining the corresponding relationship between each video frame image and the patient's respiratory movement; Step 2, performing pixel value normalization processing and grayscale conversion on the video frame image; Step 3, constructing an autocorrelation matrix whose coefficients are associated with characteristic thresholds of kidney stones, obtaining an improved GFTT method based on the autocorrelation matrix, and detecting and obtaining characteristic points of kidney stones in the video frame image based on the improved GFTT method and volume constraints; Step 4, tracking the kidney stone feature points based on the Lucas-Kanade optical flow method to obtain the kidney stone trajectory; Step 5, calculating the displacement of the kidney stone area between adjacent frames and outputting the current position of the kidney stone; The process of constructing the autocorrelation matrix of coefficients associated with characteristic thresholds of kidney stones in step 3 includes: Step 301: construct a weight function based on the strong echo characteristics of kidney stones : ; in, represents the weight at the pixel point (u,v), Represents the gray value at the pixel (u, v), Represents the weight of the kidney stone strong echo area, The threshold value of the gray value representing the strong echo group area; Step 302, constructing the autocorrelation matrix M of the kidney stone echo cluster at each pixel point in the video frame image: ; in, and Respectively represent the gradient of the pixel point (u, v) in the video frame image in the x direction and the y direction, and W represents the video frame image; The process of detecting and obtaining the kidney stone feature points in the video frame image in step 3 includes: Step 303: Calculate pixel response value , the response value Defined as the minimum eigenvalue of the autocorrelation matrix M; Step 304: Screen and obtain the response value Feature points whose grayscale values ​​are greater than the set threshold of the strong echo group area; Step 305: Eliminate the feature points whose volumes are outside a set range to obtain the kidney stone feature points.

2. The method according to claim 1, characterized in that The pixel value normalization processing of the video frame image in step 2 is Z-score normalization processing, including: Each video frame image is mean-variance normalized so that the pixel value of the video frame image has the characteristics of mean 0 and standard deviation 1.

3. The method according to claim 1, characterized in that The process of converting the video frame image into a grayscale image in step 2 includes: When the video frame image is a color image, the video frame image is converted into a grayscale image, and the conversion formula is: ; in, , and Respectively represent the pixel values ​​of the red, green and blue channels in the video frame image, Represents the converted grayscale image.

4. The method according to claim 1, characterized in that: The step 4 includes: setting initial values ​​of adaptive parameters of the optical flow method, wherein the adaptive parameters include: optical flow window size, maximum number of pyramid layers and number of optical flow calculation iterations.

5. The method according to claim 1, characterized in that After step 4, the step further includes: using a Kalman filter model to smooth the trajectory of the kidney stone, the process of which includes: Initialize the custom parameters of the Kalman filter, the custom parameters include: state vector and the state transition matrix ; The state vector Defined as: ; in, represents the position of kidney stones in the kth frame image, The speed of kidney stones; The state transfer matrix F is defined as: ; in, is the time interval between adjacent video frame images; Through the Kalman filter recursive model, the basic state transfer equation and observation equation are substituted to update the trajectory of kidney stones.

6. A real-time compensation system for kidney stone coordinates based on ultrasonic image respiratory motion, characterized in that: The system comprises: a video acquisition and analysis module, an image preprocessing module, a feature point detection and tracking module, a trajectory analysis module and a displacement estimation output module; The video acquisition and analysis module is used to extract video frame images from the real-time ultrasound video of the patient's kidney area, and determine the corresponding relationship between each video frame image and the patient's respiratory movement; The image preprocessing module is used to perform pixel value standardization processing and grayscale conversion on the video frame image; The feature point detection and tracking module is used to construct an autocorrelation matrix whose coefficients are associated with characteristic thresholds of kidney stones, obtain an improved GFTT method based on the autocorrelation matrix, and detect and obtain kidney stone feature points in the video frame image based on the improved GFTT method and volume constraints; The trajectory analysis module is used to track the kidney stone feature points based on the Lucas-Kanade optical flow method to obtain the kidney stone trajectory; The displacement estimation output module is used to calculate the displacement of the kidney stone area between adjacent frames and output the current position of the kidney stone; The process of constructing an autocorrelation matrix of coefficients associated with characteristic thresholds of kidney stones by the feature point detection and tracking module includes: Step 301: construct a weight function based on the strong echo characteristics of kidney stones : ; in, represents the weight at the pixel point (u,v), Represents the gray value at the pixel (u, v), Represents the weight of the kidney stone strong echo area, The threshold value of the gray value representing the strong echo group area; Step 302, constructing the autocorrelation matrix M of the kidney stone echo cluster at each pixel point in the video frame image: ; in, and Respectively represent the gradient of the pixel point (u, v) in the video frame image in the x direction and the y direction, and W represents the video frame image; The process of the feature point detection and tracking module detecting and obtaining the kidney stone feature points in the video frame image includes: Step 303: Calculate pixel response value , the response value Defined as the minimum eigenvalue of the autocorrelation matrix M; Step 304: Screen and obtain the response value Feature points whose grayscale values ​​are greater than the set threshold of the strong echo group area; Step 305: Eliminate the feature points whose volumes are outside a set range to obtain the kidney stone feature points.

7. An electronic device, characterized in that: It includes a memory and a processor, and the processor is used to implement the steps of the real-time compensation method for kidney stone coordinates based on ultrasonic image respiratory motion as described in any one of claims 1 to 5 when executing the computer management program stored in the memory.

8. A computer-readable storage medium, characterized in that: A computer management program is stored thereon, and when the computer management program is executed by the processor, the steps of the real-time compensation method for kidney stone coordinates based on ultrasonic image respiratory motion as described in any one of claims 1 to 5 are implemented.

Citation Information

Patent Citations

  • Traveling distance measuring device of moving object, traveling distance measuring method of moving object and program thereof

    JP2015152975A

  • Apparatus and method for target tracking of image

    KR1020160136145A

  • An apparatus, a method and a computer program for video coding and decoding

    WO2024008993A1