Ultrasound image based intraocular lens positioning method for ophthalmic surgery

Abstract

This invention relates to the field of image data processing technology, specifically to a method for lens localization in ophthalmic surgery based on ultrasound imaging. The method includes the following steps: 1) constructing a grayscale matrix and a mask matrix; 2) based on the grayscale matrix and the mask matrix, determining the left and right capsule truncated pixels, and calculating the capsule extension trend vector for each capsule truncated pixel; 3) for any left and right capsule truncated pixels, calculating the geometric deviation cost between them, and pairing the left and right capsule truncated pixels with the goal of minimizing the total geometric deviation cost, to obtain each capsule truncated pixel pair; 4) based on each capsule truncated pixel pair, determining the patient's lens region.

Description

Technical Field

[0001] This invention relates to the field of image data processing technology, and specifically to a method for lens localization in ophthalmic surgery based on ultrasound imaging. Background Technology

[0002] In ophthalmic ultrasound surgeries such as phacoemulsification, surgeons need to monitor the morphology and position of the lens capsule in real time. However, during the procedure, when metal medical instruments enter the anterior chamber of the eye, the extremely high reflectivity of the metal material causes strong reflections of the emitted ultrasound waves upon contact with the instrument surface. This results in a significant attenuation of the ultrasound echo signal directly below the instrument, creating a noticeable anechoic obstruction area in the ultrasound image. This obstruction interrupts the originally continuous image of the lens capsule tissue, leading to large areas of pixel loss.

[0003] Existing image segmentation algorithms, when dealing with large areas of pixel loss, connect the breakpoints to form a closed contour. Then, a region-filling algorithm (such as flood filling) is used to obtain all pixels within the closed contour, and this filled area is directly used as the lens region. However, because the anterior and posterior capsules of the eye lens are extremely close and arranged in approximately parallel arcs, it is very easy for the breakpoints of the upper and lower capsules to intersect and become misconnected, resulting in low lens recognition accuracy. Summary of the Invention

[0004] To address the technical problem of low lens identification accuracy caused by cross-connection of capsule breakpoints, the present invention aims to provide a lens localization method in ophthalmic surgery based on ultrasound imaging. The specific technical solution adopted is as follows: This invention provides a method for lens localization during ophthalmic surgery based on ultrasound imaging, comprising the following steps: 1) Construct a grayscale matrix and a mask matrix based on two-dimensional ultrasound images of the patient's eyes; 2) Based on the gray value matrix and the mask matrix, determine the left and right capsule truncated pixels, and calculate the capsule extension trend vector of each capsule truncated pixel. 3) For any left-side capsule truncated pixel and any right-side capsule truncated pixel, calculate the vertical height difference between them based on their vertical positions, and calculate the degree of trend mismatch between them based on their capsule extension trend vectors; fuse the vertical height difference and the degree of trend mismatch to obtain the geometric deviation cost between them, and pair the left-side capsule truncated pixels and the right-side capsule truncated pixels with the goal of minimizing the total geometric deviation cost to obtain each capsule truncated pixel pair; 4) Determine the patient's lens region based on each capsule truncated pixel pair.

[0005] Among some possible implementations, methods for determining the patient's lens region include: The connection curves of each capsule truncated pixel pair are constructed. Based on the curvature of the connection curves of each capsule truncated pixel pair, the capsule interpolation probability accumulation matrix is ​​constructed. The gray value matrix, mask matrix and capsule interpolation probability accumulation matrix are input into the trained lens prediction model to obtain the lens region prediction matrix, thereby determining the patient's lens region.

[0006] In some possible implementations, methods for calculating the degree of non-compliance of the trend include: Based on the cosine of the spatial angle between the two capsule extension trend vectors, the degree of non-coherence of their capsule extension trend vectors is calculated.

[0007] Among some possible implementations, methods for constructing the mask matrix include: Based on the gray value matrix, the pixel gradient in the vertical direction of each pixel is calculated. Based on the gray value of each pixel and the pixel gradient in the vertical direction, the pixel points with abrupt reflection changes at the lower edge of each instrument are determined. The gray value fluctuation of the region in the same column below the pixel points with abrupt reflection changes at the lower edge of each instrument is calculated, thereby determining the anechoic area of ​​the instrument occlusion. The mask values ​​of the pixels inside and outside the anechoic area of ​​the instrument occlusion are set respectively to construct the mask matrix.

[0008] In some possible implementations, methods for determining the pixel points of abrupt reflection changes at the lower edge of each instrument include: (1) Starting from the top pixel of a column of pixels, check each pixel from top to bottom to see if it meets the set reflection change condition. If it meets the set reflection change condition, then the pixel is taken as the lower edge reflection change pixel of the instrument. If it does not meet the set reflection change condition, continue to check the next pixel to see if it meets the set reflection change condition, until a pixel is found that meets the set reflection change condition or the last pixel of the column of pixels is reached. (2) Repeat the above steps for the next column of pixels until the last column of pixels has been traversed.

[0009] In some possible implementations, the setting of the reflection mutation condition includes: The grayscale value of the pixel is greater than the grayscale threshold for strong reflectivity of the instrument, and the pixel gradient in the vertical direction is less than the vertical gradient abrupt change threshold.

[0010] In some possible implementations, methods for determining the left and right capsule truncation pixels include: Morphological gradient operations are performed on the mask matrix to obtain the boundary edge matrix of the anechoic zone; Edge detection is performed on the grayscale matrix to obtain an initial binary image. The initial binary image is then multiplied pixel by pixel with the mask matrix to obtain the capsule edge binary matrix. The common pixels in the boundary edge matrix of the anechoic region and the binary matrix of the capsule edge are determined. Based on the relative spatial position of the common pixels and the anechoic region occluded by the instrument, the common pixels are divided into left capsule truncated pixels and right capsule truncated pixels.

[0011] In some possible implementations, methods for calculating the capsule extension trend vector for each capsule truncated pixel include: For any left-side capsule truncated pixel, starting from the left-side capsule truncated pixel, based on the capsule edge binary matrix, the adjacent capsule edge pixels are determined sequentially, and backtracking is performed to the left of the image to determine the left backtracking endpoint. Based on the position of the left-side capsule truncated pixel and its left backtracking endpoint, the capsule extension trend vector of the left-side capsule truncated pixel is calculated. For any right-side capsule truncated pixel, starting from the right-side capsule truncated pixel, based on the capsule edge binary matrix, the adjacent capsule edge pixels are determined sequentially, and backtracking is performed to the right of the image to determine the right backtracking endpoint. Based on the position of the right-side capsule truncated pixel and the right backtracking endpoint, the capsule extension trend vector of the right-side capsule truncated pixel is calculated.

[0012] Among some possible implementations, methods for constructing the capsule interpolation probability accumulation matrix include: (1) Based on the curvature of any pixel on the connection curve of any pair of truncated capsule pixels, calculate the Gaussian kernel amplitude of the pixel on the connection curve, construct a Gaussian kernel with the pixel as the center, multiply each value of the Gaussian kernel with the Gaussian kernel amplitude to obtain the Gaussian kernel matrix of the pixel, and then obtain the Gaussian kernel numerical matrix. (2) Summing the Gaussian kernel value matrix of each pixel on the connection curve of the truncated pixel pair of the capsule, we can obtain the Gaussian kernel value matrix of the truncated pixel pair of the capsule. (3) Summing the Gaussian kernel numerical matrix of each capsule truncated pixel pair to obtain the comprehensive Gaussian kernel numerical matrix, and then calculating the capsule interpolation probability accumulation matrix.

[0013] Among some possible implementations, methods for constructing the connection curves for each capsule truncated pixel pair include: Based on the coordinates of the truncated pixels in any capsule truncated pixel pair and the capsule extension trend vector, the expression of the connection curve of the capsule truncated pixel pair is obtained by using cubic Hermite spline polynomial operation.

[0014] The present invention has the following beneficial effects: In the lens localization method for ophthalmic surgery based on ultrasound imaging, after determining the left and right capsule truncation pixels, for any left and right capsule truncation pixels, the geometric deviation cost between them is calculated based on their longitudinal positions. Since the longitudinal position difference between the upper and lower capsule truncation points is large, the geometric deviation cost is also large. Furthermore, when pairing left and right capsule truncation pixels, the pairing is performed with the goal of minimizing the total geometric deviation cost. Under this constraint, pairing avoids matching capsule truncation pixels from different layers, thus avoiding the problem of cross-connection of interruption points in existing technologies, thereby improving the lens recognition accuracy. Attached Figure Description

[0015] Figure 1 This is a flowchart of a lens localization method in ophthalmic surgery based on ultrasound imaging, according to an embodiment of the present invention. Figure 2 This is a flowchart illustrating the construction of the capsule interpolation probability accumulation matrix according to an embodiment of the present invention. Detailed Implementation

[0016] To clearly illustrate the technical features of this solution, the invention will be described in detail below through specific embodiments and in conjunction with the accompanying drawings.

[0017] The specific procedure of the lens localization method in ophthalmic surgery based on ultrasound imaging of the present invention is as follows: Figure 1 As shown, it includes the following steps: Step 1: Based on clear two-dimensional ultrasound images of the eye, construct a training sample set and train the lens prediction model to obtain the trained lens prediction model.

[0018] 1.1) Based on clear two-dimensional ultrasound images of the eye, extract the eye region and generate two-dimensional ultrasound images of the damaged eye to construct a training sample set.

[0019] In this embodiment, a large number of clear two-dimensional ultrasound images of the eye that are not obscured by instruments are acquired. Pixel coordinates are extracted along the lens in the clear two-dimensional ultrasound images of the eye by manual annotation to obtain lens region labels.

[0020] In each clear two-dimensional ultrasound image of the eye, a spatial location is randomly selected and covered with a two-dimensional matrix with all values ​​set to 0. The grayscale values ​​of the pixels covered by the matrix are modified to 0 to simulate the area covered by the medical device. A row of bright pixels with grayscale values ​​reaching the extreme value (e.g., a pixel value of 255) is added above the matrix to simulate the strong reflection phenomenon of the medical device contact surface, thus obtaining a two-dimensional ultrasound image of the damaged eye.

[0021] Next, for each two-dimensional ultrasound image of the damaged eye, a grayscale matrix is ​​generated. Simultaneously, a corresponding training mask matrix is ​​generated using the masking operation. Then, a corresponding capsule interpolation probability accumulation matrix is ​​generated by calculating breakpoint geometric interpolation based on the masking boundary. Subsequently, the grayscale matrix, the corresponding training mask matrix, and the capsule interpolation probability accumulation matrix are stacked and concatenated to generate a multi-channel training tensor of size 3×W×H, which is stored as a complete training sample in the training sample set.

[0022] 1.2) The lens prediction model is trained using the constructed training sample set to obtain the trained lens prediction model.

[0023] The constructed training sample set is divided into a training set and an independent validation set. The parameters of the lens prediction model are initialized, and the model is trained using the training set. In this embodiment, the multi-channel training tensor and corresponding lens region labels from the training sample set are input into the lens prediction model, and the model performs a forward propagation operation. The model outputs the predicted lens region, calculates the cross-entropy loss function between the output predicted lens region and the lens region label, calculates the gradient of the loss function using the backpropagation algorithm, and updates the internal parameters of the lens prediction model based on this gradient, such as the weight matrix of the convolutional kernels in each layer and the corresponding bias vector.

[0024] In this embodiment, the lens prediction model employs a multi-channel fully convolutional neural network, whose architecture can be U-Net, AttentionU-Net, or Res-UNet. This embodiment uses the U-Net architecture as an example. The multi-channel fully convolutional neural network includes: an input layer, a decoder, skip connection layers, and an output layer. The input layer receives a multi-channel input tensor of size 3×W×H. The encoder path contains multiple downsampling modules (such as max pooling layers), each of which alternately performs convolution and activation operations to extract multi-scale image features and compress spatial dimensions. The decoder path contains multiple upsampling modules symmetrical to the encoder (such as deconvolution layers or bilinear interpolation) to progressively restore the spatial resolution of the feature maps. Skip connections directly concatenate the shallow feature maps extracted from each level of the encoder with the corresponding deep feature maps from the corresponding levels of the decoder to fuse the tissue edge details at the lower level with the global morphological semantics at the higher level. The output layer uses a 1×1 convolutional layer combined with a sigmoid activation function to map the feature maps into a single-channel continuous probability distribution matrix, outputting a lens region prediction matrix of size 1×W×H. .

[0025] The above training sample input and parameter update operations are repeated. When the cross-entropy loss function value output by the lens prediction model on the independent validation set is lower than the set convergence benchmark value for ten consecutive cycles, the system stops the training operation and saves all weight parameters inside the lens prediction model. The network after training and parameter saving is used as the trained lens prediction model.

[0026] The set convergence benchmark value ranges from [0.01, 0.08]. If the benchmark value is set too high (e.g., greater than 0.08), it means the network stops training too early. At this time, the network has not yet fully learned the nonlinear fusion law of "geometric prior" and "weak image features" in the multi-channel tensor, and in real-time processing, it is very easy for the predicted lens contour to break due to discrete noise in the shadows. If the benchmark value is set too low (e.g., less than 0.01), it will cause the network to over-memorize the artificially constructed random speckle features in the training set samples, which not only greatly increases the unnecessary pre-training time, but also reduces the model's generalization ability. When encountering abnormal background noise from different patients in actual surgery, it will be unable to accurately predict. Therefore, in this embodiment, the convergence benchmark value is 0.05, which can ensure that the network achieves high-precision pixel-level classification requirements, while retaining good feature generalization and noise resistance.

[0027] Step 2: Based on real-time acquired single-frame two-dimensional ultrasound images of the patient's eye, construct a grayscale matrix and a mask matrix. Specific methods include: 2.1) Based on a single-frame two-dimensional ultrasound image of the patient's eye at the current moment, calculate the gray value of each pixel, construct a gray value matrix, and calculate the pixel gradient in the longitudinal direction of each pixel to determine the pixel point of the abrupt change in reflection at the lower edge of the instrument.

[0028] Due to hardware limitations in ultrasound detection equipment, discrete speckle noise can occur in single-frame two-dimensional ultrasound images, affecting recognition accuracy. To address this issue, this embodiment first performs filtering and noise reduction on the current single-frame two-dimensional ultrasound image of the patient's eye. Based on the grayscale values ​​of each pixel in the filtered and denoised two-dimensional ultrasound image, a grayscale value matrix is ​​constructed. Then, based on the grayscale value matrix, the pixel gradient in the vertical direction of each pixel is calculated.

[0029] The filtering method used is Gaussian filtering.

[0030] Specifically, methods for constructing grayscale matrices include: Based on a single-frame two-dimensional ultrasound image of the patient's eye at the current moment, the number of horizontal pixels (W) and vertical pixels (H) are determined. A reference coordinate system of size W*H is established, and the coordinates of each pixel are determined. ,in The horizontal coordinate is (positive direction is to the right). The vertical axis is the vertical axis (downward is the positive direction). , Determine the grayscale value of each pixel. This yields a W*H grayscale matrix. .

[0031] Calculate each pixel in each grayscale matrix The pixel gradient in the vertical direction of each pixel is obtained by calculating the difference between its gray value and the gray value of its adjacent pixel (x, y+1). The formula is as follows: In the formula, For pixels The pixel gradient along the vertical direction. If the value of the gradient is significantly negative, it indicates that the gray level of the image has decreased sharply in the vertical direction. Let (x, y) be the grayscale value of the pixel. This represents the grayscale value of pixel (x, y+1). If pixel (x, y+1) does not exist, then set to grayscale. It is 0.

[0032] Based on the grayscale values ​​of each pixel in the grayscale matrix and the pixel gradient along the vertical direction of each pixel, the pixels with abrupt changes in reflection at the lower edge of the instrument are determined. Specific methods include: 1) Starting from the top pixel of a column of pixels, check each pixel from top to bottom to see if it meets the set reflection change condition. If it meets the set reflection change condition, then the pixel is designated as the lower edge reflection change pixel of the instrument. If it does not meet the set reflection change condition, continue to check the next pixel to see if it meets the set reflection change condition, until a pixel is found that meets the set reflection change condition or the last pixel of the column is reached, then stop checking. 2) Repeat the above steps for the next column of pixels until the last column of pixels has been traversed.

[0033] For any given pixel, the reflection abrupt change conditions include: The grayscale value of this pixel is greater than the instrument's strong reflectivity grayscale threshold, and the pixel gradient in the vertical direction of this pixel is less than the vertical gradient abrupt change threshold. In an 8-bit ultrasound image (0-255), the hyperechoic grayscale of ordinary soft tissues of the eye (such as the cornea and sclera) is usually between 100 and 150, while metal medical instruments strongly reflect ultrasound waves, causing the probe to receive signals close to saturation, with grayscale values ​​usually above 200. Therefore, the range of the instrument's strong reflectivity grayscale threshold is [150, 200]. In this embodiment, the instrument's strong reflectivity grayscale threshold is 200. Ultrasound waves cannot penetrate metal, resulting in an acoustic shadow zone directly below the instrument's contact surface. This manifests as extremely high grayscale values ​​for the upper pixels (200+), and a sudden drop in grayscale values ​​for adjacent lower pixels to extremely low background (e.g., below 50), resulting in a vertical gradient difference that is a very large negative number (e.g., 50-200=-150). The ultrasound grayscale transition of normal human tissue is smooth and gradual, and the vertical gradient difference of a single pixel is usually within -50. If the threshold is set too high (e.g., -30), normal tissue light-dark transitions will be misjudged as a cliff; if the threshold is set too low (e.g., below -150), slight diffraction at the edge of the ultrasound beam will cause the grayscale decrease to be not steep enough, which will lead to missed detection of the lower edge of the instrument. Therefore, the range of the longitudinal gradient abrupt change threshold is [-150, -80]. In this embodiment, the longitudinal gradient abrupt change threshold is set to -100.

[0034] 2.2) Determine the anechoic area obscured by the instrument and construct the mask matrix.

[0035] Methods for determining the anechoic zone of instrument occlusion include: for any pixel point with abrupt reflection at the lower edge of the instrument Calculate all pixels in the same column below the pixel with the abrupt change in reflection at the lower edge of the device. , , ..., variance of gray values The variance of the grayscale values ​​is used to characterize the degree of grayscale fluctuation in the same column region below the pixel with a sudden change in reflection at the lower edge of the instrument. The smaller the variance of the grayscale values, the weaker the grayscale fluctuation in the same column region below the pixel with a sudden change in reflection at the lower edge of the instrument, and the closer its internal characteristics are to a uniform dark area caused by the lack of ultrasonic penetration. If the variance of the grayscale values ​​of the pixel with a sudden change in reflection at the lower edge of the instrument and all the pixels in the same column below it is less than a preset anechoic variance threshold, then the grayscale value is considered to be within the range specified in the original text. Then determine the pixel point of the abrupt change in reflection at the lower edge of the device. The area in the same column below represents the anechoic region where the instrument is obscured. In this embodiment, the anechoic variance threshold is set to a value within the range of [5%, 10%] of the global pixel grayscale variance of the current single-frame two-dimensional ultrasound image. In this embodiment, it is preferably configured to be 8% of this global variance. The noise floor variance of the pure acoustic shadow region is stable below 10% of the overall image structure variance. If the ratio is set greater than 10%, the system has a high error tolerance and is prone to misjudging the normal anterior chamber of the eye containing slight turbidity (producing weak interface echoes) as an instrument obscured region; if the ratio is set less than 5%, the system conditions are too harsh and are easily interfered with by individual discrete high-frequency speckle noise points in the image, causing the real obscured region to fail verification.

[0036] Perform the above steps on all the pixels with abrupt changes in reflection at the lower edge of each instrument to obtain the anechoic zone of each instrument.

[0037] Based on the anechoic regions blocked by each instrument, a mask matrix is ​​constructed. Specific methods include: Construct a W*H empty matrix, setting the values ​​of pixels inside the anechoic region obscured by the instrument to 0 and the values ​​of pixels outside the anechoic region to 1. This results in a matrix consisting of 0s and 1s, which is the mask matrix. .

[0038] Step 3: Based on the constructed mask matrix and grayscale matrix, determine the left and right capsule truncated pixels, and calculate the capsule extension trend vector of the left and right capsule truncated pixels.

[0039] 3.1) Based on the constructed mask matrix and gray value matrix, construct the boundary edge matrix of the anechoic region and the binary matrix of the capsule edge, and then determine the truncated pixel points of the left capsule and the truncated pixel points of the right capsule.

[0040] Methods for constructing the boundary edge matrix of the anechoic region include: For mask matrix Perform morphological gradient calculations to obtain the boundary edge matrix of the anechoic region. Morphological gradient operations are performed on the mask matrix by applying morphological dilation and morphological erosion operations respectively. The difference between the morphological matrices obtained from these two operations yields a matrix where the values ​​at the anechoic region boundaries are 1 and the values ​​in other regions are 0. This matrix is ​​the anechoic region boundary matrix. .

[0041] Methods for constructing a binary matrix of the capsule edge include: processing the grayscale matrix Perform Canny edge detection to obtain an initial binary image, and then combine the initial binary image with the mask matrix. Perform pixel-by-pixel multiplication to obtain the binary matrix of the capsule edge. The pixel-by-pixel multiplication operation erases the grayscale values ​​of the original image pixels corresponding to the 0-value regions in the mask matrix.

[0042] The boundary edge matrix of the anechoic zone binary matrix of the capsule edge Pixels with a median value of 1 are designated as common pixels. For each common pixel, its relative spatial position to the anechoic area obscured by the instrument is determined. Common pixels located to the left of the anechoic area obscured by the instrument are designated as the left capsule truncation pixel. The common pixel located to the right of the anechoic area obscured by the instrument is denoted as the right capsule truncation pixel. .in, For the first The coordinates of the left-side capsule truncated pixel are: , For the first The right-side capsule truncated pixel point has the following coordinates: , , .

[0043] 3.2) Based on the left and right capsule truncated pixels, calculate the capsule extension trend vector for each capsule truncated pixel.

[0044] Because the anechoic area (mask 0 value region) obscured by the instrument is a continuous dark area in the middle of the image, the lens capsule is truncated in the middle by this dark area, and the left side of the capsule is truncated by pixels. Located at the left boundary of the dark area, the right side of the breakpoint is an anechoic dark area (no effective capsule pixels), and the left side is a reliable imaging area (with a continuously visible capsule edge). We need to use the pixels in the imageable area on the left to determine the capsule extension trend vector of the truncated pixels on the left; the right capsule truncated pixels... Located at the right boundary of the dark area, the left side of the breakpoint is an anechoic dark area (without effective capsule pixels), and the right side is a reliable imaging area (with a continuously visible capsule edge). The capsule extension trend vector of the right-side truncated capsule pixels needs to be determined using the pixels in the imageable area on the right. The specific method for calculating the capsule extension trend vector includes: For any pixel truncated by the left capsule: Starting from the left-side capsule truncated pixel, based on the capsule edge binary matrix, adjacent capsule edge pixels are sequentially determined, and backtracking is performed to the left of the image. When the backtracking path length reaches a set length (5 pixels in this embodiment), or when the line endpoint is encountered and backtracking to the left cannot continue, the left backtracking endpoint is determined, and it is determined whether the backtracking endpoint is the same as the left-side capsule truncated pixel. If they overlap, discard the left capsule truncated pixel; otherwise, calculate the path from the backtracking endpoint to the left capsule truncated pixel. The spatial direction vector is used to generate the capsule extension trend vector corresponding to that pixel. .

[0045] Specific backtracking methods include: truncating pixels on the left capsule. This is the current backtracking node. In the binary graph at the edge of the capsule... In the middle, read the three adjacent coordinate positions to the left of the current backtracking node, that is, to the left of... upper left and the lower left The system determines whether the pixel value at these three coordinate positions is 1. If a coordinate point with a pixel value of 1 exists, it is used as the next backtracking node. If multiple coordinate points with a pixel value of 1 exist, the single coordinate point with the highest priority (left, top left, bottom left) is extracted as the next backtracking node. The current backtracking node is updated, and the above left neighborhood search operation is performed repeatedly. Backtracking stops when the cumulative movement reaches 5 pixels, or when the pixel values ​​of the three adjacent coordinates on the left are all 0 and further movement is impossible. The system extracts the coordinates of the current backtracking node at the time of stopping as the backtracking endpoint corresponding to the left capsule truncated pixel. The backtracking endpoint determined by this method is the capsule edge pixel to the left of the left capsule truncated pixel.

[0046] For any pixel truncated by the right capsule: Starting from the right-side capsule truncated pixel, the algorithm backtracks to the right of the image based on the capsule edge binary matrix. When the backtracking path length reaches a set length, or when it encounters the end of a line and cannot continue backtracking to the right, the right-side backtracking endpoint is determined, and it is judged whether the backtracking endpoint is the same as the right-side capsule truncated pixel. If the pixels overlap, discard the right-side capsule truncated pixel to prevent the generated capsule extension trend vector corresponding to that pixel from being a 0 vector. If they do not overlap, calculate the path from the backtracking endpoint to the right-side capsule truncated pixel. The spatial direction vector is used to generate the capsule extension trend vector corresponding to that pixel. .

[0047] Specific backtracking methods include: truncating pixels on the right capsule. This is the current backtracking node. In the binary graph at the edge of the capsule... In the middle, read the three adjacent coordinate positions to the right of the current backtracking node, that is, directly to the right. upper right and the lower right The system determines whether the pixel values ​​at these three coordinate positions are 1. If a coordinate point with a pixel value of 1 exists, it is used as the next backtracking node. If multiple coordinate points with a pixel value of 1 exist, the single coordinate point at the top of the sorted list, in the order of right, top right, and bottom right, is extracted as the next backtracking node. The current backtracking node is updated, and the above right-side neighborhood search operation is performed repeatedly. Backtracking stops when the cumulative movement reaches 5 pixels, or when the pixel values ​​of the three adjacent coordinates on the right are all 0 and further movement is impossible. The system extracts the coordinates of the current backtracking node at the time of stopping as the backtracking endpoint corresponding to the right-side capsule truncated pixel. The backtracking endpoint determined by this method is the capsule edge pixel to the right of the right-side capsule truncated pixel.

[0048] Step 4: Pair the left and right capsule truncated pixels together to obtain capsule truncated pixel pairs.

[0049] 4.1) Pad up the number of truncated pixels on the left and right sides of the capsule, calculate the geometric deviation cost between each truncated pixel on the left and right sides of the capsule, and construct the geometric deviation cost matrix.

[0050] Determine the number of pixels N truncated on the left side of the capsule and the number of pixels M truncated on the right side of the capsule, and take the maximum value K. The side with fewer pixels is supplemented with K pixels by using virtual pixels that act as placeholders to ensure that the number of truncated pixels on the left and right sides of the capsule is equal.

[0051] The geometric deviation cost between each left-side capsule truncated pixel and each right-side capsule truncated pixel is calculated. The specific calculation method includes: In the formula, For the first One left-side capsule truncated pixel and the One right-side capsule truncated pixel The cost of geometric deviation between them For the first One left-side capsule truncated pixel The ordinate, For the first One right-side capsule truncated pixel The ordinate, This represents the number of vertical pixels. For the first One left-side capsule truncated pixel The vector of the capsule's extension trend. For the first One right-side capsule truncated pixel The vector of the capsule's extension trend. Let cosine be the spatial angle between the two capsule extension trend vectors. Adding it to a constant 1 represents the deviation penalty of the two direction vectors. The larger the value of this term, the lower the degree of conformity between the original trend directions of the two breakpoints. and These are the preset first and second weighting coefficients, respectively. In this embodiment, and They are of equal importance, therefore . Used to measure the pixel point of the left capsule truncation. And the right capsule truncated pixels The degree of vertical height difference between the two is divided by H to eliminate dimensions and perform normalization, and squared to prevent the value from being less than 0. Furthermore, a second nonlinear scaling is applied to the degree of vertical height difference. When the value is small, the reduction through non-linear scaling is greater, and the scaled value is smaller. When the value is large, the degree of shrinkage of nonlinear scaling is relatively small. Cut off pixels on the left capsule And the right capsule truncated pixels The degree to which the extension trend vectors of the lens capsule do not conform is significant. Normally, the lens capsule is a smooth, continuous curve. At the junction, the spatial directions of these two capsule extension trend vectors should be 180 degrees apart (i.e., -180 degrees). If the trends at the two breakpoints do not align (e.g., forming an acute-angled broken line or pointing in the same direction), the cosine of the included angle will be greater than -1 and approach 1. At this point... The result will approach the maximum value of 2, resulting in a huge cost penalty, and the left capsule truncates pixels in different rows. And the right capsule truncated pixels The spatial direction of the capsule extension trend vector has a large angle. The values ​​are also relatively large. In summary, the cosine of the angle between the vertical height difference and the capsule extension trend vector is used as two penalty terms to truncate pixels on the left side of the capsule in different rows. And the right capsule truncated pixels It has a large penalty.

[0052] It should be noted that the above calculation method only truncates pixels on the left side of the capsule. And the right capsule truncated pixels This setting only applies when both pixels are actual, real pixels. If one of the two pixels is a virtual pixel, then it is set directly. Cost of preset geometric deviation In this embodiment The value of is greater than or equal to the theoretically calculated maximum geometric deviation result of 3, which is generated by the combination of any two real coordinate points under the current image resolution. In this embodiment... The value is set to 3.

[0053] Will The first one written into the matrix Line number The column yields the geometric deviation cost matrix.

[0054] 4.2) Global optimization is performed based on the geometric deviation cost matrix to obtain a combination of capsule truncation pixel pairs with the minimum total geometric deviation cost.

[0055] In this embodiment, the geometric deviation cost matrix is ​​input into the Hungarian allocation algorithm to obtain a combination of capsule truncation pixel pairs that minimizes the total geometric deviation cost. The Hungarian allocation algorithm is based on the mathematical principle of minimizing the sum of matrix element extraction, and outputs a combination that minimizes the global geometric deviation cost. The K pairs of capsule truncation pixels with the smallest sum value are selected. Each pair of capsule truncation pixels includes a left capsule truncation pixel and a right capsule truncation pixel.

[0056] Since virtual pixels were added in step 4.1) of this embodiment, they should also be deleted in this step. Specifically, this involves deleting the K pairs of capsule truncated pixels that contain virtual pixels, leaving... Each capsule truncation pixel pair.

[0057] For the capsule truncated pixel pair after removing virtual pixels, the coordinates of the left capsule truncated pixel and the capsule extension trend vector, and the coordinates of the right capsule truncated pixel and the capsule extension trend vector are used to form a parameter array for the capsule truncated pixel pair. .

[0058] Because the left-side capsule truncates pixels in different rows And the right capsule truncated pixels The geometric deviation between them is costly, so when using the Hungarian algorithm for optimization, it tends to truncate pixels on the left side of the capsule in the same row. And the right capsule truncated pixels The capsule is composed of truncated pixel pairs.

[0059] Step 5: Parameter array based on each capsule truncated pixel pair Construct connection curves for each pair of truncated pixels in the capsule, and construct a capsule interpolation probability accumulation matrix based on the curvature of the connection curves.

[0060] 5.1) Parameter array based on any capsule truncated pixel pair A smooth, continuous curve (connection curve) is generated at the break point that is tangent to the original capsule direction to connect the two capsule cut-off pixels. At the same time, the curvature of each pixel on the connection curve is calculated.

[0061] For any pair of pixels truncated by the capsule, the parameter array The coordinates of the left capsule truncated pixel are used as the starting coordinates, the capsule extension trend vector of the left capsule truncated pixel is used as the starting tangent vector, and the coordinates of the right capsule truncated pixel are used as the ending coordinates. The capsule extension trend vector of the right capsule truncated pixel is multiplied by -1 to reverse its polarity (since the capsule extension trend vector of the right capsule truncated pixel is from right to left, opposite to the direction of the capsule extension trend vector of the left capsule truncated pixel, polarity reversal is required), and used as the ending tangent vector. Substituting this into a cubic Hermitian spline polynomial operation, an expression for a smooth continuous curve tangent to the original capsule direction at the breakpoint is obtained, which is the expression for the connection curve of the capsule truncated pixel pair. The specific method is existing technology and will not be described in detail in this embodiment. Based on the expression for the connection curve of the capsule truncated pixel pair, multiple discrete sampling points are taken at equal intervals, and the coordinates of each discrete sampling point are calculated (each coordinate corresponds to a pixel on the connection curve, so the coordinates below are also called pixel points), to obtain the discrete coordinate sequence of the connection curve. Calculate the curvature of the connection curve at each coordinate in its coordinate sequence to quantify the degree of curvature of each pixel of the connection curve.

[0062] 5.2) Based on the curvature of the connection curve of the capsule truncated pixel pair at each coordinate in its coordinate sequence, construct the Gaussian kernel numerical matrix of the capsule truncated pixel pair, and superimpose the Gaussian kernel numerical matrices of all capsule truncated pixel pairs to obtain the capsule interpolation probability accumulation matrix.

[0063] The flowchart for constructing the capsule interpolation probability accumulation matrix is ​​as follows: Figure 2 As shown, it specifically includes: (1) Based on the curvature of any pixel point of the connection curve of any truncated pixel point pair (curvature of the connection curve at each coordinate in its coordinate sequence), calculate the Gaussian kernel amplitude A at that coordinate (pixel point), and then generate a two-dimensional Gaussian kernel with a fixed size and a fixed standard deviation with that coordinate as the center. Multiply all values ​​in the kernel by the Gaussian kernel amplitude A to scale them, and obtain the two-dimensional Gaussian kernel matrix at that coordinate. By padding the two-dimensional Gaussian kernel matrix with zeros, the two-dimensional Gaussian kernel is constructed into a Gaussian kernel numerical matrix with a size of W*H.

[0064] The methods for calculating the Gaussian kernel amplitude A include: In the formula, A is the Gaussian kernel amplitude. The curvature sensitivity attenuation coefficient is set, ranging from 3 to 10. The larger the value, the faster the attenuation. In this embodiment... The value is 5. It is a natural exponential function. The degree of curvature.

[0065] In this embodiment, the size and standard deviation of the Gaussian kernel are fixed values ​​(e.g., standard deviation). =1.5 pixels, Gaussian kernel size is 3 (≈5×5 pixels), scale matching the actual thickness of the capsule in the ultrasound image. The only thing that changes dynamically with position is the central Gaussian kernel amplitude A, which is determined by the curvature of that point. The Gaussian kernel amplitude A represents the confidence level at the coordinate, which is obtained by exponentially back-mapping the curvature at that coordinate, so that smooth curves are treated with high confidence and kinks are treated with low confidence.

[0066] (2) Repeat step (1) for each coordinate in the coordinate sequence of the connection curve of the truncated pixel pair of the capsule to obtain the Gaussian kernel value matrix of each coordinate. Summate the Gaussian kernel value matrices of each coordinate to obtain the Gaussian kernel value matrix of the truncated pixel pair of the capsule.

[0067] (3) Repeat steps (1) and (2) for the connection curves of each capsule truncated pixel pair to obtain the Gaussian kernel value matrix of each capsule truncated pixel pair. Then sum the Gaussian kernel value matrices of each capsule truncated pixel pair to obtain the comprehensive Gaussian kernel value matrix. Normalize the matrix to obtain the capsule interpolation probability accumulation matrix. .

[0068] In this embodiment, the normalization process includes: dividing each element in the comprehensive Gaussian kernel numerical matrix by the maximum value of the element in the comprehensive Gaussian kernel numerical matrix to complete the normalization process and obtain the capsule interpolation probability accumulation matrix. The purpose of normalization is to prevent the element values ​​from exceeding the limit of the Gaussian kernel peak after multiple accumulations.

[0069] Step 6: Based on the grayscale matrix, mask matrix, and capsule interpolation probability accumulation matrix, the lens prediction model trained in Step 1 is used to obtain the lens region prediction matrix, thereby determining the patient's lens region at the current moment.

[0070] 6.1) Convert the grayscale matrix Mask matrix and capsule interpolation probability accumulation matrix The data is input into the lens prediction model trained in step one, and nonlinear feature fusion of the neural network is performed to obtain the lens region prediction matrix. .

[0071] grayscale matrix Mask matrix and capsule interpolation probability accumulation matrix These three W×H two-dimensional matrices are stacked and concatenated to generate a 3×W×H multi-channel input tensor, which is then input into the lens prediction model trained in step one. Forward propagation feature extraction is then performed to obtain a single-channel lens region prediction matrix. The value of each pixel in the matrix represents the probability that each pixel is a lens.

[0072] In the above feature extraction process, the capsule interpolation probability accumulation matrix It can only provide a priori information on pure geometric connections under ideal conditions, lacking a natural depiction of the details of tissue edges, while the grayscale matrix... Faint acoustic speckle features often remain at the occlusion edges. By combining these two with the mask matrix... Common injection network, mask matrix In the zero-value region, the disordered amplification of background noise by the network was suppressed, while the capsule interpolation probability accumulation matrix was improved. The high-confidence trajectory-guided model traverses the anechoic region. Through nonlinear calculations, the model collaboratively integrates a purely smooth ideal geometric curve with the actual weak edge features of ultrasound, effectively solving the technical defects of harsh edges and unnatural transitions caused by simply using geometric connections.

[0073] 6.2) Based on the lens region prediction matrix, the lens region prediction results are obtained.

[0074] Due to the lens region prediction matrix The values ​​within the matrix are continuously distributed small probability numbers, and ultrasonic speckle noise may produce isolated high-value pixel blocks in the matrix. Therefore, the probability map needs to be transformed into a defined set of pixels and noise filtered out. The specific method in this embodiment includes: firstly, based on the lens region prediction matrix... The probability fractions of the continuous distribution within the lens are used to predict the lens region using Otsu's method. The probability of each pixel is compared with the probability threshold. Pixels with a probability less than the probability threshold are set to 0 (indicating that the pixel is not a lens), and pixels with a probability greater than or equal to the probability threshold are set to 1 (indicating that the pixel is a lens), thus obtaining a binarized mask image.

[0075] Otsu's method iterates through all probabilities in the interval [0,1], using each as a probability threshold to classify the crystalline and amorphous regions, and calculates the inter-class variance between the foreground and background after classification. When the inter-class variance reaches its maximum value, it means that the foreground target and background noise have been most thoroughly separated in terms of probability distribution. This probability is selected as the true probability threshold.

[0076] Based on a binary mask image, a connected component labeling algorithm is used to identify and extract the largest connected region (ranked first) as the main target of the lens, while discarding the remaining smaller connected regions to obtain the patient's lens region at the current moment. This area extraction operation is objectively equivalent to filtering out isolated debris responses caused by free tissue or speckle noise.

[0077] In this embodiment, for the patient's lens region preserved at the current moment, a boundary tracing algorithm is invoked to extract pixel coordinates one by one along the outermost edge contour of the region in a clockwise direction. All extracted coordinates are recorded and output as a set of pixel coordinates Sc representing the closed contour of the lens. This coordinate set constitutes a continuous and complete lens morphology positioning boundary reference after eliminating anechoic interference, which can be rendered and displayed in the ultrasound navigation interface for doctors to use as a structural reference.

Claims

1. A method for lens localization in ophthalmic surgery based on ultrasound imaging, characterized in that, Includes the following steps: 1) Construct a grayscale matrix and a mask matrix based on two-dimensional ultrasound images of the patient's eyes; 2) Based on the gray value matrix and the mask matrix, determine the left and right capsule truncated pixels, and calculate the capsule extension trend vector of each capsule truncated pixel. 3) For any left-side capsule truncated pixel and any right-side capsule truncated pixel, calculate the vertical height difference between them based on their vertical positions, and calculate the degree of non-compliance between their trends based on their capsule extension trend vectors. By integrating the degree of vertical height difference and the degree of non-compliance with the trend, the geometric deviation cost between the two is obtained. With the goal of minimizing the total geometric deviation cost, the left capsule truncated pixel points and the right capsule truncated pixel points are paired up to obtain each capsule truncated pixel point pair. 4) Determine the patient's lens region based on each capsule truncated pixel pair.

2. The method for lens localization in ophthalmic surgery based on ultrasound imaging according to claim 1, characterized in that, Methods for determining a patient's lens region include: The connection curves of each capsule truncated pixel pair are constructed. Based on the curvature of the connection curves of each capsule truncated pixel pair, the capsule interpolation probability accumulation matrix is ​​constructed. The gray value matrix, mask matrix and capsule interpolation probability accumulation matrix are input into the trained lens prediction model to obtain the lens region prediction matrix, thereby determining the patient's lens region.

3. The method for lens localization in ophthalmic surgery based on ultrasound imaging according to claim 1, characterized in that, Methods for calculating the degree of non-compliance of the trend include: Based on the cosine of the spatial angle between the two capsule extension trend vectors, the degree of non-coherence of their capsule extension trend vectors is calculated.

4. The method for lens localization in ophthalmic surgery based on ultrasound imaging according to claim 1, characterized in that, Methods for constructing a mask matrix include: Based on the gray value matrix, the pixel gradient in the vertical direction of each pixel is calculated. Based on the gray value of each pixel and the pixel gradient in the vertical direction, the pixel points with abrupt reflection changes at the lower edge of each instrument are determined. The gray value fluctuation of the region in the same column below the pixel points with abrupt reflection changes at the lower edge of each instrument is calculated, thereby determining the anechoic area of ​​the instrument occlusion. The mask values ​​of the pixels inside and outside the anechoic area of ​​the instrument occlusion are set respectively to construct the mask matrix.

5. The method for lens localization in ophthalmic surgery based on ultrasound imaging according to claim 4, characterized in that, Methods for determining the pixel points of abrupt changes in reflection at the lower edge of each instrument include: (1) Starting from the top pixel of a column of pixels, check each pixel from top to bottom to see if it meets the set reflection change condition. If it meets the set reflection change condition, then the pixel is taken as the lower edge reflection change pixel of the instrument. If it does not meet the set reflection change condition, continue to check the next pixel to see if it meets the set reflection change condition, until a pixel is found that meets the set reflection change condition or the last pixel of the column of pixels is reached. (2) Repeat the above steps for the next column of pixels until the last column of pixels has been traversed.

6. The method for lens localization in ophthalmic surgery based on ultrasound imaging according to claim 5, characterized in that, The specified reflection mutation conditions include: The grayscale value of the pixel is greater than the grayscale threshold for strong reflectivity of the instrument, and the pixel gradient in the vertical direction is less than the vertical gradient abrupt change threshold.

7. The method for lens localization in ophthalmic surgery based on ultrasound imaging according to claim 4, characterized in that, Methods for determining the truncated pixels on the left and right sides of the capsule include: Morphological gradient operations are performed on the mask matrix to obtain the boundary edge matrix of the anechoic zone; Edge detection is performed on the grayscale matrix to obtain an initial binary image. The initial binary image is then multiplied pixel by pixel with the mask matrix to obtain the capsule edge binary matrix. The common pixels in the boundary edge matrix of the anechoic region and the binary matrix of the capsule edge are determined. Based on the relative spatial position of the common pixels and the anechoic region occluded by the instrument, the common pixels are divided into left capsule truncated pixels and right capsule truncated pixels.

8. The method for lens localization in ophthalmic surgery based on ultrasound imaging according to claim 7, characterized in that, Methods for calculating the capsule extension trend vector of each capsule truncated pixel include: For any left-side capsule truncated pixel, starting from the left-side capsule truncated pixel, based on the capsule edge binary matrix, the adjacent capsule edge pixels are determined sequentially, and backtracking is performed to the left of the image to determine the left backtracking endpoint. Based on the position of the left-side capsule truncated pixel and its left backtracking endpoint, the capsule extension trend vector of the left-side capsule truncated pixel is calculated. For any right-side capsule truncated pixel, starting from the right-side capsule truncated pixel, based on the capsule edge binary matrix, the adjacent capsule edge pixels are determined sequentially, and backtracking is performed to the right of the image to determine the right backtracking endpoint. Based on the position of the right-side capsule truncated pixel and the right backtracking endpoint, the capsule extension trend vector of the right-side capsule truncated pixel is calculated.

9. The method for lens localization in ophthalmic surgery based on ultrasound imaging according to claim 2, characterized in that, Methods for constructing the capsule interpolation probability accumulation matrix include: (1) Based on the curvature of any pixel on the connection curve of any pair of truncated capsule pixels, calculate the Gaussian kernel amplitude of the pixel on the connection curve, construct a Gaussian kernel with the pixel as the center, multiply each value of the Gaussian kernel with the Gaussian kernel amplitude to obtain the Gaussian kernel matrix of the pixel, and then obtain the Gaussian kernel numerical matrix. (2) Summing the Gaussian kernel value matrix of each pixel on the connection curve of the truncated pixel pair of the capsule, we can obtain the Gaussian kernel value matrix of the truncated pixel pair of the capsule. (3) Summing the Gaussian kernel numerical matrix of each capsule truncated pixel pair to obtain the comprehensive Gaussian kernel numerical matrix, and then calculating the capsule interpolation probability accumulation matrix.

10. The method for lens localization in ophthalmic surgery based on ultrasound imaging according to claim 2, characterized in that, Methods for constructing connection curves for each membrane truncated pixel pair include: Based on the coordinates of the truncated pixels in any capsule truncated pixel pair and the capsule extension trend vector, the expression of the connection curve of the capsule truncated pixel pair is obtained by using cubic Hermite spline polynomial operation.

Images