Image Recognition-Assisted Localization Method and System for Hypospadias Surgery in Children

Abstract

This invention discloses an image recognition-assisted localization method and system for hypospadias surgery in children, relating to the field of image recognition technology. The method includes: acquiring a frame of surgical field image to determine the ventral surgical field region of the penis; obtaining the catheter mask and the axial reference line of the catheter through image segmentation; calculating the reliability field of the catheter reference along the line and the usability score of the catheter reference; determining whether the catheter reference is usable and determining the processing state; performing lateral scale correction on the initial unfolding mapping; performing optional tissue traction offset correction based on the surgical field anatomical segment label sequence; outputting the corresponding intraoperative auxiliary localization guidance path; and backprojecting it to the original image coordinate system for display. This invention can adapt to intraoperative surface deformation, improving localization accuracy and guidance robustness.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and more specifically, to an image recognition-assisted localization method and system for hypospadias surgery in children. Background Technology

[0002] Hypospadias in children is a common congenital malformation of the urogenital system. Surgical repair, such as urethroplasty, requires meticulous flap harvesting, tube coiling, and suturing on the ventral side of the penis. During the procedure, a standardized medical catheter or stent, such as Fr6, Fr8, or Fr10, with corresponding physical outer diameters of 2.0mm, 2.7mm, and 3.3mm, is pre-placed in the urethra as support and positioning reference. Because the surgical space is on the millimeter scale, and the penis has a cylindrical or conical curved surface, changes in perspective, tissue traction, and fluoroscopic compression of the organ's curved surface can all cause visual distortion in the images, affecting the surgeon's judgment of the incision location and flap symmetry.

[0003] In intraoperative image application scenarios for delicate procedures such as hypospadias surgery in children, existing image recognition-assisted localization methods usually adopt the method of frame-by-frame recognition, directly superimposing key points or auxiliary lines in the image coordinate system, and smoothing the output results when necessary; in order to improve cross-frame consistency, some solutions will introduce tracking or simple geometric correction.

[0004] However, the generation and processing of intraoperative images has the following characteristics: most existing intraoperative image recognition methods use frame-by-frame recognition to directly overlay key points, auxiliary lines, or simple smoothing results onto the original image, lacking a specific processing mechanism for the ventral curvature of the penis, the transition shape of the distal glans, and tissue traction displacement during hypospadias surgery in children; especially when the distal end of the catheter is close to the coronal sulcus or glans, the catheter projection is prone to continuous contraction; when the assistant retracts, the instrument supports, or the suture is pulled, the axial reference of the catheter is prone to lateral displacement; if the general image overlay method is still used, the guide line is prone to deviate from the longitudinal direction of the real urethral plate, affecting the judgment of the incision direction, flap alignment, and the direction of new urethral formation;

[0005] Existing technologies generally suffer from the following shortcomings:

[0006] Most solutions fail to extract the catheter's axis and fixed outer diameter separately as a basis for subsequent positioning, making it difficult to form a unified local coordinate and scale benchmark for subsequent positioning output; the use of a uniform superposition strategy under different risk states may result in the output of misleading auxiliary lines in high-risk situations, or the forced output under conditions that cannot be unfolded;

[0007] Existing surface unfolding and refinement methods mostly rely on general mesh optimization or depth information, making it difficult to obtain a stable and consistent mapping under monocular intraoperative video. Even if a few methods attempt to unfold and correct the intraoperative surface, their axis drift correction only adopts a single linear trend correction relationship.

[0008] To address the above problems, this invention proposes a solution. Summary of the Invention

[0009] In order to overcome the above-mentioned defects of the prior art, embodiments of the present invention provide an image recognition-assisted localization method and system for hypospadias surgery in children, so as to solve the problems mentioned in the background art.

[0010] To achieve the above objectives, the present invention provides the following technical solution:

[0011] An image recognition-assisted localization method for pediatric hypospadias surgery includes the following steps: segmenting the ventral penile surgical field region of the surgical field image to obtain a urinary catheter pixel-level probability map and a urinary catheter mask; extracting and sampling the axial reference line of the urinary catheter on the urinary catheter mask; determining the segmentation probability based on the urinary catheter pixel-level probability map values ​​in the neighborhood of each sampling point; determining the observation width along the catheter based on the normal boundary search of the urinary catheter mask; determining the axial bending morphology based on the tangential direction changes of adjacent sampling points; and calculating the urinary catheter reference reliability field and urinary catheter reference availability score accordingly; determining the stable exposed segment of the urinary catheter based on the urinary catheter reference reliability field and determining the intraoperative constant diameter benchmark of the urinary catheter; generating a surgical field anatomical segment label sequence based on the deviation and change trend of the observation width along the catheter relative to the intraoperative constant diameter benchmark of the urinary catheter; and constructing an initial unfolding mapping with the urinary catheter axial reference line as the virtual anatomical axis.

[0012] The catheter reference availability score is compared with the preset reference availability threshold. If it is lower than the preset reference availability threshold, it is determined to be in an unexpandable and downgraded state, and the downgrade prompt is output before the frame ends. If the preset reference availability threshold is reached, the surgical field surface distortion risk score is calculated based on the consistency of the observed width along the route with the constant diameter reference of the catheter during the operation. The processing status is determined to be either penile in situ guided state or penile surface unfolding and correction state based on the comparison result with the preset surface risk threshold.

[0013] In the penile in-situ guided state, an intraoperative auxiliary positioning and guidance path is generated and displayed in the original image coordinate system; in the penile curved surface unfolded and corrected state, the initial unfolding mapping is corrected by lateral scale of the partition according to the anatomical segment label sequence of the surgical field, and tissue traction offset correction is selectively performed according to the correction magnitude and catheter reference availability score to obtain the final refined unfolding mapping. An intraoperative auxiliary positioning and guidance path is generated in the unfolding domain, and the final refined unfolding mapping is inversely mapped back to the original image coordinate system for display.

[0014] In a preferred embodiment, the ventral penile surgical field region of the surgical field image is segmented into a catheter to obtain a catheter pixel-level probability map and a catheter mask. This includes: segmenting the ventral penile surgical field region into a catheter pixel-level probability map; performing thresholding, denoising, and connected component filtering on the catheter pixel-level probability map to obtain candidate masks for a pre-set standard-grade outer diameter catheter; extracting the skeleton of each candidate mask to form a candidate centerline; sampling along the candidate centerline and measuring the local width at each sampling point; and calculating the average width of each candidate mask. Width variation coefficient and continuous effective length; based on preset catheter specification information and imaging scale information, determine the gradation reference width range in the current frame; determine the candidate mask whose average width falls within the gradation reference width range, whose width variation coefficient is not higher than the preset constant diameter stability threshold, and whose continuous effective length is not lower than the preset minimum length threshold as the effective candidate mask; select the one with the highest comprehensive score among the effective candidate masks as the catheter mask, and extract the skeleton on the catheter mask to obtain the catheter axial reference line, and sample along the catheter axial reference line at a preset step size to obtain the sampling point set.

[0015] In a preferred embodiment, the specific method for determining the segmentation probability, along-path observation width, and axial bending morphology of each sampling point is as follows: the segmentation probability is the average probability value of the catheter pixel-level probability map in the neighborhood of the corresponding sampling point; the along-path observation width is determined by the normal direction based on the local tangential direction of the catheter axial reference line at the corresponding sampling point, and the boundary of the mask is searched on both sides along the normal direction, with the distance between the two boundaries as the along-path observation width at the corresponding sampling point; the axial bending morphology is obtained based on the deflection angle of the tangential direction of the corresponding sampling point relative to the tangential direction of the adjacent sampling points.

[0016] The calculation of the catheter-related reference reliability field and catheter-related reference availability score includes: determining the visibility clarity along the catheter line based on segmentation probability; determining the catheter exposure stability based on local fluctuations in the observation width along the line; and determining the catheter-related abnormal bending penalty based on axial bending morphology. The visibility clarity, catheter exposure stability, and catheter-related abnormal bending penalty are combined to obtain the catheter-related reference reliability field. Sampling points in the catheter-related reference reliability field that are not lower than a preset confidence threshold are identified as high-reliability sampling points. The proportion of high-reliability sampling points to all sampling points is calculated to obtain the high-confidence coverage rate. The length of the longest continuous high-reliability exposed segment that continuously meets the preset confidence threshold along the catheter axis is calculated to obtain the longest continuous high-reliability exposed segment length. The high-reliability coverage rate and the longest continuous high-reliability exposed segment length are normalized and combined to obtain the catheter-related reference availability score. The sampling interval corresponding to the longest continuous high-reliability exposed segment is determined as the stable exposed segment of the catheter.

[0017] In a preferred embodiment, a surgical field anatomical segment label sequence is generated, and an initial unfolding mapping is constructed using the axial reference line of the urinary catheter as a virtual anatomical axis. Specifically, the intraoperative constant diameter benchmark of the urinary catheter is determined based on the stable exposed segment of the urinary catheter, which is the reference lateral pixel scale of the target urinary catheter at the current imaging scale. The constant diameter deviation at each sampling point is determined based on the difference between the observed width along the route and the intraoperative constant diameter benchmark of the urinary catheter. The changing trend of the constant diameter deviation along the axial reference line of the urinary catheter is calculated, and the sampling interval is divided into segments according to the constant diameter deviation and its changing trend along the axial direction of the urinary catheter. Among them, the continuous interval where the constant diameter deviation is continuously within a preset stable range and the width change gradient is continuously lower than a preset gradient threshold is marked as the ventral columnar maintenance segment of the penile body; the continuous interval where the width change gradient is continuously negative and its absolute value continuously exceeds the preset gradient threshold or the constant diameter deviation continuously exceeds the preset contraction threshold is marked as the coronal sulcus glans transition contraction segment.

[0018] The cumulative arc length of the axial reference line of the urinary catheter is used as the longitudinal coordinate of the unfolded domain, and the signed normal distance from any pixel point in the ventral surgical field of the penis to the projection point of the axial reference line of the urinary catheter is used as the lateral coordinate of the unfolded domain to construct the initial unfolded mapping; wherein, the signed normal distance is positive in the direction from the axial reference line of the urinary catheter to the ventral side of the penis and negative in the direction from the dorsal side.

[0019] In a preferred embodiment, a surgical field surface distortion risk score is calculated, and the processing state is determined to be either penile in situ guided state or penile surface unfolding correction state based on the comparison result with a preset surface risk threshold. This includes: calculating the overall constant diameter deviation component based on the overall deviation of the observed width along the path at a high-reliability sampling point relative to the intraoperative constant diameter reference of the catheter; calculating the distal transition distortion enhancement component based on the continuous contraction intensity of the observed width along the path within the glans penis transition contraction segment, and in combination with the catheter reference reliability field at the corresponding sampling point; weighting and combining the overall constant diameter deviation component and the distal transition distortion enhancement component to obtain the surgical field surface distortion risk score; comparing the surgical field surface distortion risk score with a preset surface risk threshold; when the surgical field surface distortion risk score is not greater than the preset surface risk threshold, the processing state is determined to be penile in situ guided state; when the surgical field surface distortion risk score is greater than the preset surface risk threshold, the processing state is determined to be penile surface unfolding correction state.

[0020] In a preferred embodiment, performing lateral scale correction on the initial unfolding mapping includes: mapping the sampling points of the catheter axial reference line to the unfolding domain via the initial unfolding mapping to obtain the coordinate sequence of the sampling points in the unfolding domain; and determining the ventral columnar maintenance segment of the penile body and the coronal sulcus glans transitional contraction segment in the longitudinal coordinate direction of the unfolding domain according to the surgical field anatomical segment label sequence.

[0021] For the sampling points in the columnar maintenance segment of the penile body, a control point sequence of the columnar maintenance segment of the penile body is constructed based on the constant diameter deviation, and a linear transverse correction relationship suitable for the relatively regular shape of the main trunk of the penile body is established.

[0022] For the sampling points in the transitional contraction segment of the coronal sulcus and glans penis, a control point sequence for the transitional contraction segment of the coronal sulcus and glans penis is constructed based on the constant diameter deviation, and a unidirectional nonlinear transverse correction relationship applicable to the shape changes of the coronal sulcus and the distal transitional part of the glans penis is established.

[0023] A smooth transition is performed between the linear lateral correction relation and the unidirectional nonlinear lateral correction relation at the segment boundary to obtain the global lateral scale correction relation; the global lateral scale correction relation is used to perform scale correction on the lateral coordinates of the expansion domain to obtain the first-stage refined expansion mapping.

[0024] In a preferred embodiment, determining whether to perform tissue traction offset correction includes: determining a lateral correction intensity score based on the degree of deviation of the global lateral scale correction relationship relative to the unit scale; combining the lateral correction intensity score with the catheter reference available score to obtain a two-stage offset correction trigger score; comparing the two-stage offset correction trigger score with a preset two-stage trigger threshold; if the two-stage offset correction trigger score is greater than or equal to the preset two-stage trigger threshold, then determining to perform tissue traction offset correction; if the two-stage offset correction trigger score is less than the preset two-stage trigger threshold, then determining not to perform tissue traction offset correction, and determining the one-stage refined unfolding mapping as the final refined unfolding mapping.

[0025] In a preferred embodiment, performing tissue traction offset correction includes: taking the axial reference line sampling points of the urinary catheter within the stable exposed segment of the urinary catheter that meet the preset confidence threshold conditions as fitting samples, and mapping the fitting samples to the expanded domain after a one-stage refinement and expansion to obtain a set of sample points with longitudinal and lateral coordinates.

[0026] Using the longitudinal coordinates of the expanded domain as the independent variable and the lateral coordinates of the sample points as the dependent variable, a tissue traction offset correction relationship for the lateral offset trend of the catheter axial reference line was established. Based on the differential deformation characteristics of the ventral columnar maintenance segment of the penile body and the glans transitional contraction segment of the coronal sulcus, which are divided by the surgical field anatomical segment label sequence, linear trend correction relationships and curvilinear trend correction relationships were used for weighted fitting according to reliability. The fitting weight was taken as the reference reliability field along the catheter line at the corresponding sampling point, thus obtaining the tissue traction offset correction relationship. Based on the tissue traction offset correction relationship, the lateral coordinates of the expanded domain obtained by the first-stage refined expanded mapping of any pixel point in the ventral surgical field area of ​​the penis were subtracted from the lateral offset value obtained by solving the tissue traction offset correction relationship at the corresponding longitudinal coordinate of the pixel point, thus obtaining the second-stage refined expanded mapping, which was used as the final refined expanded mapping.

[0027] In a preferred embodiment, generating an intraoperative auxiliary positioning and guidance path specifically includes: when the penis is in the in-situ guided state, taking the axial reference line of the urinary catheter as a virtual anatomical axis reference in the original image coordinate system, taking the arc length parameter interval corresponding to the stable exposed segment of the urinary catheter as the intraoperative auxiliary positioning and guidance path generation interval, sampling along the intraoperative auxiliary positioning and guidance path generation interval according to a preset step size to generate an intraoperative auxiliary positioning and guidance path point sequence, and superimposing it on the surgical field image;

[0028] When the penile curvature is in the corrected state and the final refined development mapping is a two-stage refined development mapping, the longitudinal coordinate interval corresponding to the stable exposed segment of the catheter is used as the basis in the development domain. The longitudinal range of the complete intraoperative auxiliary positioning and guidance path is expanded to both ends with a preset margin to obtain the complete intraoperative auxiliary positioning and guidance path point sequence. The complete intraoperative auxiliary positioning and guidance path point sequence is generated and displayed by back-projection to the original image coordinate system through the inverse mapping of the two-stage refined development mapping.

[0029] When the penile curvature is in the corrected state and the final refined unfolding mapping is a one-stage refined unfolding mapping, the conservative intraoperative auxiliary positioning guidance path point sequence is generated only in the longitudinal coordinate interval corresponding to the stable exposed segment of the catheter, and an error band is generated based on the intraoperative constant diameter benchmark of the catheter. The conservative intraoperative auxiliary positioning guidance path point sequence and the error band are then back-projected to the original image coordinate system through the inverse mapping of the one-stage refined unfolding mapping.

[0030] The image recognition-assisted localization system for pediatric hypospadias surgery includes the following modules: a catheter reference module for acquiring surgical field images and determining the ventral penile surgical field region, segmenting the catheter to obtain a catheter pixel-level probability map and a catheter mask, extracting the catheter axial reference line, and calculating the catheter reference reliability field along the line, the catheter reference availability score, the stable exposed segment of the catheter, the intraoperative constant diameter benchmark of the catheter, the surgical field anatomical segment label sequence, and the initial unfolding mapping;

[0031] The surface distortion determination module is used to compare the catheter reference availability score with the preset reference availability threshold. When the catheter reference availability score is lower than the preset reference availability threshold, the current processing state is determined to be the non-expandable degraded state. When the catheter reference availability score reaches the preset reference availability threshold, the surgical field surface distortion risk score is calculated, and the current processing state is determined to be the penile in situ guided state or the penile surface unfolding correction state.

[0032] The unfolding and refinement module is used to perform lateral scale correction on the initial unfolding map under the penile curvature unfolding correction state, and to determine whether to perform tissue traction offset correction, so as to obtain the final refined unfolding map;

[0033] The intraoperative auxiliary positioning guidance path rewrite module is used to generate downgrade prompts or intraoperative auxiliary positioning guidance paths according to the current processing status, and to project the intraoperative auxiliary positioning guidance paths in the unfolded domain back to the original image coordinate system for superposition display under the penile curvature unfolding correction state through the inverse mapping of the final refined unfolding mapping.

[0034] The technical effects and advantages of this invention's image recognition-assisted localization method for pediatric hypospadias surgery are as follows: By detecting a standard-sized catheter inserted into the child's urethra during surgery, this invention can stably establish an axial reference consistent with the anatomical orientation during hypospadias surgery. It also utilizes the standard outer diameter of the catheter to form a constant diameter benchmark during surgery, avoiding inaccuracies in simple pixel measurements caused by microscope magnification or camera scale changes. Furthermore, by identifying the columnar maintenance segment on the ventral side of the penile shaft and the transitional contraction segment of the glans penis in the coronal sulcus, the ventral surgical field surface of the penis is divided and corrected in sections, and further... It corrects the lateral displacement caused by tissue retraction, suture traction, and instrument support, making the output intraoperative auxiliary positioning guidance path more closely match the true longitudinal direction of the urethral plate in pediatric hypospadias surgery; by determining the treatment status (penile in situ guidance state, penile curvature unfolding correction state, or non-unfolding degraded state) based on the catheter reference availability score and surgical field curvature distortion risk score, it can output a complete guidance path, a conservative guidance path, or a degraded prompt under different surgical field conditions, which is more suitable for longitudinal positioning of the urethral plate, judgment of incision direction, flap alignment, and guidance of the new urethral reconstruction direction. Attached Figure Description

[0035] Figure 1 This is a schematic diagram of the image recognition-assisted localization method for hypospadias surgery in children according to the present invention;

[0036] Figure 2 This is a schematic diagram illustrating the reference availability score and stable segment determination of the urinary catheter of the present invention;

[0037] Figure 3 This is a schematic diagram of the initial expansion mapping and expansion domain coordinates of the present invention;

[0038] Figure 4 This is a schematic diagram of the image recognition-assisted positioning system for hypospadias surgery in children according to the present invention. Detailed Implementation

[0039] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0040] Example

[0041] Please see Figure 1 As shown, this invention discloses an image recognition-assisted localization method for hypospadias surgery in children, comprising the following steps:

[0042] Please see Figure 2 As shown, step one: Obtain the intraoperative pre-placed urinary catheter within the ventral surgical field region of the penis in the current frame of the surgical field image, and construct the basic quantities required for subsequent operations; specifically, obtain the urinary catheter pixel-level probability map through a segmentation network, and output the urinary catheter mask and failure flag after thresholding, denoising, and connected component filtering; extract the axial reference line of the urinary catheter on the mask and sample it according to the step size, calculate the observation width along the path and the visibility clarity along the urinary catheter line, and then form the reference reliability field along the urinary catheter line, obtain the urinary catheter reference availability score, and determine the stable exposed section of the urinary catheter; determine the intraoperative constant diameter benchmark of the urinary catheter based on the stable exposed section of the urinary catheter, and establish local coordinates based on the center line to generate the initial unfolding mapping;

[0043] In this embodiment, the method is implemented in an intraoperative video acquisition and display system, with the input being the current frame surgical field image of a real-time video stream. Where t is the frame number and its corresponding ventral penile surgical field region. ;Specifically:

[0044] Get the current frame field image Determine the ventral surgical field area of ​​the penis And scale to the preset input size. We obtain the network input tensor X; where, The input width is the number of pixels. Input the height in pixels; for example, 512×512 or 640×640, and record the back-mapping parameters. , represents the scaling ratio and offset of returning from the network input coordinates to the original ROI coordinates of the ventral surgical field of the penis; channel normalization is performed on the network input tensor X, for example, by subtracting the mean and dividing by the standard deviation. The mean and standard deviation can be common public values ​​or statistics from the training set.

[0045] The input tensor X is input into an encoder-decoder type catheter segmentation network, which outputs a pixel-level probability map of the catheter. , representing the probability that pixel (x, y) belongs to the catheter region; pixel-level probability map of the catheter. Thresholding is performed to obtain candidate binary masks: ;in, For indicator functions, The preset segmentation threshold is used to convert the probability map into a binary mask. The initial value can be 0.5, and it can be adjusted within the range of 0.3 to 0.7 according to the noise level of the surgical field.

[0046] It should be noted that the encoder-decoder type catheter segmentation network of this invention uses the U-Net++ encoder-decoder segmentation network to achieve pixel-level segmentation of the catheter. The network input is a 512×512 pixel surgical field image ROI region, and the output is a pixel-level probability map of the catheter. The network training dataset contains intraoperative surgical field images of 1000 cases of hypospadias surgery in children, covering three sizes of catheters: Fr6, Fr8, and Fr10. The image annotation uses polygon annotation of the catheter region. The training set, validation set, and test set are divided in a 7:2:1 ratio. The network achieves a segmentation precision ≥95%, a recall ≥94%, and an intersection-over-union ratio ≥93% on the test set.

[0047] For candidate binary masks Perform morphological opening and closing operations to denoise and obtain a denoising mask. Wherein, the radius of the structuring element is denoted as The initial value can be 2 to 5 pixels. The radius of the structural element is on the same order of magnitude as the typical width of the catheter but slightly smaller. It is used to remove small noise without damaging the slender body.

[0048] It should be noted that the intraoperative pre-placed urinary catheter described in this invention specifically refers to a standard-grade medical urinary catheter placed inside the urethra of the child to support the newly formed urethra, preferably a French specification urinary catheter, such as Fr6, Fr8 or Fr10, with corresponding physical outer diameters of 2.0 mm, 2.7 mm and 3.3 mm, respectively.

[0049] For noise reduction mask Extract candidate region set For each candidate region First, extract its skeleton and form candidate centerlines. Where s is the arc length parameter; then, sampling is performed along the candidate centerline at a fixed step size to obtain the candidate sampling point set. And k is the candidate sampling point number; at each candidate sampling point, the normal direction is determined according to the tangent of the candidate center line, and the candidate mask boundary is searched along the normal direction to obtain the local width of the candidate region at the corresponding sampling point. Based on this, the average width of each candidate region is calculated. Width variation coefficient and continuous effective length , where the average width Width variation coefficient ;in, Candidate region The number of sampling points Indicates standard deviation, To prevent extremely small constants from having a denominator of zero, and to prevent the denominator from being zero and to improve computational stability, the same symbols in the following text have the same meaning and will not be repeated.

[0050] The system pre-configures the specifications and imaging scale information of the urinary catheter used in the current surgery. The imaging scale information can be directly obtained from the magnification parameters output by the intraoperative microscope or camera device, or it can be obtained by the system during preoperative calibration. In the continuous processing of adjacent frames, the intraoperative constant diameter benchmark of the urinary catheter confirmed in the previous frame can be used for approximate inheritance in the current frame. Based on the specifications and imaging scale information, the standard outer diameter of the urinary catheter is mapped to the gradation reference width range of the current frame. For example, when using an Fr8 specification catheter and the current imaging scale is known, its reference pixel width range in the current frame can be obtained; a gradation consistency criterion is constructed for each candidate region: ;in, This is the reference pixel width for this specification of urinary catheter in the current frame;

[0051] Furthermore, a candidate region is deemed a valid catheter candidate region only if it simultaneously meets the following conditions: average width Falling within the aforementioned gradation reference width range; width variation coefficient Not higher than the preset constant diameter stability threshold Continuous effective length Not less than the preset minimum length threshold The initial value can be set to a certain multiple of the width of the ROI in the ventral surgical field of the penis, such as 0.3 times. Among the candidate regions that meet the above conditions, the one with the highest comprehensive score is selected as the target region for the catheter in the current frame, where the comprehensive score can be expressed as: ;in, The weights are non-negative, and This represents the longitudinal dimension of the ventral surgical field region of the penis; if no candidate region meets the above conditions, the catheter reference for the current frame is deemed unusable, and a failure flag is output. And make the current frame catheter mask If empty; if a candidate region that meets the conditions exists, output a failure flag. Output catheter mask And backmap it to the coordinate system of the ventral surgical field region of the penis in the current frame through the backmapping parameters;

[0052] Obtaining a catheter mask Next, the urinary catheter is skeletonized to extract the axial reference line C(s). Specifically, endpoint detection is performed on the mask skeleton pixels, and the farthest endpoint pair is used as the start and end endpoints of the centerline tracing. An ordered centerline point sequence is obtained by tracing along the skeleton topology, and then reparameterized according to the cumulative arc length to obtain the axial reference line C(s) of the urinary catheter. Then, the reference line is obtained by using a fixed arc length step. Sample the centerline to form a set of sampling points. ;

[0053] At each sampling point, the normal direction is obtained based on the local tangential direction of the centerline. The catheter mask boundary is then searched along both sides of the normal direction to obtain the observed width along the path. If the boundary search fails or the difference between the two sides exceeds the preset symmetry threshold... If the sampling point is not detected, its visibility will be reduced; where a preset symmetry threshold is used. To avoid unilateral boundary drift, the initial value can be taken as 0.3 to 0.5, representing the proportion of the left-right width difference to the total width; based on the pixel-level probability map of the catheter output by the segmentation network. The average probability is calculated within the neighborhood of each sampling point to obtain the visibility of the catheter along its path. Based on the observation width along the path within the neighborhood window of the sampling point, the median absolute deviation is calculated and normalized to obtain the stability of catheter exposure. The neighborhood window is centered on the sampling point of the catheter axial reference line. Pixel window, for example ; Calculate the penalty for abnormal bending of the urinary catheter based on the tangential direction change of adjacent sampling points Therefore, a reference reliability field is constructed along the catheter. ;in, This indicates that the results are limited to between zero and one; the catheter-as-a-line reference reliability field characterizes the clarity and stability of the catheter as an intraoperative geometric reference at different positions along its axis in the ventral surgical field of the penis; the catheter reference availability score characterizes whether the catheter as a whole in the current frame is sufficient to support subsequent longitudinal positioning of the urethral plate, determination of the cutting direction, and unfolding correction processing; based on the catheter-as-a-line reference reliability field, the catheter reference availability score is further calculated, specifically, a score not lower than a preset confidence threshold is selected. The sampling points are identified as high-reliability sampling points, forming a high-reliability sampling point set. The initial preset confidence threshold for high-reliability sampling points is 0.6. The proportion of the high-reliability sampling point set to all sampling points is calculated to obtain the high-reliability coverage. ; and calculate the length of the longest continuous high-reliability exposed segment that continuously satisfies the high-reliability condition along the centerline direction, thus obtaining the length of the longest continuous high-reliability exposed segment. The high-reliability coverage rate and the longest continuous high-reliability exposure length are normalized and then combined to obtain the catheter reference availability score: ;in, and These represent the normalized high-reliability coverage and the longest continuous high-reliability exposed segment length, respectively. The preset combination weights are used; at the same time, the sampling interval corresponding to the longest continuous high-reliability exposure segment is determined as the stable exposure segment of the urinary catheter.

[0054] After obtaining a stable exposed segment of the urinary catheter, calculate the baseline constant diameter of the urinary catheter during the procedure. The constant diameter reference during catheterization refers to the reference horizontal pixel width of the standard-grade catheter used in the current surgery at the current frame imaging scale; if the system knows the current specifications and imaging scale, the constant diameter reference during catheterization... The conversion is obtained by altering the physical outer diameter of the urinary catheter to the imaging scale. In this embodiment, the conversion formula is as follows: ,in The physical outer diameter of the urinary catheter. M represents the pixel density of the imaging device, and M represents the magnification of the surgical microscope. If the imaging device lacks pixel density calibration, it can be obtained using a preoperative calibration ruler. The calibration ruler has an accuracy of 0.01 mm;

[0055] If the current frame lacks explicit imaging scale information, the observation width along the path of highly reliable sampling points within the stable exposed segment of the catheter can be preferentially selected, and the median of these widths can be used as the intraoperative constant diameter reference for the current frame. After obtaining the intraoperative constant diameter reference, segment labels are further generated based on the observation width sequence along the path. Specifically, the constant diameter deviation is defined. Simultaneously calculate the width gradient. When within a certain continuous interval Continuously below the preset stability threshold The initial value is 0.1, and Continuously below the preset gradient threshold Initially, the value is 0.5, and this continuous interval is marked as the ventral columnar maintenance segment of the penile body; when a certain continuous interval... If the value remains negative and its absolute value continuously exceeds the gradient threshold, or corresponds to a constant path deviation... Continuously exceeding the preset contraction threshold Initially, the value was 0.2, and this continuous interval was marked as the coronal sulcus glans transitional contraction segment; the ventral columnar maintenance segment of the penile body corresponds to the surgical field interval in pediatric hypospadias surgery where the ventral main shaft of the penile body is exposed in a relatively regular manner and the projection width of the catheter changes little; the coronal sulcus glans transitional contraction segment corresponds to the surgical field interval where the projection width continuously shrinks due to the transition of local anatomical shape and changes in the curvature of the surgical field when the catheter approaches the coronal sulcus, glans, or distal urethral reconstruction area; thus, a surgical field anatomical segment label sequence arranged along the axial reference line of the catheter is obtained. ;

[0056] Please see Figure 3 As shown, an initial unfolding map is constructed based on the axial reference line C(s) of the urinary catheter. For any pixel (x, y) within the ventral surgical field region of the penis, its nearest projection point is found on the axial reference line C(s) of the urinary catheter. The longitudinal coordinate *u* of the expanded domain is defined as the cumulative arc length from the starting point of the axial reference line of the catheter to the projection point. The lateral coordinate *v* of the expanded domain is defined as the signed normal distance from the pixel to the projection point. The signed normal distance is positive in the direction from the axial reference line of the catheter towards the ventral side of the penis and negative in the direction towards the dorsal side. This yields the initial expanded mapping. The purpose of this initial unfolding mapping is to use the axial reference line of the catheter as a virtual anatomical axis to unify the curved projections corresponding to the relatively regular area of ​​the ventral main trunk of the penis and the coronal sulcus and the distal transition area of ​​the glans in pediatric hypospadias surgery into a two-dimensional coordinate system unfolded along the axial direction of the catheter, providing a basis for subsequent generation and position correction of guide lines around the longitudinal direction of the urethral plate.

[0057] Step Two: Based on the catheter reference availability score, determine whether the reference meets the minimum conditions. If not, enter the non-expandable degraded state and write the degraded output flag; if it meets the conditions, calculate the surgical field surface distortion risk score, and then compare it with the surface risk threshold to determine whether to enter the penile in situ guided state or the penile surface unfolding correction state, and write the processing status, output flag, and key decision quantities into the status cache; specifically:

[0058] In this embodiment, based on the failure flag Available scores for reference with urinary catheter Determine whether the reference meets the minimum conditions for subsequent processing; if Then directly set the processing state of this frame. It is a non-expandable degraded state; if Then the urinary catheter will be used according to the availability score. Compared with the preset reference available threshold In comparison, among them The initial value can be 0.4 to 0.6. During deployment, it is calibrated by an offline validation set to ensure that intraoperative auxiliary positioning guidance path results are not output when there is insufficient reference.

[0059] when Set the processing state for this frame. It is a non-expandable degraded state; when Enter the surface risk assessment at that time;

[0060] Once the surgical field is determined to be in the curved surface risk assessment stage, the surgical field curved surface distortion risk score is further calculated. It is used to characterize whether there are significant non-planar projections or non-affine changes within the ROI, thereby determining whether it is necessary to enter the surface expansion branch;

[0061] Specifically, in pediatric hypospadias surgery, the aforementioned non-planar projection changes mainly stem from the columnar anatomical morphology of the ventral trunk of the penis and the transition in shape from the coronal sulcus to the distal glans. When the surgical camera is in a lateral or oblique view, or when there is retraction or instrument support during the operation, the surgical field surface will undergo perspective compression and local deformation on the two-dimensional image, causing the longitudinal reference of the urethral plate, the planned incision direction, or the flap alignment reference, which should originally be aligned straight along the catheter axis, to appear bent or offset on the screen; the aforementioned surgical field surface distortion risk score Used to identify whether the current surgical field has deviated from the conditions suitable for direct in-situ superposition guidance. When the degree of deviation exceeds the threshold, penile surface unfolding correction processing is triggered.

[0062] Surgical field surface distortion risk score It consists of a global constant diameter deviation component and a distal transition distortion enhancement component; among which, the global constant diameter deviation component This is used to characterize the deviation of the overall imaging width of the urinary catheter in the current frame from the constant diameter baseline during catheterization, and is expressed as: ;in, Represents the set of highly reliable sampling points. This indicates the number of its elements; the larger this component is, the more the overall projection of the catheter deviates from the ideal constant diameter state, and the more obvious the overall viewing angle tilt or curved projection compression exists in the current field of view; distal transition distortion enhancement component. This is used to characterize the local sustained contraction intensity of the urinary catheter within the transitional contraction segment of the glans penis in the coronal sulcus; specifically, from the surgical field anatomical segment label sequence obtained in step one, a set of sampling points marked as the transitional contraction segment of the glans penis in the coronal sulcus is extracted. and calculate If the current frame does not detect the transitional contraction segment of the glans penis at the coronal sulcus, then... The larger this component is, the more significant the contraction trend of the catheter at the distal end of the surgical field, and the more obvious the influence of the distal curved perspective changes on the current image.

[0063] The overall constant diameter deviation component and the distal transition distortion enhancement component are weighted and combined to obtain the surgical field surface distortion risk score. ;in, The weights are non-negative, with initial values ​​α=0.5 and β=0.5, and are obtained from the offline validation set.

[0064] A surgical field surface distortion risk score will be obtained. With preset surface risk threshold The comparison included a preset surface risk threshold. The initial value can be 0.5, and it is calibrated by the offline validation set during deployment to balance the frequency of entering the expanded branch with the benefit;

[0065] when This indicates that the overall diameter of the catheter in the current frame is relatively consistent. Although there are slight perspective changes in the catheter reference, they have not yet reached the point where detailed processing of the expanded domain is required. Therefore, the processing state for this frame is set. The penis is in situ guided.

[0066] when This indicates that the catheter in the current frame has shown a significant overall deviation from its constant diameter or distal constriction enhancement, and the current surgical field is strongly affected by cylindrical or transitional curved surface projections. The processing state for this frame is then set. Corrected state for penile curvature unfolding;

[0067] Optionally, to avoid frequent state switching caused by jitter near the threshold, in one embodiment, the processing state is... Introducing hysteresis gating based on the processing state of the previous frame. With the surgical field surface distortion risk score of this frame The switching conditions are jointly determined, specifically: a higher threshold is used when entering the penile curvature unfolding correction state. Retracting the penis to its original guided state using a lower threshold ,in Regardless of whether hysteresis gating is used, step two writes the processing status of this frame, the catheter reference availability score, and the surgical field surface distortion risk score into the status cache.

[0068] Step 3: Under the penile curvature unfolding correction state, the initial unfolding mapping is refined by dividing it into sections using the constant diameter deviation of the catheter and the segment labels. First, the lateral scale correction is performed, and then the tissue traction offset correction is performed based on the correction intensity to obtain the final refined unfolding mapping.

[0069] This step is only used to determine the processing status. When the penis is in the corrected unfolded state, a targeted and detailed analysis of the initial unfolding map is performed, including: calling the initial unfolding map output in step one. Set of sampling points for the axial reference line of the urinary catheter Mapping to the expanded domain yields the coordinate sequence of the sampling points within the expanded domain. ,in, The monotonic mapping result corresponding to the centerline arc length parameter, therefore the surgical field anatomical segment label sequence It can be directly converted into a sequence of surgical field anatomical segment labels along the longitudinal coordinate direction of the expanded domain. ;

[0070] Subsequently, the constant diameter benchmark during catheterization was used. For reference, the constant diameter deviation at each sampling point was... For the expanded domain modeling, the sampling points marked as the ventral columnar maintenance segment of the penile body are considered to primarily reflect the lateral scale changes caused by the overall viewing angle tilt or slight curved surface projection under the relatively regular shape of the ventral main trunk of the penile body. Therefore, a linear correction model is adopted. Specifically, the sampling points in the ventral columnar maintenance segment of the penile body are classified into several control intervals along the longitudinal coordinate u of the expanded domain. Within each control interval, the constant diameter deviation is... The median was used to obtain the control point sequence of the columnar maintenance segment on the ventral side of the penile shaft. Where j is the control interval number; based on this, the weighted least squares method is used to fit the linear correction relationship of the ventral columnar maintenance segment of the penile body. The fitting weights are taken from the reference reliability field along the catheter at the corresponding sampling points. The fitting results are then cropped within a preset cropping scale. Inside; among them, Preset the cropping boundaries to, for example, 0.7 and 1.3, to avoid over-scaling caused by outliers;

[0071] For sampling points marked as the transitional contraction segment of the coronal sulcus and glans penis, it is assumed that their width changes mainly reflect the projected contraction caused by rapid changes in local anatomical shape when the catheter approaches the coronal sulcus and distal glans penis. Therefore, a monotonic nonlinear correction model is adopted. Specifically, the sampling points in the transitional contraction segment of the coronal sulcus and glans penis are grouped into denser control intervals along the longitudinal coordinate u of the expanded domain. Within each control interval, the constant diameter deviation is adjusted. Taking the median, the control point sequence of the coronal sulcus glans transitional contraction segment was obtained. Subsequently, monotonic constraints were applied to the control point sequence, and monotonic cubic interpolation was used to generate a nonlinear correction relationship for the coronal sulcus glans transition contraction segment: ;in, This indicates a monotonic cubic interpolation operation, and the interpolation result is also clipped within the preset clipping scale. Inside;

[0072] At the junction of the columnar maintenance segment on the ventral side of the penile shaft and the transitional contraction segment of the glans penis in the coronal sulcus, to avoid abrupt changes in the correction relationship, within the adjacent area of ​​the junction... and A smooth transition is performed to establish a global lateral scale correction relation g(u). Therefore, when the observed width of the catheter at a certain longitudinal position is greater than the intraoperative constant diameter reference, the corresponding correction relation value is less than one, indicating a contraction of the lateral coordinates of the unfolded domain; when the observed width of the catheter is less than the intraoperative constant diameter reference, the corresponding correction relation value is greater than one, indicating an expansion of the lateral coordinates of the unfolded domain. For any pixel point, an initial unfolding mapping is performed... The obtained expanded domain coordinates (u,v) are then subjected to lateral scale correction: Thus, a first-stage refined expansion mapping is obtained. ;

[0073] After completing the first-stage lateral scale correction, it is further determined whether a second-stage tissue traction offset correction is needed. Specifically, the degree of deviation of the lateral scale correction relationship from the unit scale is statistically analyzed to obtain a lateral correction strength score. ;in, This represents median operations;

[0074] Furthermore, the lateral correction strength score was compared with the catheter reference availability score. The combined scores yield a two-stage offset correction trigger score, which is used to assess whether organ curvature distortion and tissue traction offset in the current surgical field have reached a level requiring further axial correction; for example, the lateral correction intensity score... Available scores for reference with urinary catheter The result of the multiplication is defined as the two-stage offset correction triggered score. The two-stage offset correction trigger score is compared with the preset two-stage trigger threshold. In comparison, the preset two-stage trigger threshold Its initial value can be 0.1 to 0.2, and the offline verification set is calibrated to control false triggering:

[0075] when It was believed that the horizontal scale correction was sufficient to skip the two-stage correction, allowing the two-stage detailed expansion mapping to proceed. It also outputs a flag indicating that the second phase was not executed.

[0076] when Then it will enter the second phase of correction;

[0077] When determining whether to enter the second-stage correction, sampling points of the axial reference line of the urinary catheter that meet the high-confidence condition are selected from the stable exposed segment of the urinary catheter as fitting samples, and these samples are then refined and mapped in the first stage. Mapping to the expanded domain yields a point set. ,Right now ;in, For the sample values ​​of the vertical coordinates of the expanded domain, These are the sample values ​​of the horizontal coordinates of the expanded domain after one-stage correction;

[0078] Ideally, the axial reference line of the catheter should be as parallel to the longitudinal axis as possible in the unfolding field. Therefore, the lateral drift of the centerline in the unfolding field can be regarded as a longitudinal coordinate relationship. Based on the differential deformation characteristics of the ventral columnar maintenance segment of the penile body and the glans penis transitional contraction segment in the coronal sulcus, which are divided by the surgical field anatomical segment label sequence, a regional tissue traction offset correction relationship is constructed: For the area labeled as the ventral columnar maintenance segment of the penile body, a linear trend correction relationship is used. ,in, The linear trend term representing the lateral drift of the centerline as a function of the longitudinal coordinate. This indicates the overall horizontal baseline offset.

[0079] For the regions labeled as the coronal sulcus glans transitional contraction segment and the adjacent ventral columnar maintenance segment of the penile shaft, a curve trend correction relationship was applied: ;in, The second-order nonlinear term representing the lateral drift of the centerline. Represents a first-order linear trend term; the and , By analyzing the sample set The reliability-weighted fitting was performed, with the fitting weights taken as the reference reliability fields along the catheter. The fitting objective is to minimize the weighted sum of squared residuals. Constraints were applied to the fitting of the curve trend correction relationship: the fitting range was limited to the transitional contraction segment of the glans penis in the coronal sulcus marked by the surgical field anatomical segment label sequence and the adjacent 5mm ventral columnar maintenance segment of the penile body. Monotonic constraints were applied during fitting to avoid coordinate reversal after correction. The sum of squares of the fitting residuals must be less than or equal to 0.15. Otherwise, a second fitting was performed using highly reliable sampling points within the stable exposure segment of the catheter.

[0080] After obtaining the linear trend correction relationship of the ventral columnar maintenance segment of the penile body and the curvilinear trend correction relationship of the glans penis transitional contraction segment in the coronal sulcus, the two types of models were segmented and called according to the surgical field anatomical segment label sequence, and continuous splicing was performed at the segment boundaries to obtain the partitioned tissue traction offset correction relationship h(u); for the expanded domain coordinates obtained after one stage of refinement and expansion mapping The mathematical expression for deducting the lateral offset at the corresponding longitudinal coordinate based on the traction offset correction relationship of the zoning organization can be as follows: Perform organizational traction offset correction to obtain the unfolded domain coordinates corresponding to the two-stage refined unfolding mapping. .

[0081] Step 4: Generate the corresponding intraoperative auxiliary positioning guidance path based on the processing status and overlay it to form a closed-loop output; in the non-expandable degraded state, no through-type intraoperative auxiliary positioning guidance path is output, only degraded prompts are output and cached; in the penile in-situ guidance state, in the original penile ventral surgical field area, with the internal catheter axial reference line as a reference, generate the intraoperative auxiliary positioning guidance path point sequence in the corresponding interval of the stable exposed segment of the catheter, and can combine it with the intraoperative constant diameter reference of the catheter to generate boundary prompt bands and overlay them; in the penile curvature unfolding correction state, first generate the intraoperative auxiliary positioning guidance path in the unfolding domain. If a two-stage refined unfolding mapping is obtained, expand the corresponding interval of the stable exposed segment of the catheter to generate a complete intraoperative auxiliary positioning guidance path and back-project it. If only a one-stage refined unfolding mapping is available, generate a conservative intraoperative auxiliary positioning guidance path and add an error band before back-projecting it; finally, write the intraoperative auxiliary positioning guidance path type, mapping type and key decision quantity for reuse in adjacent frames.

[0082] In this embodiment, the intraoperative auxiliary positioning guidance path includes the original image domain intraoperative auxiliary positioning guidance path in the penile in situ guidance state, the complete intraoperative auxiliary positioning guidance path in the penile curvature unfolding correction state, and the conservative intraoperative auxiliary positioning guidance path; wherein, the complete intraoperative auxiliary positioning guidance path and the conservative intraoperative auxiliary positioning guidance path are first generated in the unfolding domain, and then projected back to the original image coordinate system for display through the corresponding refined unfolding mapping.

[0083] This step is based on the processing status of the current frame. Intraoperative auxiliary positioning guidance path is generated for auxiliary positioning, and the intraoperative auxiliary positioning guidance path is written back to the original image coordinate system for overlay display. At the same time, the intraoperative auxiliary positioning guidance path type and key decision values ​​are written into the state cache for reuse in adjacent frames. Specifically:

[0084] Based on processing status Generate a corresponding intraoperative auxiliary positioning guidance path, wherein generating the intraoperative auxiliary positioning guidance path includes:

[0085] When processing status When the state is in an unexpandable degradation state, execute the degradation output strategy;

[0086] When processing status When the penis is in situ guided, the intraoperative auxiliary positioning and guidance path generation strategy based on the original image coordinate system is executed.

[0087] When processing status When the penile curvature is in the corrected state, the strategy for generating an intraoperative auxiliary positioning and guidance path for the unfolding domain is executed, and the choice is made between the complete intraoperative auxiliary positioning and guidance path strategy and the conservative intraoperative auxiliary positioning and guidance path strategy according to the two-stage execution marker in step three.

[0088] In this embodiment, when the processing state When the procedure is in a non-expandable, downgraded state, no continuous intraoperative auxiliary positioning guidance path is output; only a downgrade prompt is output. And write a downgrade flag to avoid misleading positioning, the downgrade prompt It is overlaid on the current frame and simultaneously written to the state buffer for reuse in adjacent frames and display consistency;

[0089] In this embodiment, when the processing state When the penis is in situ guided, the intraoperative auxiliary positioning and guidance path generation strategy of the original image coordinate system is executed. That is, the intraoperative auxiliary positioning and guidance path is generated in the original image coordinate system, with the axial reference line C(s) of the catheter as the axis reference, and the stable exposed segment of the catheter is selected. The interval directly used as an intraoperative auxiliary positioning and guidance path generation interval is denoted as the interval representation of the arc length parameter s. ;in, The parameter is the arc length of the starting end of the stable exposed segment of the urinary catheter. The arc length parameter of the stable exposed section of the urinary catheter termination point must meet the following requirements. , This is the interval representation of the stable exposed segment of the urinary catheter within the parameter domain of the arc length parameter s; along the interval The current frame's intraoperative auxiliary positioning and guidance path point sequence is generated by sampling at a preset step size. in, This represents the l-th intraoperative auxiliary positioning guidance path point in the current frame's intraoperative auxiliary positioning guidance path point sequence, and... l is the sequence number of the intraoperative auxiliary positioning and guidance path point, and l increases in the order in which the intraoperative auxiliary positioning and guidance path is generated;

[0090] To reduce display jitter between adjacent frames, the arc length parameter range corresponding to the stable exposure segment of the urinary catheter in the current frame is used. Resample the intraoperative auxiliary positioning and guidance path from the previous frame to obtain the point sequence of the intraoperative auxiliary positioning and guidance path in the current frame. In the previous frame resampling technique with a unified arc length parameter benchmark, the auxiliary positioning and guidance path is... Based on this, the sequence of auxiliary positioning and guidance path points in the current frame is used. Intraoperative auxiliary positioning guidance path with the previous frame resampling The fusion was performed, and the intraoperative auxiliary positioning and guidance path point sequence was adjusted according to the fusion results. The update is performed; the updated l-th intraoperative assisted localization guidance path point can be represented as: ;in, For weighting;

[0091] To provide a width reference, the constant diameter benchmark during catheterization is used. A symmetrical boundary indicator band is constructed in the direction normal to the axial reference line of the urinary catheter, for example, based on the normal offset of the axial reference line of the urinary catheter. Two reference lines, and the updated intraoperative auxiliary positioning guidance path point sequence. They are displayed together on the current frame;

[0092] In this embodiment, when When the penile curvature is in the corrected unfolded state, an intraoperative auxiliary positioning and guidance path generation strategy is implemented in the unfolded domain. That is, the intraoperative auxiliary positioning and guidance path is first generated in the unfolded domain, and then projected back onto the original image coordinate system through the inverse mapping of the corresponding mapping for superposition and display; the catheter is stably exposed. The longitudinal coordinate sample sequence is obtained by mapping the axial reference line sampling points of the internal catheter through step three. The minimum and maximum values ​​are taken as the corresponding longitudinal coordinate intervals of the stable exposed segment of the urinary catheter in the longitudinal coordinate u parameter domain of the unfolded domain. ,in ;

[0093] It should be noted that the inverse mapping is performed within the ROI on... Local search or table lookup is used to achieve this: the pixels in the ROI are pre-refined and expanded through a first-stage mapping process. Or a two-stage refined expansion mapping Calculate its And establish a sparse grid lookup table for a given Obtained by bilinear interpolation in reverse lookup If the value exceeds the valid domain, it will be clipped to the boundary of the valid domain.

[0094] The selection criteria for the complete intraoperative assisted positioning and guidance path strategy are as follows: if step three determines that the second-stage correction has been entered and the second-stage refinement and expansion mapping has been obtained, then the second-stage refinement and expansion mapping has been executed. At that time, a complete intraoperative auxiliary positioning and guidance path strategy should be selected, that is, based on the longitudinal coordinate interval. Based on the pre-set margin, the longitudinal range of the complete intraoperative auxiliary positioning and guidance path is obtained by expanding it to both ends. Within this range, a complete sequence of intraoperative auxiliary positioning and guidance path points is generated using u as the primary parameter. The superscript "full" indicates a complete intraoperative auxiliary positioning and guidance path that spans the preset longitudinal range; the complete intraoperative auxiliary positioning and guidance path point sequence The intraoperative auxiliary positioning and guidance path points are mapped through a two-stage refinement process. The inverse mapping back projection onto the original image coordinate system yields the intraoperative auxiliary positioning guidance path point sequence. And overlaid on the current frame, the complete intraoperative auxiliary positioning guidance path is used to provide continuous positioning guidance throughout the preset range, and in hypospadias surgery, it provides doctors with a longitudinal incision planning reference line that is strictly parallel to the anatomical axis after surface unfolding and tissue traction offset correction;

[0095] It should be noted that the preset margin is used in the stable exposure section of the urinary catheter. Corresponding vertical coordinate range Both ends provide safety extensions to ensure continuous coverage of the complete intraoperative auxiliary positioning and guidance path and prevent interruption of the intraoperative auxiliary positioning and guidance path due to local uncertainties at the boundary of the stable exposed segment of the catheter; the preset margin is denoted as Its value is determined using the constant diameter benchmark used during catheterization. Method for determining dimensionless quantities: Let ,in The preset margin coefficient has an initial value of 2 to 5 and is calibrated using an offline validation set to balance coverage and error risk; in one embodiment, the preset margin coefficient... A value of 3 is chosen, so that the margin length is approximately equal to the longitudinal distance of three constant diameters, used to cover possible short-term obstruction or segmentation confidence fluctuations near the boundary of the stable exposed section of the catheter; furthermore, to avoid over-expansion when the reference quality is insufficient, the preset margin coefficient... The usability score can be referenced along with the urinary catheter. Adaptive adjustment, for example: when Time to take ;when Time to take ;when Time to take ;in The preset reference threshold for step two is used; thus, the longitudinal range of the complete intraoperative assisted positioning guidance path is determined as follows: , ; and will Limited to the valid u-domain of the expandable map in the current frame, i.e., by The reversible mapping covers the longitudinal coordinate range to ensure that the generated complete intraoperative auxiliary positioning guidance path can be backprojected to the original image coordinate system;

[0096] The selection of the conservative intraoperative auxiliary positioning guidance path strategy is based on the following: if the output of step three does not execute the two-stage identification and only obtains the one-stage refined expansion mapping... To avoid long-distance cumulative errors caused by uncorrected shearing, only the section corresponding to the stable exposure segment of the catheter is corrected. The intraoperative auxiliary positioning and guidance path with limited internal generation length is denoted as the conservative intraoperative auxiliary positioning and guidance path point sequence. The superscript "safe" indicates a conservative intraoperative auxiliary positioning and guidance path with limited length; an error band is also generated. Its error band half-width is taken ; Conservative intraoperative assisted localization and guidance path point sequence With error band The mapping is refined in one stage. The inverse mapping is projected back to the original image coordinate system and superimposed for display; the conservative intraoperative auxiliary positioning guidance path is used to provide a more robust positioning prompt range when the reference or expansion refinement is insufficient. In hypospadias surgery, the additional error band clearly informs the doctor of the positioning accuracy boundary of the current intraoperative auxiliary positioning guidance path, avoiding misleading guidance when the surface correction is incomplete.

[0097] Finally, the final output is written back and cached in a unified structure, including the intraoperative auxiliary localization and guidance path point sequence. or downgrade prompt Intraoperative assisted localization and guidance path type identifiers include complete, conservative, downgraded, and corresponding treatment status. The mapping type used in this frame includes initial unrolling mapping. Phase 1: Detailed Mapping Two-stage detailed expansion mapping Key diagnostic metrics include catheter reference availability score and surgical field distortion risk score. The penile curvature was unfolded and the scale correction intensity index was recorded. With the second phase execution identifier;

[0098] The write-back is used to reuse the output type and key decision quantity in adjacent frames, suppress state jumps and form continuous and stable auxiliary positioning feedback, thereby forming a complete closed loop;

[0099] In the specific application of pediatric hypospadias surgery, the intraoperative auxiliary positioning guidance path output by this invention is superimposed and displayed on the current surgical field image, providing doctors with longitudinal positioning references for the urethral plate, incision direction references, flap alignment references, and new urethral formation direction references consistent with the catheter axis. Among them, the intraoperative auxiliary positioning guidance path in the original image domain under the penile in situ guidance state is suitable for situations where the catheter diameter is relatively consistent in the current image; the complete intraoperative auxiliary positioning guidance path under the penile curvature unfolding correction state is suitable for situations where the distal end of the catheter is significantly contracted, but the unfolding and refinement are already sufficient; the conservative intraoperative auxiliary positioning guidance path and error band under the penile curvature unfolding correction state are suitable for situations where there is still some uncorrected uncertainty in the current image, thus providing different levels of auxiliary guidance results under different reference conditions.

[0100] Please see Figure 4As shown, this invention discloses an image recognition-assisted positioning system for pediatric hypospadias surgery, comprising the following modules: a catheter reference module for acquiring surgical field images and determining the ventral surgical field region of the penis, performing catheter segmentation to obtain a catheter pixel-level probability map and a catheter mask, extracting the catheter axial reference line, and calculating the catheter reference reliability field along the line, the catheter reference availability score, the stable exposed segment of the catheter, the intraoperative constant diameter benchmark of the catheter, the surgical field anatomical segment label sequence, and the initial unfolding mapping;

[0101] The surface distortion determination module is used to compare the catheter reference availability score with the preset reference availability threshold. When the catheter reference availability score is lower than the preset reference availability threshold, the current processing state is determined to be the non-expandable degraded state. When the catheter reference availability score reaches the preset reference availability threshold, the surgical field surface distortion risk score is calculated, and the current processing state is determined to be the penile in situ guided state or the penile surface unfolding correction state.

[0102] The unfolding and refinement module is used to perform lateral scale correction on the initial unfolding map under the penile curvature unfolding correction state, and to determine whether to perform tissue traction offset correction, so as to obtain the final refined unfolding map;

[0103] The intraoperative auxiliary positioning guidance path rewrite module is used to generate downgrade prompts or intraoperative auxiliary positioning guidance paths according to the current processing status, and to project the intraoperative auxiliary positioning guidance paths in the unfolded domain back to the original image coordinate system for superposition display under the penile curvature unfolding correction state through the inverse mapping of the final refined unfolding mapping.

[0104] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0105] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, in the form of a computer program product.

[0106] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and inventive constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0107] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0108] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0109] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An image recognition-assisted localization method for hypospadias surgery in children, characterized in that, Includes the following steps: The ventral penile surgical field region of the surgical field image was segmented using the urinary catheter to obtain a pixel-level probability map and a urinary catheter mask. The axial reference line of the urinary catheter was extracted and sampled on the urinary catheter mask. The segmentation probability was determined based on the values ​​of the urinary catheter pixel-level probability map in the neighborhood of each sampling point. The observation width along the catheter was determined based on the normal boundary search of the urinary catheter mask. The axial bending morphology was determined based on the tangential direction change of adjacent sampling points. Based on this, the reliability field of the urinary catheter reference and the usability score of the urinary catheter reference were calculated. The stable exposed segment of the urinary catheter was determined based on the reliability field of the urinary catheter reference, and the intraoperative constant diameter benchmark of the urinary catheter was determined. The surgical field anatomical segment label sequence was generated according to the deviation and change trend of the observation width along the catheter relative to the intraoperative constant diameter benchmark of the urinary catheter. An initial unfolding mapping was constructed using the axial reference line of the urinary catheter as the virtual anatomical axis. The catheter reference availability score is compared with the preset reference availability threshold. If it is lower than the preset reference availability threshold, it is determined to be in an unexpandable and downgraded state, and the downgrade prompt is output before the frame ends. If the preset reference availability threshold is reached, the surgical field surface distortion risk score is calculated based on the consistency of the observed width along the route with the constant diameter reference of the catheter during the operation. The processing status is determined to be either penile in situ guided state or penile surface unfolding and correction state based on the comparison result with the preset surface risk threshold. In the penile in-situ guided state, an intraoperative auxiliary positioning and guidance path is generated and displayed in the original image coordinate system; in the penile curved surface unfolded and corrected state, the initial unfolding mapping is corrected by lateral scale of the partition according to the anatomical segment label sequence of the surgical field, and tissue traction offset correction is selectively performed according to the correction magnitude and catheter reference availability score to obtain the final refined unfolding mapping. An intraoperative auxiliary positioning and guidance path is generated in the unfolding domain, and the final refined unfolding mapping is inversely mapped back to the original image coordinate system for display.

2. The image recognition-assisted localization method for hypospadias surgery in children according to claim 1, characterized in that, The ventral penile surgical field region of the surgical field image was segmented using the urinary catheter to obtain a pixel-level probability map and a urinary catheter mask. This included: performing pixel-level segmentation of the ventral penile surgical field region to obtain a pixel-level probability map of the urinary catheter; performing thresholding, denoising, and connected component filtering on the pixel-level probability map of the urinary catheter to obtain candidate masks for a pre-set standard-grade outer diameter urinary catheter; extracting the skeleton of each candidate mask to form a candidate centerline; sampling along the candidate centerline and measuring the local width at each sampling point; and calculating the average width and width variation coefficient of each candidate mask. The system determines the gradation reference width range for the current frame based on preset catheter specifications and imaging scale information. Candidate masks whose average width falls within the gradation reference width range, whose width variation coefficient is not higher than a preset constant diameter stability threshold, and whose continuous effective length is not lower than a preset minimum length threshold are identified as valid candidate masks. The candidate mask with the highest comprehensive score among the valid candidate masks is selected as the catheter mask, and a skeleton is extracted from the catheter mask to obtain the catheter axial reference line. A set of sampling points is obtained by sampling along the catheter axial reference line at a preset step size.

3. The image recognition-assisted localization method for hypospadias surgery in children according to claim 1, characterized in that, The specific methods for determining the segmentation probability, along-path observation width, and axial bending morphology of each sampling point are as follows: The segmentation probability is the average probability value of the catheter pixel-level probability map in the neighborhood of the corresponding sampling point; the along-path observation width is determined by the normal direction based on the local tangential direction of the catheter axial reference line at the corresponding sampling point, and the boundary of the mask is searched on both sides along the normal direction, with the distance between the two boundaries as the along-path observation width at the corresponding sampling point; the axial bending morphology is obtained based on the deflection angle of the tangential direction of the corresponding sampling point relative to the tangential direction of the adjacent sampling points. The calculation of the catheter-related reference reliability field and catheter-related reference availability score includes: determining the visibility clarity along the catheter line based on segmentation probability; determining the catheter exposure stability based on local fluctuations in the observation width along the line; and determining the catheter-related abnormal bending penalty based on axial bending morphology. The visibility clarity, catheter exposure stability, and catheter-related abnormal bending penalty are combined to obtain the catheter-related reference reliability field. Sampling points in the catheter-related reference reliability field that are not lower than a preset confidence threshold are identified as high-reliability sampling points. The proportion of high-reliability sampling points to all sampling points is calculated to obtain the high-confidence coverage rate. The length of the longest continuous high-reliability exposed segment that continuously meets the preset confidence threshold along the catheter axis is calculated to obtain the longest continuous high-reliability exposed segment length. The high-reliability coverage rate and the longest continuous high-reliability exposed segment length are normalized and combined to obtain the catheter-related reference availability score. The sampling interval corresponding to the longest continuous high-reliability exposed segment is determined as the stable exposed segment of the catheter.

4. The image recognition-assisted localization method for hypospadias surgery in children according to claim 1, characterized in that, A surgical field anatomical segment label sequence is generated, and an initial unfolding mapping is constructed using the axial reference line of the urinary catheter as the virtual anatomical axis. Specifically, the intraoperative constant diameter benchmark of the urinary catheter is determined based on the stable exposed segment of the urinary catheter. The intraoperative constant diameter benchmark of the urinary catheter is the reference lateral pixel scale of the target urinary catheter under the current imaging scale. The constant diameter deviation at each sampling point is determined according to the difference between the observed width along the route and the intraoperative constant diameter benchmark of the urinary catheter. The changing trend of the constant diameter deviation along the axial reference line of the urinary catheter is calculated, and the sampling interval is divided into segments according to the constant diameter deviation and its changing trend along the axial direction of the urinary catheter. Among them, the continuous interval where the constant diameter deviation is continuously within the preset stable range and the width change gradient is continuously lower than the preset gradient threshold is marked as the ventral columnar maintenance segment of the penile body; the continuous interval where the width change gradient is continuously negative and its absolute value continuously exceeds the preset gradient threshold or the constant diameter deviation continuously exceeds the preset contraction threshold is marked as the coronal sulcus glans transition contraction segment. The cumulative arc length of the axial reference line of the urinary catheter is used as the longitudinal coordinate of the unfolded domain, and the signed normal distance from any pixel point in the ventral surgical field of the penis to the projection point of the axial reference line of the urinary catheter is used as the lateral coordinate of the unfolded domain to construct the initial unfolded mapping; wherein, the signed normal distance is positive in the direction from the axial reference line of the urinary catheter to the ventral side of the penis and negative in the direction from the dorsal side.

5. The image recognition-assisted localization method for hypospadias surgery in children according to claim 4, characterized in that, The surgical field surface distortion risk score is calculated, and the treatment state is determined as either penile in situ guided state or penile surface unfolding correction state based on the comparison result with the preset surface risk threshold. This includes: calculating the overall constant diameter deviation component based on the overall deviation of the observed width along the path at high-reliability sampling points relative to the intraoperative constant diameter baseline of the catheter; calculating the distal transition distortion enhancement component based on the continuous contraction intensity of the observed width along the path within the glans penis transition contraction segment, combined with the catheter reference reliability field along the corresponding sampling point; weighting and combining the overall constant diameter deviation component and the distal transition distortion enhancement component to obtain the surgical field surface distortion risk score; comparing the surgical field surface distortion risk score with the preset surface risk threshold, when the surgical field surface distortion risk score is not greater than the preset surface risk threshold, the treatment state is determined as penile in situ guided state; when the surgical field surface distortion risk score is greater than the preset surface risk threshold, the treatment state is determined as penile surface unfolding correction state.

6. The image recognition-assisted localization method for hypospadias surgery in children according to claim 4, characterized in that, Perform lateral scale correction on the initial unfolding mapping, including: mapping the sampling points of the axial reference line of the catheter to the unfolding domain through the initial unfolding mapping to obtain the coordinate sequence of the sampling points in the unfolding domain; and determining the ventral columnar maintenance segment of the penile body and the glans transition contraction segment of the coronal sulcus in the longitudinal coordinate direction of the unfolding domain according to the surgical field anatomical segment label sequence. For the sampling points in the columnar maintenance segment of the penile body, a control point sequence of the columnar maintenance segment of the penile body is constructed based on the constant diameter deviation, and a linear transverse correction relationship suitable for the relatively regular shape of the main trunk of the penile body is established. For the sampling points in the transitional contraction segment of the coronal sulcus and glans penis, a control point sequence for the transitional contraction segment of the coronal sulcus and glans penis is constructed based on the constant diameter deviation, and a unidirectional nonlinear transverse correction relationship applicable to the shape changes of the coronal sulcus and the distal transitional part of the glans penis is established. A smooth transition is performed between the linear lateral correction relation and the unidirectional nonlinear lateral correction relation at the segment boundary to obtain the global lateral scale correction relation; the global lateral scale correction relation is used to perform scale correction on the lateral coordinates of the expansion domain to obtain the first-stage refined expansion mapping.

7. The image recognition-assisted localization method for hypospadias surgery in children according to claim 6, characterized in that, Determining whether to perform tissue traction offset correction includes: determining the lateral correction strength score based on the degree of deviation of the global lateral scale correction relationship relative to the unit scale; The lateral correction strength score is combined with the catheter reference available score to obtain the two-stage offset correction trigger score; The two-stage offset correction trigger score is compared with the preset two-stage trigger threshold. If the two-stage offset correction trigger score is greater than or equal to the preset two-stage trigger threshold, it is determined that the tissue traction offset correction will be performed. If the two-stage offset correction trigger score is less than the preset two-stage trigger threshold, it is determined that the tissue traction offset correction will not be performed, and the one-stage refined unfolding mapping will be determined as the final refined unfolding mapping.

8. The image recognition-assisted localization method for hypospadias surgery in children according to claim 7, characterized in that, Perform tissue traction offset correction, including: taking the axial reference line sampling points of the catheter within the stable exposed segment of the catheter that meet the preset confidence threshold conditions as fitting samples, and mapping the fitting samples to the expanded domain after a one-stage refinement and expansion to obtain a set of sample points with longitudinal and lateral coordinates. Using the longitudinal coordinates of the expanded domain as the independent variable and the lateral coordinates of the sample points as the dependent variable, a tissue traction offset correction relationship for the lateral offset trend of the catheter axial reference line was established. Based on the differential deformation characteristics of the ventral columnar maintenance segment of the penile body and the glans transitional contraction segment of the coronal sulcus, which are divided by the surgical field anatomical segment label sequence, linear trend correction relationships and curvilinear trend correction relationships were used for weighted fitting according to reliability. The fitting weight was taken as the reference reliability field along the catheter line at the corresponding sampling point, thus obtaining the tissue traction offset correction relationship. Based on the tissue traction offset correction relationship, the lateral coordinates of the expanded domain obtained by the first-stage refined expanded mapping of any pixel point in the ventral surgical field area of ​​the penis were subtracted from the lateral offset value obtained by solving the tissue traction offset correction relationship at the corresponding longitudinal coordinate of the pixel point, thus obtaining the second-stage refined expanded mapping, which was used as the final refined expanded mapping.

9. The image recognition-assisted localization method for hypospadias surgery in children according to claim 1, characterized in that, The process of generating an intraoperative auxiliary positioning and guidance path specifically includes: when the penis is in the in-situ guided state, taking the axial reference line of the urinary catheter as the virtual anatomical axis reference in the original image coordinate system, taking the arc length parameter interval corresponding to the stable exposed segment of the urinary catheter as the intraoperative auxiliary positioning and guidance path generation interval, sampling along the intraoperative auxiliary positioning and guidance path generation interval according to the preset step size to generate an intraoperative auxiliary positioning and guidance path point sequence, and superimposing it on the surgical field image; When the penile curvature is in the corrected state and the final refined development mapping is a two-stage refined development mapping, the longitudinal coordinate interval corresponding to the stable exposed segment of the catheter is used as the basis in the development domain. The longitudinal range of the complete intraoperative auxiliary positioning and guidance path is expanded to both ends with a preset margin to obtain the complete intraoperative auxiliary positioning and guidance path point sequence. The complete intraoperative auxiliary positioning and guidance path point sequence is generated and displayed by back-projection to the original image coordinate system through the inverse mapping of the two-stage refined development mapping. When the penile curvature is in the corrected state and the final refined unfolding mapping is a one-stage refined unfolding mapping, the conservative intraoperative auxiliary positioning guidance path point sequence is generated only in the longitudinal coordinate interval corresponding to the stable exposed segment of the catheter, and an error band is generated based on the intraoperative constant diameter benchmark of the catheter. The conservative intraoperative auxiliary positioning guidance path point sequence and the error band are then back-projected to the original image coordinate system through the inverse mapping of the one-stage refined unfolding mapping.

10. An image recognition-assisted positioning system for pediatric hypospadias surgery, used to implement the image recognition-assisted positioning method for pediatric hypospadias surgery as described in any one of claims 1-9, characterized in that, The module includes the following components: The catheter reference module is used to acquire surgical field images and determine the ventral surgical field region of the penis. It performs catheter segmentation to obtain a catheter pixel-level probability map and a catheter mask, extracts the catheter axial reference line, and calculates the catheter reference reliability field along the line, the catheter reference availability score, the stable exposed segment of the catheter, the intraoperative constant diameter benchmark of the catheter, the surgical field anatomical segment label sequence, and the initial unfolding mapping. The surface distortion determination module is used to compare the catheter reference availability score with the preset reference availability threshold. When the catheter reference availability score is lower than the preset reference availability threshold, the current processing state is determined to be the non-expandable degraded state. When the catheter reference availability score reaches the preset reference availability threshold, the surgical field surface distortion risk score is calculated, and the current processing state is determined to be the penile in situ guided state or the penile surface unfolding correction state. The unfolding and refinement module is used to perform lateral scale correction on the initial unfolding map under the penile curvature unfolding correction state, and to determine whether to perform tissue traction offset correction, so as to obtain the final refined unfolding map; The intraoperative auxiliary positioning guidance path rewrite module is used to generate downgrade prompts or intraoperative auxiliary positioning guidance paths according to the current processing status, and to project the intraoperative auxiliary positioning guidance paths in the unfolded domain back to the original image coordinate system for superposition display under the penile curvature unfolding correction state through the inverse mapping of the final refined unfolding mapping.

Images