An image recognition-based epitympanic recess lateral wall defect repair auxiliary system and method

Abstract

The application provides an attic lateral wall defect repair auxiliary system and method based on image recognition, which is based on an intraoperative conventional ear endoscope image, measures and evaluates the size of the defect through image recognition and three-dimensional reconstruction technology, facilitates accurate construction of the graft, and is more fitted to the defect area. Through intraoperative photography to collect defect area images, image recognition technology is used to extract the defect contour (attic defect), a matched three-dimensional contour graph of the graft is automatically generated, and printing is output. The application solves the problem of insufficient precision of traditional manual trimming, shortens the operation time, improves the repair effect, and has the characteristics of simple operation and strong adaptability.

Description

Technical Field

[0001] This invention relates to the field of prosthesis implantation technology, and in particular to an image recognition-based auxiliary system for repairing defects in the outer wall of the superior tympanic membrane, its application control method, electronic device, and computer-readable storage medium. Background Technology

[0002] In endoscopic middle ear surgery, opening the epitympanum is a core step in treating diseases such as epitympanum cholesteatoma and chronic suppurative otitis media. During the operation, the lateral wall of the diseased epitympanum needs to be removed. After the operation, the defect area needs to be reconstructed and repaired using tragus cartilage, concha cartilage or bone material to avoid postoperative soft tissue retraction and cholesteatoma recurrence.

[0003] The lateral wall of the epitympanum is not a flat plane, but a complex three-dimensional curved structure with depressions, protrusions, and irregular bony ridges. Furthermore, the defect area is typically only 2-5 mm in size, requiring delicate repair procedures within a confined space. Current clinical techniques face a core challenge that cannot be overcome: 1. Insufficient precision in manual trimming and extremely poor fit: Current graft trimming relies entirely on the surgeon's clinical experience and spatial imagination. Manually trimming cartilage outside the body cannot accurately match the irregular three-dimensional curved surface of the defect area. The fit gap between the graft and the defect area generally exceeds 0.5mm, which easily leads to complications such as displacement, poor healing, and cholesteatoma recurrence.

[0004] 2. Lack of real-time intraoperative measurement capability and large dimensional error: Current intraoperative measurements can only be performed in two-dimensional linear form using micro calipers, which cannot capture the three-dimensional curved surface morphology of the defect; the defect morphology reconstructed by preoperative CT / CBCT deviates significantly from the actual defect after intraoperative bone removal, which cannot meet the needs of real-time and accurate intraoperative repair.

[0005] 3. Existing reconstruction solutions have high equipment requirements and cannot be adapted to primary clinical scenarios: Existing 3D reconstruction technology requires specialized intraoperative OCT and CBCT equipment or multi-view image acquisition. The equipment costs are high and the operation process is complicated. It is not compatible with the ear endoscope equipment commonly used during surgery, making it difficult to popularize in primary hospitals.

[0006] 4. Lack of standardized trimming templates and long learning curve: There are no standardized references for current graft trimming, which rely entirely on the surgeon's experience. The learning curve for young doctors is as long as 3-5 years, and the trimming time during surgery generally exceeds 15 minutes, which significantly prolongs the operation time and anesthesia risks.

[0007] 5. Interference from magnification of the otoscope and insufficient accuracy in size restoration: The otoscope is a wide-angle magnification device. Adjustments to the focal length and magnification during the operation will cause changes in the image scale. Current technology does not have a real-time calibration mechanism during the operation, and cannot achieve a 1:1 accurate restoration of the defect size.

[0008] Current patents and technical solutions are mostly focused on the repair of large-scale bone defects in maxillofacial surgery and dentistry. They have not been specifically optimized for the clinical characteristics of the epitympanum, such as its narrow space, irregular three-dimensional curved surface, and the need for aseptic operation during surgery. They have also not achieved compatibility with existing conventional otoscope equipment, leaving a clear technological gap. Summary of the Invention

[0009] To address the technical problems existing in the prior art, the present invention provides the following technical solution: On the one hand, an image recognition-based auxiliary system for repairing lateral wall defects of the epitympanum is provided, including an intraoperative image acquisition and interaction module, a ROI region extraction and image preprocessing module, an epitympanum defect contour recognition module, a defect three-dimensional morphology reconstruction module, a graft contour adaptation generation module, a print template generation module, a multi-terminal output module, an intraoperative real-time calibration module, and a data security and case management module, wherein: The intraoperative image acquisition and interaction module is used to be compatible with medical otoscope equipment to complete real-time image acquisition of the epitympanic cavity defect area during surgery, and provides a human-computer interaction interface to complete the selection of the target area ROI and parameter settings; The ROI region extraction and image preprocessing module is used to perform ROI cropping, noise reduction, image enhancement and lens distortion correction on the acquired raw image, and output a standardized ROI image. The epitympanic cavity defect contour recognition module, based on the improved U-Net semantic segmentation model, is used to automatically identify the epitympanic cavity defect region in the preprocessed image and extract the defect edge contour and size parameters at the subpixel level. The defect 3D morphology reconstruction module, based on the optimized Shape from Shading monocular 3D reconstruction algorithm, is used to reconstruct a 3D curved mesh model of the epitympanic cavity defect area from a single endoscopic image. The graft contour adaptation generation module is used to generate a three-dimensional solid model of the graft with a fixed margin that is completely adapted to the defect area based on the three-dimensional shape of the defect. The printing template generation module is used to convert the implant outline into a 2D / 3D printing template file with dimension annotations and calibration scales; The multi-terminal output module is used to be compatible with multi-terminal devices in the operating room and to complete the encrypted transmission and real-time output of the printed template; The intraoperative real-time calibration module is used to update the calibration coefficients in real time based on the known intraoperative size reference to ensure the 1:1 accuracy of the template size; The data security and case management module is used to complete the encrypted storage of patient case data, electronic medical record synchronization, and compliance auditing.

[0010] Preferably, the intraoperative image acquisition and interaction module incorporates a pixel-physical size calibration algorithm, with the calibration formula being: actual physical size L (mm) = P × K, and calibration coefficient. , in: P is the pixel distance to the target region. The known actual dimensions of the working cannula for the intraoperative ear endoscope. This represents the pixel distance of the working sleeve in the image, requiring no additional calibration equipment.

[0011] Preferably, the ROI region extraction and image preprocessing module uses adaptive median filtering to remove image noise, employs the contrast-limited adaptive histogram equalization (CLAHE) algorithm to enhance the image, and uses the Zhang Zhengyou calibration method to correct the radial distortion of the otoscope lens.

[0012] Preferably, the improved U-Net semantic segmentation model of the epitympanic cavity defect contour recognition module uses ResNet34 as the encoder backbone network, introduces the CBAM attention mechanism to enhance the defect edge features, and uses a combination loss function of Dice Loss and Focal Loss to complete the model training.

[0013] Preferably, the optimized Shape from Shading algorithm of the defect 3D morphology reconstruction module sets the light source direction to be consistent with the lens optical axis based on the characteristics of the coaxial light source of the endoscope, and the core reflection function formula is: , in: I(x,y) is the gray value at position (x,y) in the image. , For surface gradient, , The direction vector of the coaxial light source is used; the depth value of each pixel is obtained by iterative solution, generating a defective 3D point cloud and mesh model.

[0014] Preferably, the graft contour adaptation generation module generates the outer contour of the graft based on a fixed allowance of 0.3-0.5mm offset outward from the defect boundary contour. The outer contour is then matched to the three-dimensional surface of the defect via surface mapping. The surface mapping formula is: the z-coordinate of graft point P(x,y) ,in This represents the depth value corresponding to the location of the defective area. To compensate for the fit, stretch the material along the normal direction of the bonding surface by a preset thickness of 0.5-1.0 mm to generate a three-dimensional solid model of the graft.

[0015] Preferably, the printing template generation module, for 2D printing templates, scales the outline proportionally according to a magnification factor of 1-3 times set by medical personnel, and the scaling formula is: M represents the magnification factor; it automatically adds actual dimension annotations, scaling ratio annotations, and standard calibration rulers to generate a 300DPI PDF / SVG file; for 3D printing templates, it converts the 3D model of the transplant body into STL format to generate a model file adapted for desktop 3D printers.

[0016] Preferably, the intraoperative real-time calibration module updates the calibration coefficients in real time based on the intraoperative otoscope working cannula, and the calibration formula is: , in Given the known actual dimensions of the working sleeve. The pixel distance of the working sleeve in the image; when When the deviation from the original calibration coefficient K exceeds 5%, the error warning and recalibration process will be automatically triggered.

[0017] Preferably, the data security and case management module uses the national cryptographic SM4 algorithm to encrypt and store patient data end-to-end, uses blockchain technology to generate tamper-proof operation audit logs, and interfaces with the hospital's HIS / EMR electronic medical record system through the HL7 protocol to complete the compliant archiving of case data.

[0018] On the other hand, an application control method for the system described above is provided, comprising the following steps: S1: System initialization, completes docking with the otoscope device, loads the recognition model and calibration parameters, enters patient information, and completes initial calibration; S2: During the operation, endoscopic images of the epitympanic cavity defect area are acquired. Medical staff select the target ROI region and crop the ROI image. S3: Perform denoising, enhancement, and distortion correction preprocessing on the ROI image to obtain a standardized ROI image; S4: Based on the improved U-Net model, the automatic identification of the epitympanic cavity defect area is completed, and the sub-pixel level defect contour and size parameters are extracted; S5: Based on a single endoscopic ear image, an optimized Shape from Shading algorithm is used to reconstruct the three-dimensional morphology of the defect area and generate a three-dimensional mesh model of the defect. S6: Based on the defective 3D model, generate a 3D solid model of the graft with fixed margin that is perfectly adapted to the defective surface; S7: Generate a 2D / 3D printing template with dimension annotations and calibration scale based on the printing mode and magnification set by medical staff; S8: Output the printed template to the corresponding terminal in the operating room to complete the printing. Medical staff then trim the graft according to the template to complete the defect repair. S9: After the surgery, complete the encrypted storage of patient medical records, synchronization of electronic medical records, and compliant archiving.

[0019] On the other hand, an electronic device is provided, comprising: a processor; and a memory storing computer-readable instructions, which, when executed by the processor, implement the method described above.

[0020] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored therein, the at least one instruction being loaded and executed by a processor to implement the above method.

[0021] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: 1. High real-time performance during surgery, fully adapted to clinical surgical procedures: This system is based on the commonly used otoscope equipment during surgery, without the need for preoperative CT / CBCT or additional special hardware. During surgery, only a single image is needed to complete defect identification, 3D reconstruction, and template generation. The entire process takes ≤15 seconds without interrupting the surgical procedure. It is fully adapted to the requirements of aseptic operation during surgery and can be quickly popularized in primary hospitals.

[0022] 2. High-precision recognition and reconstruction to solve the problem of fitting complex curved surfaces: This system adopts an improved U-Net model to achieve sub-pixel level defect contour recognition with a size error of ≤2%; based on the SFS monocular 3D reconstruction algorithm optimized by the coaxial light source of the endoscopic endoscope, it accurately restores the irregular 3D curved surface of the outer wall of the epitympanum, and the gap between the generated implant and the defect area is ≤0.1mm, which completely solves the core pain point of poor fit of traditional manual trimming.

[0023] 3. Standardized printing templates significantly reduce reliance on surgeon experience: This system automatically generates 2D / 3D printing templates with precise annotations and calibration rulers. The 1-3x magnified templates facilitate precise cartilage trimming by medical staff in vitro, eliminating the need for surgeons to rely on spatial imagination and manual experience. This significantly shortens the learning curve for young doctors and also significantly reduces intraoperative trimming time, thereby reducing surgical and anesthesia risks.

[0024] 4. Real-time calibration throughout the entire process ensures 1:1 accurate size reproduction: This system uses the working cannula of the otoscope routinely used during surgery as a standard reference, eliminating the need for an additional calibration plate. It completes pixel-to-physical size calibration and adjustment in real time. When the equipment parameters are adjusted, an error warning is automatically triggered to ensure that the template size is reproduced 1:1 with the actual defect, and the size error is controlled within 2%, solving the problem of size deviation caused by changes in the magnification of the otoscope.

[0025] 5. Compatible with existing clinical equipment, highly adaptable and compliant: This system adopts standardized protocols that are compatible with mainstream medical otoscope equipment, eliminating the need to modify existing operating room equipment and resulting in low deployment costs; it uses national cryptographic encryption algorithms and blockchain audit logs, meeting the compliance requirements for medical data security and electronic medical record management, and can be directly connected to hospital HIS / EMR systems.

[0026] 6. Deeply optimized for epitympanic cavity defects, with strong clinical adaptability: This system has been specifically optimized for the clinical characteristics of the epitympanic cavity, such as its small space, irregular three-dimensional curved surface, and poor image quality in the surgical area during surgery. At the same time, it takes into account the fixation margin and thickness requirements for clinical transplantation. The generated grafts fully meet the clinical needs of middle ear surgery, avoiding the problem of poor adaptability of general repair systems. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a flowchart of the automatic generation method for the transplantation template of the present invention; Figure 2 This is a schematic diagram of the system architecture of the present invention; Figure 3 This is a flowchart of ROI region extraction and image preprocessing provided by an embodiment of the present invention; Figure 4 This is an example diagram of a defect marking area of ​​the ROI in the epitympanum provided by the present invention; Figure 5 This is a schematic diagram of the working mechanism of the tympanic cavity defect contour recognition module of the present invention; Figure 6 This is a diagram of the transplant contour adaptation generation mechanism of the present invention. Detailed Implementation

[0029] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0030] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0031] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0032] In this embodiment of the invention, sometimes a subscript such as W1 may be mistakenly written as a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0033] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0034] This invention designs a measuring tool to assess the size of defects, facilitating precise graft repair and a better fit to the defect area. Images are directly acquired via an ear endoscope and uploaded to the system backend for image processing. The images can be enlarged and printed as 2D or 3D images using the system's zoom function. The printed information includes a zoom level note, allowing medical staff to specify the desired size, such as a 1-3x magnification. Doctors can precisely trim the tragus or concha cartilage according to a template (the area to be repaired is a portion of the surgical area; image acquisition can be done by selecting a region, for example, by adding a selection window to the backend system. After the ear endoscope acquires the image, medical staff can select the region. The image area is set via the window, and the subsequently acquired and transmitted images only cover that region), achieving a high degree of fit between the graft and the defect area.

[0035] like Figure 1 As shown, this application employs a fully closed-loop rigid control logic of "acquisition-preprocessing-identification-reconstruction-generation-output-calibration-archiving" to achieve accurate identification of epitympanic cavity defects and automatic generation of transplant templates, completely solving the core pain points of traditional technologies. The core process is as follows: 1. Initialization: After the system starts up, it completes the communication interface with the otoscope device, loads the pre-trained recognition model and lens distortion correction parameters, medical staff complete the permission verification and patient information entry, and completes the initial calibration based on the otoscope working sleeve.

[0036] 2. Image Acquisition: Medical staff trigger image acquisition, the system captures the original image without damage, a ROI selection window pops up, and the preprocessing process is automatically triggered after the ROI selection is completed, without any additional manual operation.

[0037] 3. Recognition and Reconstruction: The system automatically completes image preprocessing, defect contour recognition, and 3D morphology reconstruction. The entire process takes ≤2 seconds. After processing, the defect parameters and 3D model are displayed on the interactive interface for medical staff to confirm.

[0038] 4. Template generation: Medical staff set the printing mode and magnification, and the system automatically generates a labeled printing template. After size verification, the template is sent to the output module.

[0039] 5. Output: Medical staff select the output terminal, and the system automatically completes encrypted transmission and print triggering, providing real-time feedback on the output status.

[0040] 6. Real-time calibration: The system monitors changes in otoscope parameters in real time. When the deviation of the calibration coefficient exceeds the threshold, it automatically triggers an error warning and recalibration process to ensure the accuracy of the template size.

[0041] 7. Archiving: After the surgery, the system automatically completes the encrypted storage of patient data, synchronizes electronic medical records, generates unalterable audit logs, and completes the compliant archiving of medical records.

[0042] The following section details the technical implementation principle and effects of an image recognition-based auxiliary system for repairing defects in the outer sidewall of an upper drum.

[0043] This system adopts a three-tiered medical compliance architecture of "intraoperative terminal - local edge server - hospital central server". All real-time intraoperative processing is completed on the local edge server in the operating room to ensure low latency and data security.

[0044] like Figure 2 As shown, the system consists of the following modules: (I) Intraoperative Image Acquisition and Interaction Module This module is compatible with mainstream clinical medical otoscopes, enabling non-invasive real-time image acquisition of the epitympanic cavity defect area during surgery. It provides a human-computer interaction interface adapted for aseptic operation, allowing medical staff to select the target area ROI, set parameters, and trigger processes. It serves as the system's intraoperative data entry point.

[0045] It adopts the standardized USB UVC / Video4Linux protocol, is compatible with routine clinical ear endoscopes such as 0° and 30°, supports lossless frame capture at 1080P / 2K resolution, and avoids motion blur through a frame buffering mechanism; it has a built-in pixel-physical size calibration algorithm, using the working cannula of the ear endoscope routinely used during surgery as the standard reference, requiring no additional calibration equipment. The calibration formula is as follows: L(mm) = P × K, where: , In the formula: L is the actual physical size of the target area, P is the pixel distance of the target area, and K is the initial calibration coefficient; The known actual size of the standard reference (ear endoscope working cannula) (commonly used in clinical practice: 2.7mm / 4.0mm). This represents the pixel distance to the reference object in the image. The interactive interface features a sterile touch-adaptive design, supporting operation while wearing gloves during surgery to avoid contamination of the surgical area.

[0046] During the procedure, the otoscope enters the surgical area and clearly exposes the defect of the lateral wall of the epitympanum. The medical staff triggers image acquisition, and the module automatically captures the lossless original image of the current frame. Simultaneously, a ROI selection window pops up, and the medical staff selects the target area where the defect is located using a sterile touch screen / mouse. After confirmation, the focal length and magnification parameters of the acquisition device are recorded, and the image marked with the ROI and the calibration parameters are sent to the next module.

[0047] (II) ROI Region Extraction and Image Preprocessing Module This module performs ROI cropping, noise reduction, image enhancement, and lens distortion correction on the acquired raw images, resolving image quality issues caused by intraoperative bleeding in the surgical area, bone surface reflection, and wide-angle lens distortion, and providing a standardized, high-quality data source for subsequent contour recognition.

[0048] like Figure 3 As shown, the ROI region extraction and image preprocessing process is as follows: a. Adaptive Median Filtering for Denoising: To address salt-and-pepper noise caused by intraoperative blood and tissue debris, an adaptive median filter with a variable window is employed to preserve defect edge details. A denoising algorithm is provided below: , In the formula, (i,j) represents the coordinate index of the image pixel, where i represents the row number (vertical position) and j represents the column number (horizontal position). (i,j) allows for precise location of any pixel in the image. For example, in an M×N image, the value of i typically ranges from 0 to M-1, and the value of j ranges from 0 to N-1. S is the adaptive filtering window. When the pixel value within the window is within the effective grayscale range, the original value is retained; otherwise, the window's median value is used to replace it, avoiding edge blurring caused by fixed-window filtering. The median value is the median value after sorting all pixel grayscale values ​​within the filtering window. It is the core computational unit for achieving noise suppression and detail preservation (unlike mean filtering, median filtering does not perform pixel averaging, thus avoiding blurring of image edges; in adaptive median filtering, the system dynamically adjusts the window size based on noise density, achieving a balance between noise reduction and detail preservation).

[0049] b. Limit contrast adaptive histogram equalization (CLAHE): Set clipLimit=2.0 and grid size 8×8 to solve the problems of uneven lighting in the surgical area and reflection on the bone surface, and enhance the contrast of the defect edge.

[0050] c. Zhang Zhengyou's calibration method for distortion correction: Based on the factory calibration parameters (x, y) of the otoscope, radial distortion correction of the wide-angle lens is performed. The correction formula is as follows: , In the formula, Here, is the radial distortion coefficient, and r is the distance from the pixel to the image center, restoring the true proportions of the image. These are the coordinate parameters of the corrected image.

[0051] The module receives the original image with ROI markers, performs target region cropping based on ROI coordinates, removes irrelevant background, sequentially performs adaptive median filtering for noise reduction, CLAHE image enhancement, and lens radial distortion correction, and completes pixel-to-physical size mapping based on initial calibration coefficients, outputting a standardized ROI image (e.g., ...). Figure 4 Example of marking the defective area of ​​the epitympanum ROI (shown in the example).

[0052] The module also processes the initial calibration coefficients K and the acquisition device parameters, and outputs the updated calibration coefficients K after preprocessing. In the adaptive median filtering system for medical images, updating the calibration coefficients K is the core mechanism for achieving a balance between dynamic noise suppression and detail preservation. The K coefficient update process can be divided into the following key stages: 1. Initial K value setting When the system starts up, the K coefficient is usually preset to an initial value based on equipment parameters and clinical experience: Equipment-related parameters: Set the baseline K value according to the inherent noise characteristics of medical imaging equipment (such as CT, MRI, ultrasound); Clinical scenario adaptation: Adjust the initial K value for different examination sites (such as brain, chest, abdomen) and imaging modes; Noise density prediction: Based on historical data or pre-scan results, make a preliminary estimate of the noise density and set the corresponding K value range; 2. Real-time K-value adjustment algorithm The K coefficient is updated using a multi-dimensional feedback mechanism, dynamically adjusted through the following steps: K_new (new coefficient) = K_old (old coefficient) × α × β × γ Wherein: α (noise density factor): dynamically adjusted according to the proportion of noisy pixels in the current window, α = 1 + (number of noisy pixels / total number of pixels in the window) × gain coefficient. When the noise density exceeds the threshold (e.g., 20%), the value of α increases, enhancing the filtering strength; β (edge ​​protection factor): calculated based on image gradient information, β = 1 - (edge ​​gradient value / maximum gradient threshold) × protection coefficient. In edge regions (high gradient value), the value of β decreases, reducing the filtering strength to protect edge details; γ (structural complexity factor): adjusted according to local texture complexity, γ = 1 - (texture entropy value / maximum entropy threshold) × complexity coefficient. In texture-rich regions (high entropy value), the value of γ decreases, reducing the destruction of details by filtering.

[0053] 3. Coordinated updates of adaptive window and K value The K coefficient is adjusted in conjunction with the filter window size to form a closed-loop feedback: Window size assessment: Determine whether the window is appropriate based on the current K value and noise detection results; K-value pre-adjustment: If the window is too small and noise remains, temporarily increase the K-value to enhance the filter; Window size update: If increasing the K value to the threshold still cannot effectively remove noise, then increase the window size; Final calibration of K value: After window adjustment, the K value is recalibrated based on the statistical characteristics within the new window.

[0054] 4. K-value update strategy for multimodal images The K-value update algorithm will be optimized for different types of medical images: CT imaging: The impact of X-ray dose, slice thickness, and reconstruction algorithm on noise should be considered in detail. MRI images: K value adjusted based on sequence type (T1 / T2 weighted), magnetic field strength, and scanning parameters; Ultrasound imaging: The K-value is dynamically updated based on probe frequency, depth gain compensation, and tissue type.

[0055] 5. Clinical feedback and K-value optimization The system records clinical diagnostic results and continuously optimizes the K-value update strategy through machine learning algorithms: Expert-annotated data: Noise areas and key structures annotated by doctors are used as training samples; Performance index evaluation: Optimize the K-value adjustment rule based on objective indicators such as PSNR and SSIM and subjective clinical evaluation; Online learning mechanism: Data is collected in real time during system operation to continuously improve the K-value update algorithm; This dynamic update mechanism ensures that the system can automatically adjust to the optimal working state under different noise environments and clinical scenarios, effectively removing noise while preserving the diagnostic information of medical images to the maximum extent.

[0056] (III) Hypertympanic cavity defect contour recognition module This module is based on a deep learning semantic segmentation algorithm to automatically identify the epitympanic cavity defect area in the preprocessed image, extract the defect edge contour and size parameters at the subpixel level, and solve the problems of low accuracy and low efficiency of manual delineation.

[0057] like Figure 5 As shown, an improved U-Net semantic segmentation model is adopted, optimized for the characteristics of small targets, irregular edges, and low contrast in epitympanic cavity defects: the encoder uses ResNet34 as the backbone network and introduces the CBAM attention mechanism to strengthen the feature weights of the defect edges; the decoder uses upsampling + skip connections to restore edge details; the loss function adopts a combination of Dice Loss + Focal Loss to solve the problem of imbalanced positive and negative samples. The core formula is: ,in: , In the formula, A represents the defect region predicted by the model, and B represents the defect region labeled by the gold standard; α=0.6 and β=0.4 are weighting coefficients. The Focal Loss balancing parameters are used. The model was pre-trained and fine-tuned on a dataset of 1200 intraoperative epitympanic cavity defect images, and the Dice coefficient on the validation set was ≥0.94, meeting the clinical accuracy requirements.

[0058] The preprocessed standardized ROI image is input into the trained improved U-Net model, and the model outputs a binary segmentation mask of the defect area (defect area is 1, background is 0). The Suzuki85 contour tracking algorithm is used to extract the sub-pixel level edge contour of the defect area from the binary mask, generating an ordered set of contour pixel coordinates. A 3rd-order B-spline curve is used to smooth the contour and remove jagged noise. The actual physical parameters of the defect (major axis, minor axis, area, perimeter) are calculated based on the calibration coefficient K and output to the interactive interface for medical staff to confirm.

[0059] (iv) Module for Reconstructing the Three-Dimensional Morphology of Defects This module reconstructs the three-dimensional curved surface morphology of the lateral wall defect of the epitympanum based on the light and shadow information of a single endoscopic image. This addresses the problem that traditional two-dimensional contours cannot reflect the depressions and protrusions of the defect, providing a foundation for graft surface adaptation. The module receives the preprocessed ROI image, the defect contour coordinate set, and coaxial light source parameters, setting the light source direction to be consistent with the lens optical axis. It iteratively solves the depth value of each pixel to generate a three-dimensional point cloud of the defect area. Statistical filtering is used to remove outliers from the point cloud, completing point cloud denoising. A three-dimensional mesh model of the defect area is generated through Poisson surface reconstruction, and the three-dimensional contour of the defect boundary is extracted and sent to the graft adaptation module.

[0060] Specifically, the reconstruction method employs an optimized Shape from Shading (SFS) monocular 3D reconstruction algorithm, optimized for the characteristics of the coaxial light source of the endoscope (the endoscope light source is coaxial with the lens optical axis, requiring no additional light source calibration). The core reflection function formula is:

[0061] In the formula: I(x,y) is the gray value at (x,y) in the image, and R(p,q) is the Lambertian reflection function. , The gradient of the curved surface; under coaxial light source of the otoscope This significantly simplifies computational complexity. By iteratively minimizing grayscale error, the depth value z(x,y) of each pixel is solved to generate a 3D point cloud of the missing region. A Poisson surface reconstruction algorithm is then used to convert the 3D point cloud into a continuous 3D mesh model with a resolution of 0.05mm.

[0062] The reconstruction algorithm mechanism is as follows: The technical principle of this algorithm is based on the Lambertian reflection model. It assumes that the object surface is an ideal diffuse reflector, and its brightness I(x,y) satisfies the following relationship with the incident light direction L, the surface normal vector N(x,y), and the reflectivity ρ(x,y): I(x,y) = ρ(x,y)·max(0, L·N(x,y)). When the surface is assumed to be a homogeneous material (i.e., ρ is a constant, often normalized to 1) and the light source is a far-field parallel light (such as L = [0, 0, 1] or any unit vector), the brightness image directly reflects the projection of the normal vector onto the light source direction. Therefore, the original image I can be regarded as a first-order nonlinear partial differential equation concerning the unknown surface height function z(x,y): ,in , These are the gradient components in the x and y directions, respectively, i.e., the surface normal vector. This equation is the classic Horn-type SFS fundamental equation, and its high nonlinearity and ill-conditioned nature necessitate a numerical solution. In practice, image preprocessing is first performed, including grayscale normalization, noise suppression (such as Gaussian filtering or anisotropic diffusion), and edge enhancement to improve image quality. Next, illumination parameters are calibrated, allowing users to specify the light source direction or automatically estimate them using specular highlight / shadow boundaries. Then, PDE discretization is performed, often using finite difference methods to transform the continuous equation into a large-scale nonlinear system, such as using upwind schemes or weighted central difference to improve stability. Finally, optimization is performed, with common strategies including variational methods (minimizing the regularized energy functional, gradient descent, Newton's method, and the Fast Marching Method). It is particularly noteworthy that reconstructing the height map z(x,y) using the normal vector integral is itself a Poisson integral problem: the estimated p(x,y) and q(x,y) must satisfy compatibility conditions. (i.e., curl is zero), otherwise it will lead to integral path dependence and distortion. Therefore, least squares integral or Fourier domain solution is often introduced (such as using FFT to efficiently calculate the Poisson equation). Furthermore, robust boundary condition processing is required. Due to the lack of gradient information at image boundaries, Neumann boundaries (normal vectors perpendicular to the boundary) or Dirichlet boundaries (setting the boundary height to zero or the average value) are often used, supplemented by a mask mechanism to eliminate background interference. Finally, result evaluation is performed, providing multi-dimensional verification methods such as 3D mesh visualization (surf / mesh), contour plots, normal field arrow diagrams, and error heatmaps (RMSE / SSIM) with real scan data.

[0063] (v) Transplant contour adaptation generation module This module generates a 3D solid model of the implant that perfectly matches the defect area based on the 3D morphology of the defect, while also taking into account clinical fixation margins and fit compensation, meeting the clinical operational requirements of middle ear surgery. The fitting process is as follows: a. Boundary offset: Based on the three-dimensional boundary contour of the defect, offset outward by a fixed margin of 0.3-0.5mm (clinically used for fixing the graft to the defect edge) to generate the outer contour of the graft; b. Surface Mapping: The external contour of the implant is mapped onto the three-dimensional surface of the defect to ensure a perfect match between the implant's fitting surface and the defect surface. The mapping formula is as follows:

[0064] In the formula, Let P(x,y) be the z-axis coordinate of the graft contour point. This represents the depth value corresponding to the location of the defective area. To ensure proper fit and avoid gaps in the fit; c. Solid generation: Stretch along the normal of the mating surface to a preset thickness of 0.5-1.0mm (the thickness of the tragus cartilage commonly used in clinical practice) to generate a three-dimensional solid model of the implant with thickness.

[0065] like Figure 6 As shown, the module receives the 3D mesh model of the defect and the 3D contour of the boundary. Medical staff set clinical parameters in the interactive interface (the default margin is 0.4mm and the default thickness of the transplant is 0.8mm). The module offsets the defect boundary to generate the outer contour of the transplant. It generates a transplant fitting surface that fits the defect perfectly through surface mapping. It stretches the preset thickness along the normal to generate a 3D solid model of the transplant. The model is smoothed to remove sharp edges, and a 2D contour projection of the transplant is generated at the same time.

[0066] (vi) Print Template Generation Module This module converts the implant outline into a standardized, printable template file, supporting both 2D and 3D printing. It automatically adds dimension annotations, calibration rulers, and clinical notes, solving the problems of traditional templates lacking annotations and having large scaling errors. The printing operation is as follows: a. 2D Printing Template: Projecting the 2D outline of the graft into a vector graphic (original image coordinates) The vector graphics coordinates were scaled proportionally to the magnification factor of 1-3 times set by the medical staff to obtain new vector graphics coordinates. The scaling is calculated as follows: , In the formula, M is the magnification factor (default 2x); it automatically adds actual physical dimension annotations for the major and minor axes, scaling ratio annotations, and a 10mm standard calibration ruler, generating a 300DPI PDF / SVG file to ensure printing accuracy.

[0067] b. 3D Printing Template: Convert the 3D solid model of the transplant body into STL format, add lightweight support structures, set the layer thickness to 0.1mm, and adapt it to commonly used desktop 3D printers in clinical practice.

[0068] The system receives the 3D model and 2D outline of the transplant recipient. Medical staff select the printing mode (2D / 3D) and set the magnification for 2D printing. The module automatically completes the outline scaling, dimension annotation, and ruler addition to generate a print file in the corresponding format. Notes are added to a fixed location in the file, including patient ID, surgery date, actual size, scaling ratio, and calibration method.

[0069] (vii) Multi-terminal output module This module is compatible with various terminal devices in the operating room, enabling encrypted transmission and real-time output of printed templates, adapting to the requirements of aseptic operation during surgery, and avoiding cross-contamination.

[0070] It uses TCP / IP protocol to connect to the medical LAN, supports wireless / wired encrypted transmission, and complies with the hospital network security level protection 2.0 standard; it has a built-in virtual printing driver, which is compatible with all models of medical sterile printers in the operating room; it adopts HTML5 responsive design, adapts to mobile terminals such as intraoperative displays and sterile tablets, and supports real-time viewing of template outlines and dimensions.

[0071] The system receives print template files and automatically converts them to the corresponding compatible format based on the output terminal selected by medical staff. It then sends the files to the target terminal via a medical local area network encrypted with the national cryptographic standard SM2. For sterile printers, it automatically triggers printing and outputs annotated templates. For display screens / tablet terminals, it generates a real-time visual interface for intraoperative reference.

[0072] You can refer to the existing mobile app printing process.

[0073] In this facility, the otoscope equipment, intraoperative sterile touch screen, sterile printer, mobile tablet, and operating room edge server use HTTPS national cryptographic SM2 / SM4 encrypted communication. Image data is transmitted in lossless segments to adapt to the weak network environment of the operating room. All terminal operations are completed under sterile isolation conditions.

[0074] (viii) Intraoperative real-time calibration module This module addresses dimensional errors caused by adjustments to the focal length and magnification of the endoscopic lens during surgery, and updates the calibration coefficients in real time to ensure a 1:1 accurate reproduction of the template size to the actual defect.

[0075] Based on the intraoperatively known size of the ear endoscope working cannula, reference objects in the images are detected in real time, and the calibration coefficients are updated as follows:

[0076] In the formula, The known actual size of the reference object, The pixel distance to the reference object in the image; set an error threshold, when When the deviation from the original calibration coefficient K exceeds 5%, an error warning is automatically triggered, prompting medical staff to confirm whether to re-acquire and process the image to ensure that the size error is ≤2%.

[0077] During the procedure, after medical staff adjust the focal length and magnification of the otoscope, the module automatically identifies the working cannula of the otoscope in the image and calculates the new calibration coefficients. If the deviation exceeds 5% when compared with the original calibration coefficients, an automatic warning will pop up. After confirmation by medical staff, the calibration coefficients of the entire process will be updated, and the contour recognition and template generation will be completed again to ensure the accuracy of the dimensions.

[0078] System Service Layer Functions: The service layer primarily handles the encrypted storage of patient case data, electronic medical record synchronization, and compliance auditing, complying with the requirements of the "Personal Information Protection Law" and the "Regulations on Medical Record Management in Medical Institutions," ensuring the security and traceability of medical data. It employs the national cryptographic algorithm SM4 for end-to-end encrypted storage of patient image data, case information, and template files; utilizes blockchain technology for immutable on-chain recording of the entire process operation log, including collection time, operator, modification records, and output records; and interfaces with the hospital's HIS / EMR electronic medical record system via the standardized HL7 protocol to complete compliant archiving of case data. It receives patient data and operation logs from the entire system process, encrypts and stores the data in separate databases; automatically generates standardized surgical auxiliary records, including defect parameters, transplant models, and print records; synchronizes data to the hospital's electronic medical record system via the HL7 protocol; sets tiered access permissions, authorizing only medical personnel to access patient data, and generates compliance audit logs.

[0079] The service layer primarily consists of the hospital's central server, which communicates with various modules via the intranet gRPC protocol for low latency. Core data is stored using multiple encrypted copies, and compliant data synchronization with the hospital's HIS / EMR system is achieved through the standardized HL7 protocol. All operations are recorded in tamper-proof audit logs. The operating room's local edge server communicates with the hospital's central server via a wired TCP / IP protocol over a medical LAN. All data packets are encrypted using the national cryptographic algorithm SM4. Intraoperative real-time image processing and model inference are performed on the edge server, with end-to-end processing latency ≤2 seconds, ensuring real-time performance during surgery.

[0080] The specific implementation process of this system is described in detail below.

[0081] The application method and steps of this system are as follows: The system starts up, completes communication with the intraoperative otoscopic device, loads the pre-trained improved U-Net defect recognition model, lens distortion correction parameters, and coaxial light source parameters; medical staff complete system permission verification, enter basic patient information and surgical information; confirm the known outer diameter parameters of the otoscopic working cannula, complete the initial calibration coefficient settings, and the system enters the data acquisition state. This completes the preoperative preparation of the system, ensuring compatibility with existing routine intraoperative otoscopic devices without requiring additional dedicated hardware, establishing a foundation for dimensional calibration, and fully adapting to intraoperative aseptic operation requirements.

[0082] After clearing the epitympanic cavity lesion and fully exposing the defect area of ​​the lateral wall of the epitympanum, medical staff operate the otoscope to adjust the focus and magnification, clearly displaying the defect area and surrounding bone surface. The system's image acquisition function is triggered, and the system automatically captures a 1080P / 2K lossless original image of the current frame. A Region of Interest (ROI) selection window pops up, allowing medical staff to select the target area of ​​the defect using a sterile touchscreen. After confirmation, the system crops the ROI image, simultaneously recording the focus and magnification parameters of the acquisition device. This real-time image acquisition of the defect area during surgery, along with ROI selection to remove irrelevant background and focus on the target area, improves the efficiency and accuracy of subsequent processing. The operation is simple, does not interrupt the surgical procedure, and does not add any extra operational burden during surgery.

[0083] The system receives the ROI region image and first uses adaptive median filtering to remove salt-and-pepper noise caused by intraoperative blood and tissue debris, preserving the details of the defect edges. Then, the CLAHE algorithm is used for image enhancement to address uneven illumination and bone surface reflection in the surgical area, improving the contrast of the defect edges. Next, pre-loaded distortion correction parameters are used to correct radial distortion of the wide-angle lens of the otoscope, restoring the true proportions of the image. Finally, based on the initial calibration coefficients, the image pixels are mapped to physical dimensions, outputting a standardized ROI image. This solves the core problems of poor intraoperative surgical area image quality and lens distortion, outputting standardized, high-quality ROI images, providing a reliable data source for subsequent contour recognition, and improving contour recognition accuracy by more than 15%.

[0084] The preprocessed standardized ROI image is input into a pre-trained improved U-Net semantic segmentation model, which outputs a binary segmentation mask of the defect area. The Suzuki85 contour tracking algorithm is used to extract the sub-pixel-level edge contour of the defect area from the binary mask, generating an ordered set of contour pixel coordinates. A 3rd-order B-spline curve is used to smoothly fit the contour and remove jagged noise. Based on calibration coefficients, the actual physical parameters of the defect contour, including the major axis, minor axis, area, and perimeter, are calculated. The contour and parameters are displayed in real time on the system's interactive interface for medical staff to confirm. This method achieves automatic and high-precision recognition of epitympanic cavity defect contours. Sub-pixel-level edge extraction ensures a size error of ≤2%, eliminating the need for manual contour outlining and reducing intraoperative contour extraction time from the traditional 5 minutes to less than 1 second, significantly reducing reliance on surgeon experience.

[0085] The system receives the preprocessed ROI image, the defect contour coordinate set, and coaxial light source parameters. Using an optimized SFS monocular 3D reconstruction algorithm, iteratively solves for the depth value of each pixel based on image grayscale information, generating a 3D point cloud of the defect area. Statistical filtering is applied to the 3D point cloud to remove outliers. A Poisson surface reconstruction algorithm is then used to generate a continuous 3D mesh model of the defect with a resolution of 0.05 mm. The 3D contour of the defect area's boundary is extracted and sent to the transplant fitting module. The system can complete the 3D surface reconstruction of the defect area based on a single intraoperative endoscopic image, without the need for additional equipment or multi-view images. It offers strong real-time performance and accurately reconstructs the irregular 3D curved surface structure of the outer wall of the epitympanum, solving the problem that traditional 2D contours cannot reflect the surface morphology, thus laying the foundation for accurate transplant fitting.

[0086] The system receives a 3D mesh model of the defect and its 3D contour. Medical staff set clinical parameters in the system interface, including fixation allowance (default 0.4mm) and graft thickness (default 0.8mm). The system offsets the defect boundary contour outward to generate the outer contour of the graft, ensuring that the graft can cover the defect edge for intraoperative fixation. The outer contour of the graft is mapped onto the 3D curved surface of the defect to generate a graft fitting surface that perfectly matches the defect surface. The preset graft thickness is stretched along the normal of the fitting surface to generate a 3D solid model of the graft with thickness. The model is smoothed to remove sharp edges, and a 2D contour projection of the graft is generated. This generates a graft model that perfectly matches the 3D curved surface of the defect, while also taking into account clinical fixation needs, ensuring that the fitting gap between the graft and the defect area is ≤0.1mm, completely solving the problems of poor fitting and excessive gaps caused by traditional manual trimming.

[0087] Medical staff select the printing mode (2D magnification printing / 3D printing) and set the magnification for 2D printing (1-3x, default 2x). For 2D printing, the system projects the two-dimensional outline of the transplant into a vector graphic, scales it proportionally according to the set magnification, automatically adds actual physical dimension annotations for the major and minor axes, scaling annotations, and a 10mm standard calibration ruler, generating a printable PDF / SVG file at 300 DPI. For 3D printing, the system converts the three-dimensional solid model of the transplant into STL format, adds suitable support structures, and generates a model file directly usable for 3D printing. Simultaneously, patient ID, surgery date, and parameter notes are added to the template file. Standardized, precisely annotated printing templates are generated. The 1-3x magnification templates facilitate precise cartilage trimming in vitro, and the standard calibration ruler ensures the accuracy of the trimming dimensions, solving the problems of traditional templates lacking annotations and having large scaling errors.

[0088] Medical staff select an output terminal (sterile printer, intraoperative display screen, mobile tablet). The system sends the print template file to the target terminal via a medical encrypted LAN, triggering the sterile printer to complete the template printing, or displays the template in real time on the display screen / tablet. During the operation, when medical staff adjust the focus and magnification of the otoscope, the system automatically identifies the otoscope working cannula in the image and updates the calibration coefficients in real time. When the calibration coefficient deviation exceeds 5%, an automatic warning is triggered, prompting medical staff to confirm whether to re-complete image acquisition and processing to ensure 1:1 template size accuracy. This achieves real-time intraoperative template output, fully adapting to the aseptic operation requirements of the operating room. The real-time calibration mechanism ensures the accuracy of the template size, avoiding size errors caused by intraoperative equipment adjustments, with size errors controlled within 2%.

[0089] After the surgery, the system automatically summarizes all data from the entire procedure, including acquired images, defect parameters, graft models, printed templates, and operation logs, and stores them encrypted using the national cryptographic algorithm SM4. It generates standardized surgical auxiliary records, which are synchronized to the hospital's HIS / EMR electronic medical record system via the HL7 protocol. Blockchain technology is used to generate tamper-proof operation audit logs, completing case archiving. Hierarchical access permissions are set, authorizing only medical staff to access patient data. This achieves compliant storage and archiving of patient medical data, fully complying with the requirements of the Personal Information Protection Law and the Regulations on Medical Record Management in Medical Institutions, ensuring the security and traceability of medical data. Example 1

[0090] The patient was a 38-year-old male diagnosed with right epitympanic cholesteatoma. He underwent endoscopic epitympanic cavity opening, cholesteatoma removal, and epitympanic lateral wall reconstruction under general anesthesia. This system was used intraoperatively to assist in graft trimming and defect repair. The procedure is as follows: 1. System initialization: The system starts up, completes communication docking with the 0° high-definition ear endoscope (working cannula outer diameter 2.7mm) used in the operation, loads the pre-trained improved U-Net model and lens distortion correction parameters, enters the patient's basic information, and sets the initial calibration reference to the 2.7mm ear endoscope working cannula.

[0091] 2. Image Acquisition and ROI Selection: During the operation, the cholesteatoma in the epitympanum is removed, and the defect area of ​​the lateral wall of the epitympanum is fully exposed. The focal length of the otoscope is adjusted to clearly display the defect, triggering image acquisition. The system captures a 1080P lossless original image. Medical staff select the ROI area where the defect is located through the sterile touch screen during the operation, and after confirmation, the ROI image is obtained by cropping.

[0092] 3. Image preprocessing: The system sequentially performs adaptive median filtering for noise reduction, CLAHE image enhancement, and radial distortion correction on the ROI image to obtain a standardized ROI image; based on the 2.7mm working sleeve, the initial calibration coefficient K = 0.018mm / pixel is calculated (the pixel distance of the working sleeve in the image is 150 pixels, 2.7÷150=0.018).

[0093] 4. Defect Contour Recognition: The preprocessed ROI image is input into the improved U-Net model, which outputs a binary mask of the defect area. Subpixel-level contour coordinate sets are extracted, and after B-spline smoothing, the actual defect parameters are calculated: major axis 3.6mm, minor axis 2.8mm, defect area 7.9mm². 2 The system displays the outline and parameters on the interface, and medical staff confirm that everything is correct.

[0094] 5. 3D morphology reconstruction: Based on the ROI image, contour coordinates, and coaxial light source parameters, the optimized SFS algorithm is used to solve the depth value, generate a 3D point cloud of the defect area, and Poisson reconstruction is used to obtain a 3D mesh model with a resolution of 0.05mm, which restores the concave surface of the defect area with a maximum concavity depth of 0.6mm.

[0095] 6. Transplant model generation: Medical staff set a fixed margin of 0.4mm and a transplant thickness of 0.8mm. The system offsets the defect boundary outward by 0.4mm to generate the outer contour of the transplant, maps it to the three-dimensional curved surface of the defect, stretches it to a thickness of 0.8mm, and generates a three-dimensional solid model of the transplant, while simultaneously generating a two-dimensional contour projection map.

[0096] 7. Print Template Generation: Medical staff select 2D printing mode, magnification 2x, the system generates SVG format print template, magnifies the outline by 2x, adds annotations: actual major axis 3.6mm, actual minor axis 2.8mm, scaling ratio 2:1, 10mm calibration ruler, and generates a 300DPI PDF file.

[0097] 8. Template output and graft trimming: The system sends the template file to the sterile printer in the operating room to print out the labeled template. Medical staff trim the tragus cartilage according to the template outline to obtain a graft that is completely consistent with the template. During the operation, the graft is placed in the epitympanic cavity defect area, completely fitting with a fitting gap of ≤0.1mm, and fixed with fibrin glue.

[0098] 9. Postoperative archiving: After the surgery, the system automatically encrypts and stores all data, generates surgical auxiliary records, and synchronizes them to the hospital's electronic medical record system to complete compliant archiving.

[0099] Results: In this patient's implantation procedure, the total time from image acquisition to template printing was ≤15s, and the fit between the graft and the defect area was ≤0.1mm, which is far superior to the gap of more than 0.5mm achieved by traditional manual trimming. The graft trimming time during the operation was reduced from the traditional 15-20 minutes to 3 minutes, and the total operation time was reduced by about 25 minutes. One month after the operation, the graft healed well, with no displacement or cholesteatoma recurrence, and the epitympanum morphology recovered well.

[0100] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0101] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0102] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0103] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0104] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0105] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0106] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0107] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0108] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0109] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0110] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An image recognition-based auxiliary system for repairing defects in the outer sidewall of an upper drum, characterized in that, This includes modules for intraoperative image acquisition and interaction, ROI region extraction and image preprocessing, epitympanic cavity defect contour recognition, defect 3D morphology reconstruction, graft contour adaptation generation, print template generation, multi-terminal output, intraoperative real-time calibration, and data security and case management. The intraoperative image acquisition and interaction module is used to be compatible with medical otoscope equipment to complete real-time image acquisition of the epitympanic cavity defect area during surgery, and provides a human-computer interaction interface to complete the selection of the target area ROI and parameter settings; The ROI region extraction and image preprocessing module is used to perform ROI cropping, noise reduction, image enhancement and lens distortion correction on the acquired raw image, and output a standardized ROI image. The epitympanic cavity defect contour recognition module, based on the improved U-Net semantic segmentation model, is used to automatically identify the epitympanic cavity defect region in the preprocessed image and extract the defect edge contour and size parameters at the subpixel level. The defect 3D morphology reconstruction module, based on the optimized Shape from Shading monocular 3D reconstruction algorithm, is used to reconstruct a 3D curved mesh model of the epitympanic cavity defect area from a single endoscopic image. The graft contour adaptation generation module is used to generate a three-dimensional solid model of the graft with a fixed margin that is completely adapted to the defect area based on the three-dimensional shape of the defect. The printing template generation module is used to convert the implant outline into a 2D / 3D printing template file with dimension annotations and calibration scales; The multi-terminal output module is used to be compatible with multi-terminal devices in the operating room and to complete the encrypted transmission and real-time output of the printed template; The intraoperative real-time calibration module is used to update the calibration coefficients in real time based on the known intraoperative size reference to ensure the 1:1 accuracy of the template size; The data security and case management module is used to complete the encrypted storage of patient case data, electronic medical record synchronization, and compliance auditing.

2. The system according to claim 1, characterized in that, The intraoperative image acquisition and interaction module has a built-in pixel-physical size calibration algorithm. The calibration formula is: actual physical size L (mm) = P × K, calibration coefficient. ， in: P is the pixel distance to the target region. The known actual dimensions of the working cannula for the intraoperative ear endoscope. This represents the pixel distance of the working sleeve in the image, requiring no additional calibration equipment.

3. The system according to claim 1, characterized in that, The ROI region extraction and image preprocessing module uses adaptive median filtering to remove image noise, the contrast-limited adaptive histogram equalization (CLAHE) algorithm to enhance the image, and Zhang Zhengyou calibration method to correct radial distortion of the endoscope lens.

4. The system according to claim 1, characterized in that, The improved U-Net semantic segmentation model of the epitympanic cavity defect contour recognition module uses ResNet34 as the encoder backbone network, introduces the CBAM attention mechanism to enhance the defect edge features, and uses a combination of Dice Loss and Focal Loss loss function to complete the model training.

5. The system according to claim 1, characterized in that, The optimized Shape from Shading algorithm of the defect 3D morphology reconstruction module sets the light source direction to be consistent with the lens optical axis based on the characteristics of the coaxial light source of the endoscope. The core reflection function formula is: ， in: I(x,y) is the gray value at position (x,y) in the image. , For surface gradient, , The direction vector of the coaxial light source is used; the depth value of each pixel is obtained by iterative solution, generating a defective 3D point cloud and mesh model.

6. The system according to claim 1, characterized in that, The graft contour adaptation generation module generates the outer contour of the graft based on a fixed margin of 0.3-0.5mm offset outward from the defect boundary contour. The outer contour is then matched to the three-dimensional surface of the defect via surface mapping. The surface mapping formula is: the z-coordinate of graft point P(x,y)... ,in This represents the depth value corresponding to the location of the defective area. To compensate for the fit, stretch the material along the normal direction of the bonding surface by a preset thickness of 0.5-1.0 mm to generate a three-dimensional solid model of the graft.

7. The system according to claim 1, characterized in that, The printing template generation module, for 2D printing templates, scales the outline proportionally according to a magnification factor of 1-3 times set by medical personnel. The scaling formula is as follows: M represents the magnification factor; it automatically adds actual dimension annotations, scaling ratio annotations, and standard calibration rulers to generate a 300DPI PDF / SVG file; for 3D printing templates, it converts the 3D model of the transplant body into STL format to generate a model file adapted for desktop 3D printers.

8. The system according to claim 1, characterized in that, The intraoperative real-time calibration module updates the calibration coefficients in real time based on the intraoperative ear endoscope working cannula. The calibration formula is as follows: ， in Given the known actual dimensions of the working sleeve. The pixel distance of the working sleeve in the image; when When the deviation from the original calibration coefficient K exceeds 5%, an error warning and recalibration process will be automatically triggered.

9. The system according to claim 1, characterized in that, The data security and case management module uses the national cryptographic SM4 algorithm to encrypt and store patient data end-to-end, uses blockchain technology to generate tamper-proof operation audit logs, and interfaces with the hospital's HIS / EMR electronic medical record system through the HL7 protocol to complete the compliant archiving of case data.

10. An application control method for the system according to any one of claims 1-9, characterized in that, Includes the following steps: S1: System initialization, completes docking with the otoscope device, loads the recognition model and calibration parameters, enters patient information, and completes initial calibration; S2: During the operation, endoscopic images of the epitympanic cavity defect area are acquired. Medical staff select the target ROI region and crop the ROI image. S3: Perform denoising, enhancement, and distortion correction preprocessing on the ROI image to obtain a standardized ROI image; S4: Based on the improved U-Net model, the automatic identification of the epitympanic cavity defect area is completed, and the sub-pixel level defect contour and size parameters are extracted; S5: Based on a single endoscopic ear image, an optimized Shape from Shading algorithm is used to reconstruct the three-dimensional morphology of the defect area and generate a three-dimensional mesh model of the defect. S6: Based on the defective 3D model, generate a 3D solid model of the graft with fixed margin that is perfectly adapted to the defective surface; S7: Generate a 2D / 3D printing template with dimension annotations and calibration scale based on the printing mode and magnification set by medical staff; S8: Output the printed template to the corresponding terminal in the operating room to complete the printing. Medical staff then trim the graft according to the template to complete the defect repair. S9: After the surgery, complete the encrypted storage of patient medical records, synchronization of electronic medical records, and compliant archiving.

Images