Telescopic wound endoscope based on AI recognition and monitoring method

Abstract

The invention relates to the technical field of medical treatment, and discloses a telescopic wound endoscope based on AI recognition and a monitoring method.The monitoring method comprises the steps that image data and state data in a wound are collected through a high-resolution endoscope, the image data comprise visible light images and multispectral images, and the state data comprise temperature data and pressure data in the wound; after image data is subjected to noise removal through a Gaussian filtering algorithm, the contour of a wound and the contour of a foreign matter are extracted through an edge detection algorithm, and the edge detection algorithm comprises Canny edge detection and Sobel gradient operators; the image data and the state data are fused to form a temperature heat map and a pressure heat map, and data fusion is carried out through a weighted average method; identifying and classifying foreign matters through an AI model; estimating the three-dimensional position and size of the foreign body based on the contour of the wound and the foreign body; and generating and outputting a report based on the temperature heat map, the pressure heat map, the three-dimensional position and the size. The wound treatment efficiency can be effectively improved, and manual errors are reduced.

Description

Technical Field

[0001] This invention relates to the field of medical technology, and more specifically, to an AI-based scalable wound endoscope and monitoring method. Background Technology

[0002] With the development of medical imaging technology, wound monitoring and foreign body detection have become crucial tasks in clinical medicine, especially in the treatment of complex wounds. Timely detection of the presence and location of foreign bodies within the wound can effectively improve treatment efficiency and accuracy. Traditional wound monitoring methods rely on manual observation and routine examinations, which suffer from incomplete detection, low efficiency, and a high rate of misdiagnosis. Furthermore, while existing endoscopic techniques can provide visual images of wounds, they often lack the ability to automatically identify and locate foreign bodies within the wound, and conventional temperature and pressure detection methods are relatively simple and cannot accurately provide important information such as the size and location of foreign bodies.

[0003] Therefore, there is an urgent need for a monitoring method that can automatically identify foreign objects in wounds and provide accurate location information. Summary of the Invention

[0004] In recent years, wound monitoring methods based on computer vision and artificial intelligence (AI) technologies have gradually become a research hotspot. By combining high-resolution endoscopic images and data from multiple sensors (such as temperature and pressure sensors), along with edge detection algorithms, data fusion techniques, and deep learning models, wound areas can be identified more accurately, foreign objects can be located, and their size estimated. The combination of these technologies significantly improves the efficiency and accuracy of wound monitoring, reduces human error, and can provide real-time spatial information on foreign objects within the wound, offering more precise decision-making support for clinical treatment. Therefore, this invention proposes an AI-based scalable endoscopic wound monitoring method to address the aforementioned problems.

[0005] This invention proposes an AI-based scalable wound endoscope and monitoring method, comprising:

[0006] Image and status data of the wound are acquired using a high-resolution endoscope. The image data includes visible light images and multispectral images, and the status data includes temperature and pressure data of the wound.

[0007] After removing noise from the image data using a Gaussian filtering algorithm, an edge detection algorithm is used to extract the contours of the wound and foreign objects; wherein, the edge detection algorithm includes Canny edge detection and Sobel gradient operator;

[0008] The image data and state data are fused to form a temperature heatmap and a pressure heatmap; wherein the data fusion is performed using a weighted average method.

[0009] Foreign objects are identified and classified using AI models; the three-dimensional location and size of the foreign objects are estimated based on the contours of the wound and the foreign objects.

[0010] A report is generated and output based on the temperature heatmap, pressure heatmap, three-dimensional position, and dimensions.

[0011] Preferably, the formula for calculating the Gaussian filter is:

[0012]

[0013] Where G(x, y) represents the Gaussian filter kernel; x and y represent the horizontal and vertical coordinates of pixels in the image, respectively; σ represents the standard deviation of the Gaussian filter, which is used to control the smoothness of the filter.

[0014] Preferably, the steps of the Canny edge detection and Sobel gradient operator include:

[0015] The denoised image is then converted to grayscale to obtain a grayscale image.

[0016] The Sobel operator is used to perform convolution operations on the image, and the gradients in the x and y directions are calculated as follows:

[0017]

[0018] Where Gx and Gy represent the gradients of the image in the x and y directions, respectively; I(i,j) represents the gray value of a pixel in the image; Sx(i,j) and Sy(i,j) represent the convolution kernels of the Sobel operator in the x and y directions, respectively.

[0019] Edge detection is performed using a dual threshold algorithm, and the calculation formula is as follows:

[0020]

[0021] Where, intermediate represents a pixel whose gradient value is between the low threshold and the high threshold; highthreshold and low threshold represent the set edge strength thresholds, respectively.

[0022] Preferably, the temperature and pressure heatmaps are generated using a weighted average method, specifically calculated as follows:

[0023] T fusion (x, y) = w T ·T image (x, y) + w P ·P image (x, y);

[0024] P fusion (x, y) = wT ·P image (x, y) + w P ·T image (x, y);

[0025] Among them, T fusion (x, y) represents the temperature heatmap after fusion; P fusion (x, y) represents the pressure heatmap after fusion; w T and w P These represent the weighting coefficients for temperature and pressure data, respectively; T image (x, y) and P image (x, y) represent the pixel values ​​of the temperature image and the pressure image, respectively.

[0026] Preferably, the AI ​​model is YOLOv4.

[0027] Preferably, the training steps of the YOLOv4 model include:

[0028] Image data is divided into a grid, and each grid predicts the object's class probability, bounding box coordinates, and confidence score. Joint training is performed using the cross-entropy loss function and the squared error loss function, calculated as follows:

[0029]

[0030] Where L represents the value of the loss function; L classification L represents the classification loss, i.e., the cross-entropy loss; location This represents the position loss, i.e., the squared error loss; L confidence λ represents the confidence loss, i.e., the confidence loss; λ1 and λ2 represent the adjustment coefficients, respectively.

[0031] Preferably, the formula for calculating the cross-entropy loss is:

[0032]

[0033] The formula for calculating the squared error loss is:

[0034]

[0035] The formula for calculating the confidence loss is:

[0036]

[0037] Where yi represents the label of the real category; represents the class probability predicted by the model; bi represents the class probability predicted by the model. Represents the coordinates of the bounding box predicted by the model; ci represents the label indicating whether the target exists, ci is 0 or 1; This represents the confidence level of the model prediction; i represents the index variable.

[0038] Preferably, the three-dimensional position is calculated using triangulation, the steps of which include:

[0039] Image data from two different locations are acquired using a camera, and the coordinates of corresponding points at these two locations are extracted. Depth information is calculated using the triangle theorem; the formula for calculating the three-dimensional position is:

[0040]

[0041] Where Z represents the depth of the object; B represents the baseline distance between the two viewpoints; f represents the focal length of the camera; and d represents the parallax of corresponding points in the two images.

[0042] Preferably, the size of the foreign object is estimated using a fitting algorithm based on the contour, calculated as follows:

[0043]

[0044] Where yi represents the actual measurement point; f(xi, θ) represents the fitting function; θ represents the fitting parameter; and n represents the number of data points.

[0045] Preferably, the real-time monitoring of the temperature and pressure data is achieved through a sensor array, which includes multiple temperature and pressure sensors. The sensor array timestamps the collected status data through a data synchronization module and performs data fusion using a weighted average method.

[0046] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0047] 1. Accurate wound area and foreign body contour extraction: By using edge detection algorithms (such as Canny or Sobel operators), the system can accurately extract the contour information of the wound and foreign bodies within it. Combined with high-resolution image data acquired by endoscopy, the wound area and foreign body contour can be effectively segmented, enabling efficient identification even in complex wound environments.

[0048] 2. Precise Capture of Temperature and Pressure Distribution: By fusing temperature and pressure thermograms, the system can monitor real-time temperature and pressure changes around the wound and foreign bodies. Temperature and pressure maps can reveal potential foreign body influences within the wound, providing valuable clues about the contact between the foreign body and the wound. Non-uniform temperature and pressure distributions typically indicate the presence of a foreign body and its contact pattern with tissue, assisting the system in accurately locating the foreign body in the images.

[0049] 3. Intelligent Foreign Object Recognition and Classification: Through an AI model (YOLOv4), the system can intelligently analyze the extracted wound area and foreign object outlines, automatically identifying and classifying different types of foreign objects. These models, after training, can accurately classify and label foreign objects of different shapes and materials based on endoscopic image data and combined temperature and pressure information.

[0050] 4. Three-dimensional position and size estimation: Combining image data, temperature thermograms, and pressure thermograms, the system can accurately estimate the three-dimensional position and size of foreign objects based on triangulation. Spatial location and size calculation of foreign objects not only help determine their type (e.g., metal, glass), but also provide detailed information for subsequent clinical treatment, assisting doctors in selecting appropriate surgical plans.

[0051] 5. Multimodal data fusion improves accuracy: By fusing multiple data sources such as edge detection algorithms, temperature heatmaps, pressure heatmaps, and AI recognition models, the system can comprehensively analyze multiple physical and geometric features within the wound, reducing false positives that may arise from a single data source. For example, abnormal changes in temperature and pressure may help the system eliminate false positives and improve the accuracy of foreign body detection.

[0052] 6. Real-time Monitoring and Feedback: This system not only acquires wound images and sensor data in real time, but also performs image processing, data fusion, and AI analysis during the acquisition process, providing real-time feedback. Doctors can monitor the presence of foreign objects in the wound at any time during treatment, enabling timely intervention and reducing medical risks.

[0053] 7. Non-invasive and efficient: This technology relies on the integrated use of endoscopic images and sensor data, possessing the advantage of being non-invasive. Compared with traditional invasive examination methods, this system reduces patient pain and risks, while significantly improving the efficiency and accuracy of wound monitoring through automated algorithm processing.

[0054] In summary, the AI-based wound endoscopy and monitoring method proposed in this invention combines edge detection algorithms, temperature thermograms, pressure thermograms, and deep learning models. This not only improves the accuracy of foreign body detection in wounds but also provides more accurate foreign body localization and size estimation. This technology can provide clinicians with real-time, precise foreign body monitoring data, effectively improving wound treatment efficiency, reducing human error, and has high clinical application value. Attached Figure Description

[0055] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0056] Figure 1 This is a flowchart illustrating the AI-based scalable wound endoscope and monitoring method of the present invention.

[0057] Figure 2 This is a schematic diagram of the structure of the AI-based retractable wound endoscope of the present invention.

[0058] Figure 3 This is a schematic diagram of the flexible probe structure of the AI-based retractable wound endoscope of the present invention.

[0059] Figure 4 This is a schematic diagram of the display unit structure of the AI-based retractable wound endoscope of the present invention.

[0060] In the diagram, 1 is the handle body; 2 is the flexible probe; 3 is the connection and sealing assembly; 4 is the display unit; and 5 is the power supply / data interface. Detailed Implementation

[0061] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the disclosure to those skilled in the art. It should be noted that, unless otherwise specified, embodiments and features in the embodiments of the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0062] See Figure 1 As shown, this embodiment provides an AI-based scalable wound endoscope and monitoring method, including:

[0063] Image and status data of the wound are acquired using a high-resolution endoscope. The image data includes visible light images and multispectral images, and the status data includes temperature and pressure data of the wound.

[0064] After removing noise from the image data using a Gaussian filtering algorithm, an edge detection algorithm is used to extract the contours of the wound and foreign objects; wherein, the edge detection algorithm includes Canny edge detection and Sobel gradient operator;

[0065] The image data and state data are fused to form a temperature heatmap and a pressure heatmap; wherein the data fusion is performed using a weighted average method.

[0066] Foreign objects are identified and classified using AI models; the three-dimensional location and size of the foreign objects are estimated based on the contours of the wound and the foreign objects.

[0067] A report is generated and output based on the temperature heatmap, pressure heatmap, three-dimensional position, and dimensions.

[0068] As can be seen, this embodiment provides a wound endoscopy and monitoring method based on artificial intelligence (AI) recognition technology. This method can effectively monitor and analyze wounds in real time, thereby providing important information for medical diagnosis and treatment. The specific steps are as follows:

[0069] High-resolution endoscopy is used to perform detailed examinations of the wound, acquiring image and status data within the wound. Image data includes visible light and multispectral images, providing detailed visual information about the wound. Status data includes temperature and pressure data within the wound, reflecting its physiological state and environmental conditions.

[0070] After acquiring the data, the image data is first preprocessed using a Gaussian filtering algorithm to remove noise and improve the accuracy of subsequent processing. Then, an edge detection algorithm is used to extract the contours of the wound and any foreign objects inside. Edge detection algorithms include, but are not limited to, Canny edge detection and the Sobel gradient operator; these algorithms can accurately identify edge information in the image.

[0071] The processed image data is fused with the status data to create temperature and pressure heatmaps. The temperature heatmap visually displays the temperature distribution in the wound area, while the pressure heatmap reflects the pressure distribution. The data fusion process can be achieved using a weighted average method, ensuring that information from different data sources is appropriately combined.

[0072] AI models are used to analyze and process the fused data to identify and classify foreign objects within the wound. These AI models can be deep learning networks, such as convolutional neural networks (CNNs), which can learn the characteristics of foreign objects from image data and perform accurate identification and classification.

[0073] After the foreign object is identified and classified, its three-dimensional location and size are estimated based on the contour information of the wound and the foreign object. This step is crucial for understanding the specific location of the foreign object in the wound and the potential damage it may cause.

[0074] Finally, based on information such as temperature thermograms, pressure thermograms, and the three-dimensional location and size of the foreign object, a comprehensive report is generated. This report will provide medical professionals with a comprehensive wound analysis result, helping them make more accurate diagnostic and treatment decisions.

[0075] In summary, the monitoring method of this embodiment provides an efficient and accurate solution for wound monitoring by combining high-resolution endoscopy technology, image processing algorithms, AI recognition technology, and data fusion technology.

[0076] Understandably, temperature and pressure thermograms reveal the physical characteristics of wounds and foreign objects within them, while edge detection focuses on geometric shape recognition. Combining these techniques can provide more comprehensive contextual information, thereby improving the accuracy of foreign object detection.

[0077] Specifically, the correlation between temperature heatmaps, pressure heatmaps, and edge detection is as follows:

[0078] (1) Combination of temperature thermogram and edge detection

[0079] Relationship between temperature distribution and edges: Temperature distribution typically varies significantly between different areas within the wound, at the wound edges, and on the surface of foreign objects. For example, foreign objects (such as metallic objects) may form distinct high- or low-temperature regions on a temperature thermogram due to their different thermal conductivity compared to surrounding tissue. Using temperature thermograms as background information helps edge detection algorithms more accurately locate the wound area and the position of foreign objects.

[0080] Edge detection-assisted temperature thermography: The wound contour and foreign object boundary information extracted by the edge detection algorithm can assist in the analysis of the temperature thermography, especially at the edges, which may be transitional areas of temperature change. Combined with edge detection information, the system can more accurately determine whether temperature changes are related to foreign objects, rather than simply changes in the wound itself.

[0081] Temperature characteristics of foreign objects: If the edge detection algorithm has identified the outline of the foreign object, the material properties of the foreign object can be inferred by combining it with the temperature thermogram. For example, a higher temperature region may indicate that the foreign object has higher thermal conductivity, or that it is a special medical material (such as metal or ceramic), whose temperature may be significantly different from that of the surrounding tissue.

[0082] (2) Combination of pressure heatmap and edge detection

[0083] Relationship between pressure changes and edges: The presence of foreign bodies typically alters the pressure distribution in a localized area. In wounds or other tissue areas, foreign bodies may cause increased local pressure or produce uneven pressure changes, which can be visualized using pressure thermography.

[0084] Edge detection-assisted pressure thermography: Edge detection extracts the contours of wounds and foreign objects, providing useful geometric information for pressure thermography analysis. In pressure thermography, areas at the contour edges typically exhibit greater pressure differences, especially around foreign objects. Combining edge detection information helps pinpoint the specific location of pressure anomalies, determining whether they are pressure changes at the wound edges or caused by the foreign object itself.

[0085] The effect of foreign bodies on pressure: The effect of foreign bodies on pressure thermography is usually manifested as the pressure difference between the foreign body area and the surrounding tissue. The foreign body contour extracted by edge detection helps to better identify these areas, and combined with pressure thermography, the size, shape, and contact mode of the foreign body with the wound can be inferred.

[0086] In summary, combining temperature and pressure thermograms with edge detection results allows for multi-dimensional analysis of wounds and foreign objects. For example, by combining temperature, pressure, and contour information, it is helpful to analyze the location, shape, thermal conductivity, and pressure distribution of wounds and foreign objects, thereby improving the accuracy of foreign object detection. After the edges of wounds and foreign objects are detected and extracted, combining the information from temperature and pressure thermograms allows for a more accurate estimation of the size of the foreign object. For instance, extreme values ​​of temperature and pressure may occur at the edge of the foreign object or in its contact with the wound; by combining contour information, the system can infer the three-dimensional location and size of the foreign object.

[0087] In some embodiments of this application, the Gaussian filter is calculated as follows:

[0088]

[0089] Where G(x, y) represents the Gaussian filter kernel; x and y represent the horizontal and vertical coordinates of pixels in the image, respectively; σ represents the standard deviation of the Gaussian filter, which is used to control the smoothness of the filter.

[0090] It is understood that in the formula of this embodiment, e is the base of the natural logarithm, approximately equal to 2.71828. This formula indicates that the Gaussian filter kernel is a two-dimensional Gaussian function with a maximum value at the center, which gradually decreases with increasing distance from the center. The larger the value of σ, the stronger the smoothing effect of the Gaussian filter kernel, but the more details in the image are lost; conversely, the smaller the value of σ, the more image details the filter kernel retains, but the weaker the smoothing effect. In practical applications, the choice of σ needs to be determined according to the specific task requirements to achieve the best image processing effect.

[0091] In some embodiments of this application, the steps of the Canny edge detection and Sobel gradient operator include:

[0092] The denoised image is then converted to grayscale to obtain a grayscale image.

[0093] The Sobel operator is used to perform convolution operations on the image, and the gradients in the x and y directions are calculated as follows:

[0094]

[0095] Where Gx and Gy represent the gradients of the image in the x and y directions, respectively; I(i,j) represents the gray value of a pixel in the image; Sx(i,j) and Sy(i,j) represent the convolution kernels of the Sobel operator in the x and y directions, respectively.

[0096] Edge detection is performed using a dual threshold algorithm, and the calculation formula is as follows:

[0097]

[0098]

[0099] Where, intermediate represents a pixel whose gradient value is between the low threshold and the high threshold; highthreshold and low threshold represent the set edge strength thresholds, respectively.

[0100] Understandably, in this embodiment, the dual-threshold algorithm is an effective edge detection method that can reduce noise and false edges in edge detection. By setting a high threshold and a low threshold, the algorithm first marks pixels above the high threshold as edge points, and then suppresses pixels below the low threshold, i.e., considers them not to be edges. For pixels in between, the algorithm decides whether to mark them as edges based on their connectivity with the marked edge points. If these pixels are connected to edge points above the high threshold, they will also be marked as edge points. This method can better preserve important edge information in the image while suppressing the influence of noise.

[0101] In some embodiments of this application, the temperature and pressure heatmaps are generated using a weighted average method, specifically calculated as follows:

[0102] T fusion (x, y) = w T ·T image (x, y) + w P ·P image (x, y);

[0103] P fusion (x, y) = w T ·P image (x, y) + w P ·T image (x, y);

[0104] Among them, T fusion (x, y) represents the temperature heatmap after fusion; P fusion (x, y) represents the pressure heatmap after fusion; w T and w P These represent the weighting coefficients for temperature and pressure data, respectively; T image (x, y) and P image (x, y) represent the pixel values ​​of the temperature image and the pressure image, respectively.

[0105] It is understood that the temperature and pressure heatmaps generated by the weighted average method in this embodiment can comprehensively consider information from both temperature and pressure. This is achieved by adjusting the weighting coefficients w for the temperature and pressure data. T and w P This allows control over the relative importance of temperature and pressure information when generating heatmaps. For example, if temperature information is more critical, the w value can be increased. T The value of w is increased if the stress information is more important; conversely, if the stress information is more important, then w is increased. P The value of T. image (x, y) and P image (x, y) represent the pixel values ​​at position (x, y) of the original temperature and pressure images, respectively. This weighted fusion yields a heatmap that integrates both temperature and pressure information, which helps in a more comprehensive analysis and understanding of the scene.

[0106] In some embodiments of this application, the AI ​​model is YOLOv4.

[0107] In some embodiments of this application, the training steps of the YOLOv4 model include:

[0108] Image data is divided into a grid, and each grid predicts the object's class probability, bounding box coordinates, and confidence score. Joint training is performed using the cross-entropy loss function and the squared error loss function, calculated as follows:

[0109]

[0110] Where L represents the value of the loss function; L classification L represents the classification loss, i.e., the cross-entropy loss; location This represents the position loss, i.e., the squared error loss; L confidence λ represents the confidence loss, i.e., the confidence loss; λ1 and λ2 represent the adjustment coefficients, respectively.

[0111] Understandably, this embodiment utilizes the YOLOv4 model to effectively detect and classify objects in a scene in real time. The YOLOv4 model is renowned for its fast and accurate detection capabilities, making it suitable for applications requiring real-time processing. During training, by dividing image data into multiple grids and predicting the object's class probability, bounding box coordinates, and confidence score in each grid, the model learns how to identify and locate objects in images. The cross-entropy loss function optimizes the classification task, while the squared error loss function optimizes the accuracy of bounding box coordinate prediction. The confidence loss ensures the model's accuracy in predicting the presence or absence of objects. By adjusting the adjustment coefficients λ1 and λ2, the contributions of classification loss, location loss, and confidence loss to the total loss can be balanced, thereby optimizing the model's overall performance. In practical applications, this training method can improve the model's detection accuracy for different scenes and objects, maintaining high detection efficiency and accuracy even in various complex environments.

[0112] In some embodiments of this application, the cross-entropy loss is calculated as follows:

[0113]

[0114] The formula for calculating the squared error loss is:

[0115]

[0116] The formula for calculating the confidence loss is:

[0117]

[0118] Where yi represents the label of the real category; represents the class probability predicted by the model; bi represents the class probability predicted by the model. Represents the coordinates of the bounding box predicted by the model; ci represents the label indicating whether the target exists, ci is 0 or 1; This represents the confidence level of the model prediction; i represents the index variable.

[0119] Understandably, this embodiment defines in detail the components of the loss function, ensuring that the model can accurately learn all aspects of classification, localization, and confidence prediction during training. The cross-entropy loss function measures the difference between the model's predicted class probability distribution and the true label; the squared error loss function measures the difference between the predicted bounding box coordinates and the true coordinates; and the confidence loss function ensures that the model can accurately predict the existence of the target. Through the combination of these loss functions, the YOLOv4 model can comprehensively consider different types of errors during training and perform corresponding optimizations, thereby improving the model's performance in practical applications.

[0120] In some embodiments of this application, the three-dimensional position is calculated by triangulation, the steps of which include:

[0121] Image data from two different locations are acquired using a camera, and the coordinates of corresponding points at these two locations are extracted. Depth information is calculated using the triangle theorem; the formula for calculating the three-dimensional position is:

[0122]

[0123] Where Z represents the depth of the object; B represents the baseline distance between the two viewpoints; f represents the focal length of the camera; and d represents the parallax of corresponding points in the two images.

[0124] Understandably, this embodiment provides a method for calculating the three-dimensional position of an object using the principles of stereo vision. Triangulation is a commonly used stereo vision technique that estimates the depth information of an object by analyzing images taken from two different viewpoints. In this embodiment, a camera is first used to capture images of the scene from two different locations to obtain image data. Then, image processing techniques are used to extract the coordinates of corresponding points in the two images. These corresponding points are the imaging points of the same object from two different viewpoints.

[0125] Next, using the triangle theorem, combined with the camera's intrinsic parameters (such as focal length f), the baseline distance B between the two viewpoints, and the parallax d between corresponding points, the object's depth information Z can be calculated. Depth information Z is crucial for 3D position calculation, helping to determine the object's precise location in 3D space. The calculation formula reflects the relationship between depth information and parallax, focal length, and baseline distance in stereo vision.

[0126] This method allows for the calculation of accurate 3D coordinates for each object in a scene. By combining the real-time object detection capabilities of the YOLOv4 model with the 3D position calculation of triangulation, this application provides a powerful technical solution capable of achieving precise detection and localization of objects in complex scenes.

[0127] In some embodiments of this application, the size of the foreign object is estimated using a contour-based fitting algorithm, calculated as follows:

[0128]

[0129] Where yi represents the actual measurement point; f(xi, θ) represents the fitting function; θ represents the fitting parameter; and n represents the number of data points.

[0130] Understandably, this embodiment provides a method for estimating the size of a foreign object, based on a contour fitting algorithm. The fitting algorithm approximates the actual contour data points yi by selecting a suitable function model f(xi, θ). The fitting parameter θ is determined by minimizing the difference between the actual measured point yi and the fitting function f(xi, θ), typically solved using optimization techniques such as the least squares method. The number of data points n represents the total number of contour points used for fitting.

[0131] The purpose of fitting algorithms is to find a functional model that best describes the shape of an object's profile and uses this model to estimate the object's size. In practical applications, the fitting function can be linear, polynomial, exponential, or other types of functions, depending on the shape and complexity of the object's profile. The model obtained through fitting algorithms can be used to calculate the object's length, width, area, volume, and other dimensional parameters.

[0132] By combining the object detection capabilities of the YOLOv4 model with a contour-based fitting algorithm, this application can not only detect foreign objects in a scene but also estimate their size, providing crucial information for subsequent processing and decision-making. This technical solution has broad application prospects in fields such as quality control, safety monitoring, and automated inspection.

[0133] In some embodiments of this application, real-time monitoring of the temperature and pressure data is achieved through a sensor array, which includes multiple temperature and pressure sensors. The sensor array timestamps the collected status data through a data synchronization module and performs data fusion using a weighted average method.

[0134] Understandably, the use of the sensor array in this embodiment enables simultaneous monitoring of temperature and pressure changes at multiple locations, which is crucial for ensuring the equipment operates at its optimal state. The data synchronization module ensures that data collected by different sensors is recorded synchronously, guaranteeing data consistency over time. The weighted average data fusion technology further improves the accuracy and reliability of data processing; by assigning different weights to data from different sensors, it can more accurately reflect the actual operating status of the equipment.

[0135] See Figure 2 As shown, the handle body (1) is a shell structure for holding and control, and is equipped with power management, imaging signal processing and lighting control circuits. The outer surface of the handle body (1) is provided with ergonomic curved surfaces and operation buttons, which are used to complete control commands such as imaging start, lighting intensity switching and image freezing. The rear side of the handle body (1) is provided with a cable channel for connecting to the display unit.

[0136] Flexible probe (2): It is a slender and flexible component. Its proximal end is connected to the handle body (1), and its distal end is constructed as an observation end facing the wound. The interior of the flexible probe (2) is used to arrange imaging transmission components and lighting guides / wires. The outer surface of the flexible probe (2) can be fitted with a disposable sterile protective sleeve to achieve cross-infection isolation.

[0137] Connection and sealing assembly (3): Located between the handle body (1) and the flexible probe (2), it is used to achieve a detachable or fixed connection between the two and form an axial seal and tensile lock; the connection and sealing assembly (3) may include a quick-connect lock and an annular sealing ring to keep the internal channel clean and gas-liquid sealed when replacing the flexible probe (2) or its disposable protective parts.

[0138] Display unit (4): It is electrically connected to the handle body (1) via a cable and is used to display the wound image acquired by the remote imaging unit of the flexible probe (2) in real time; the display unit (4) can be further used to display the measurement scale, shooting status and power information, and support the feedback of image freeze and data storage instructions.

[0139] Power supply / data interface (5): Located at the bottom of the handle body (1), it is used to connect an external power source or charging base and serves as a wired transmission port for image data and control commands; the power supply / data interface (5) preferably adopts a medical connector structure with waterproof and electrical isolation.

[0140] In one alternative embodiment, an imaging unit and an annular illumination unit are provided at the distal end of the flexible probe (2). The electrical connection between the imaging unit and the illumination unit is connected to the display unit (4) via the internal wires / light guides of the flexible probe (2), the connection and sealing assembly (3), and the circuit in the handle body (1). The connection and sealing assembly (3) is used to limit the axial displacement of the flexible probe (2) relative to the handle body (1) and allow rotation of a limited amplitude, thereby completing the fine adjustment of the incident angle while maintaining a stable distance between the observation end and the surface. The power supply / data interface (5) is used to supply power to the internal circuit of the handle body (1) and to export the acquired image data to an external device for storage and archiving when needed.

[0141] The above structure enables non-contact visual observation of deep or narrow wounds through the separate connection between the flexible probe (2) and the handle body (1) and the external display of the display unit (4); it enables rapid replacement and sealing protection through the connection and sealing components (3); and it enables stable power supply and data retention through the power supply / data interface (5), making it suitable for application scenarios such as pre-hospital emergency care, ward dressing change and remote consultation.

[0142] The flexible probe 2 is formed by nesting multiple coaxially arranged branch pipes 21. The outer diameter of each branch pipe 21 decreases sequentially from the proximal end to the distal end, and the inner diameter forms a clearance fit with the outer diameter of the next branch pipe 21. A low-friction bushing or a low-friction coating is inserted into the clearance fit to reduce expansion resistance and improve reciprocating life.

[0143] Each branch pipe 21 has an annular positioning step on its proximal outer circle and a corresponding limiting shoulder on its distal inner circle. In the maximum extended state, the limiting shoulder abuts against the positioning step to prevent the branch pipe 21 from coming out. In the minimum retracted state, the proximal stop ring abuts against the near end cap to prevent excessive retraction from causing jamming.

[0144] A keyway anti-rotation fit or a flat guide section is set between adjacent branch pipes 21 to suppress the visual deviation caused by relative torsion; the wall thickness and material elastic modulus of the flexible probe 2 are designed according to the bending stiffness grade to ensure that the bending radius and bending resistance remain stable when the extension length changes.

[0145] The outermost distal branch tube 21 of the flexible probe 2 is provided with a fixed bending section (the arc transition section shown in the figure). The bending radius and bending angle of the bending section are used to form a stable incident angle when close to the tissue surface, reducing reflection and facilitating observation of the wound edge and depth.

[0146] Multiple miniature O-rings or hydrophilic-oleophobic composite lip seals are installed at the mating parts of adjacent branch pipes 21 to prevent body fluids and flushing fluids from entering the nested gaps; the outermost layer of the flexible probe 2 can be fitted with a disposable sterile sheath, and an anti-fog layer or hydrophilic coating is set between the front end of the sheath and the window of the imaging unit 22 to reduce condensation and fogging.

[0147] The imaging unit 22 is fixed to the end of the distal branch pipe 21. The imaging unit 22 includes an imaging sensor assembly, an objective lens assembly, and a transparent protective window. The protective window can be made of sapphire glass or tempered glass, with an anti-scratch coating on the outer surface and an anti-fog treatment on the inner surface.

[0148] The objective lens focal length and image plane position of the imaging unit 22 are configured according to a fixed working distance, which is limited by the limiting structure of the flexible probe 2 and the geometry of the distal curved section. In another embodiment, the imaging unit 22 is equipped with a miniature electric or mechanical focusing structure, and the focusing command is transmitted through the cable in the connection and sealing assembly 3 and the handle body 1.

[0149] Display unit 4 is used to display the image output by imaging unit 22 in real time. Scale lines 41 are set on the visible area of ​​display unit 4. The scale lines 41 are regular grids or equidistant scale lines, and the spacing between the scale lines corresponds one-to-one with the pixel-length conversion factor.

[0150] Display unit 4 undergoes one or more point geometric calibrations at the factory:

[0151] (1) Under the standard working distance, establish the conversion relationship from pixel size to length using a standard size target;

[0152] (2) For different extension lengths of the flexible probe 2, record the corresponding conversion coefficients and write them into the calibration table of the display unit 4;

[0153] (3) When the operator changes the extension length, the display unit 4 automatically switches the numerical marking of the scale line 41 according to the displacement sensor feedback in the handle body 1 or according to the scale selection command, so that the displayed reading matches the actual size.

[0154] Display unit 4 supports frozen screen and on-screen point measurement function: In the frozen state, the operator can read the wound length, rift width or foreign object outer diameter with reference to the scale line 41; the reading can be stored along with the image and a timestamp and extension length label can be generated, which is convenient for medical record archiving and retest comparison.

[0155] The image signal from the imaging unit 22 enters the image processing circuit of the handle body 1 via the transmission cable inside the flexible probe 2 and the connection and sealing assembly 3; the processed video signal is output to the display unit 4 via a wired connection. The operation buttons on the handle body 1 are used to control the extension length recording, illumination intensity, and image freeze; power supply and data export are completed through the bottom interface 5.

[0156] Operation process and application scenarios: The operator selects the extension length of the flexible probe 2 according to the location and depth of the wound, and confirms that the multi-section branch tube 21 is in place; the imaging unit 22 is aligned with the observation area, and the fixed curved section is used to achieve a suitable incident angle.

[0157] The display unit 4 synchronously displays the screen and overlays the scale lines 41; when size evaluation is required, the screen is frozen and the length or area estimation result is read according to the scale lines 41; when high precision is required, the corresponding calibration range for the extended length is selected to ensure accurate scale conversion.

[0158] After observation is completed, retract the flexible probe 2 to its minimum length, remove and discard the disposable sheath, and clean and disinfect the exposed parts as required. If replacement or maintenance is required, the distal branch tube 21 or imaging unit 22 can be replaced by disassembling the connection and sealing assembly 3.

[0159] By adopting a flexible probe 2 structure with "multi-section nested branch tube 21 + limiting and guiding cooperation", the effective length and incident angle can be continuously adjusted without changing the position of the handle body 1, reducing the need for movement operations on the injured person; a scale line 41 matching the calibration of the imaging unit 22 is set on the display unit 4 to achieve rapid and verifiable reading of the wound size, improving the consistency of records and the comparability of review; the protective and anti-fog design of the remote imaging unit 22 improves the image stability of long-term observation.

[0160] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program goods. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program goods embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0161] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program goods according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0162] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0163] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0164] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the claims of the present invention.

Claims

1. An AI-based identification of a scalable endoscope for a wound and a monitoring method, characterized by, The application relates to a method for identifying and classifying foreign matter in a wound, comprising the following steps: Collecting image data and state data in the wound by high-resolution endoscopy, wherein the image data comprises visible light images and multispectral images, and the state data comprises temperature data and pressure data in the wound; Using a Gaussian filter algorithm to remove noise from the image data, and then using an edge detection algorithm to extract the contours of the wound and the foreign matter, wherein the edge detection algorithm comprises Canny edge detection and a Sobel gradient operator; Fusing the image data and the state data to form a temperature heat map and a pressure heat map, wherein the data fusion is performed by a weighted average method; Identifying and classifying the foreign matter by an AI model; based on the contours of the wound and the foreign matter, estimating the three-dimensional position and size of the foreign matter; Generating a report based on the temperature heat map, the pressure heat map, the three-dimensional position and the size, and outputting the report. 2.The AI-based identification and monitoring method of a scalable endoscope for a wound according to claim 1, wherein, The calculation formula of the Gaussian filter is as follows: Wherein, G(x, y) represents the Gaussian filter kernel; x and y represent the horizontal and vertical coordinates of the pixels in the image; sigma represents the standard deviation of the Gaussian filter, which is used to control the smoothing degree of the filter. 3.The AI-based identification and monitoring method of a scalable endoscope for a wound according to claim 1, wherein, The steps of the Canny edge detection and the Sobel gradient operator include: Performing grayscale processing on the denoised image to obtain a grayscale image; Using a Sobel operator to perform convolution operation on the image to calculate the gradients of the image in the x direction and the y direction, and the calculation formula is as follows: Wherein, Gx and Gy represent the gradients of the image in the x direction and the y direction; I(i, j) represents the grayscale value of the pixels in the image; Sx(i, j) and Sy(i, j) represent the convolution kernels of the Sobel operator in the x direction and the y direction; Applying a double-threshold algorithm for edge detection, and the calculation formula is as follows: Wherein, intermediate represents the pixel points whose gradient values are between the low threshold and the high threshold; high threshold and low threshold represent the set edge intensity thresholds. 4.The AI-based identification and monitoring method of a scalable endoscope for a wound according to claim 1, wherein, The generation of the temperature heat map and the pressure heat map is realized by the weighted average method, and the specific calculation formula is as follows: T fusion (x, y) = w T ·T image (x, y) + w P ·P image (x, y); P fusion (x, y) = w T · P image (x, y) + w P · T image (x, y); where T fusion (x, y) represents the fused temperature heat map; P fusion (x, y) represents the fused pressure heat map; w T and w P represent the weight coefficients of the temperature data and the pressure data, respectively; T image (x, y) and P image (x, y) represent the pixel values of the temperature image and the pressure image, respectively. 5.The AI-based identification and monitoring method of a scalable endoscope for a wound according to claim 1, wherein, The AI model is YOL0v4. 6.The AI-based identification and monitoring method of a scalable endoscope for a wound according to claim 5, wherein, The training steps of the YOL0v4 model include: Dividing the image data into grids, and predicting the class probability, boundary box coordinates and confidence of the object in each grid; using a cross-entropy loss function and a square error loss function for joint training, and the calculation formula is as follows: wherein L represents a value of the loss function; L olassification represents a classification loss, i.e., a cross-entropy loss; L looation represents a position loss, i.e., a squared error loss; L oonfidenoe represents a confidence loss, i.e., a confidence loss; and λ1and λ2represent adjustment coefficients, respectively. 7.The AI-based identification and monitoring method of a scalable endoscope for a wound according to claim 6, The calculation formula of the cross-entropy loss is as follows: The calculation formula of the square error loss is as follows: The calculation formula of the confidence loss is as follows: where yi represents a label of a true class; represents a class probability predicted by the model; bi represents a class probability predicted by the model; represents coordinates of a bounding box predicted by the model; ci represents a label of whether a target exists, and ci is 0 or 1; represents a confidence value predicted by the model; i represents an index variable. 8.The AI-based identification and monitoring method of a scalable endoscope for a wound according to claim 1, wherein, The three-dimensional position is obtained by triangulation, and the steps of the triangulation include: Using a camera to obtain image data at two different positions, and extracting the coordinates of the corresponding points at the two different positions; using the theorem of triangle to calculate the depth information, and the calculation formula of the three-dimensional position is as follows: Wherein, Z represents the depth of the object; B represents the baseline distance between the two viewpoints; f represents the focal length of the camera; and d represents the parallax of the corresponding points in the two images. 9.The AI-based identification and monitoring method of a scalable endoscope for a wound according to claim 1, wherein, The size of the foreign matter is estimated by a fitting algorithm based on the contours, and the calculation formula is as follows: Wherein, yi represents the actual measurement point; f(xi, theta) represents the fitting function; theta represents the fitting parameter; and n represents the number of data points. 10.The AI-based identification and monitoring method of a scalable endoscope for a wound according to claim 1, wherein, The real-time monitoring of the temperature data and the pressure data is realized through a sensor array, the sensor array comprising a plurality of temperature sensors and pressure sensors, the sensor array time-stamping the collected state data through a data synchronization module and performing data fusion through a weighted average method.

Images