Cervical lesion prediction method and device based on image processing

Abstract

The application relates to the technical field of image processing, and discloses a cervical lesion prediction method and device based on image processing, which comprises the following steps: imaging the cervical surface through a colposcope to obtain a cervical surface image; analyzing the cervical surface image to locate a lesion area; configuring a pose parameter of an ultrasonic probe according to the position of the lesion area, so that the ultrasonic probe is aligned with the lesion area for scanning to obtain an ultrasonic image; analyzing the ultrasonic image to obtain a tissue infiltration depth and a blood flow resistance index of the lesion area; and predicting cervical lesions according to the tissue infiltration depth and the blood flow resistance index. The cervical lesion prediction method and device based on image processing combine colposcope and ultrasonic in-process detection of cervical lesions, and improve detection accuracy and efficiency.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and device for predicting cervical lesions based on image processing. Background Technology

[0002] Cervical cancer is a common malignant tumor threatening women's health worldwide. Early detection and accurate subtyping are crucial for improving the cure rate. Currently, commonly used clinical methods for cervical cancer screening and lesion detection include cervical cytology, HPV testing, colposcopy, and tissue biopsy. Among these, colposcopy, as a direct imaging method, can effectively display the morphological information of the lesion area, assisting doctors in making a preliminary diagnosis. However, traditional colposcopy can only obtain two-dimensional planar images of the cervical surface, making it difficult to accurately reflect the three-dimensional invasion depth and pathological extent of the lesion tissue within the cervical stroma.

[0003] On the other hand, ultrasound imaging, especially transvaginal ultrasound, is widely used in the field of gynecology and can be used to assess cervical structure and mass size.

[0004] Although colposcopy and ultrasound are both commonly used methods for detecting cervical lesions, current ultrasound and colposcopy are performed separately (colposcopy is used for initial screening, followed by ultrasound for further evaluation). Ultrasound lacks precise spatial registration with colposcopy, making it difficult to accurately correspond the specific location, depth, and extent of lesions, which can easily lead to missed diagnoses or misdiagnoses.

[0005] Furthermore, the ultrasound examination and colposcopy examination are performed separately, resulting in delays between the two examination procedures and making it difficult to make a comprehensive decision in a timely manner. Summary of the Invention

[0006] Therefore, the purpose of this invention is to overcome the problem of false positives and false negatives caused by the difficulty in combining colposcopy and ultrasound in the same process for detecting cervical lesions in the prior art, and to provide a cervical lesion prediction method and device based on image processing, which combines colposcopy and ultrasound in the same process for detecting cervical lesions, thereby improving the accuracy and efficiency of detection.

[0007] In a first aspect, to solve the above-mentioned technical problems, the present invention provides a cervical lesion prediction method based on image processing, comprising,

[0008] The cervical surface is imaged using a colposcope to obtain an image of the cervical surface.

[0009] The cervical surface image was analyzed to locate the lesion area;

[0010] The position parameters of the ultrasound probe are configured according to the location of the lesion area, so that the ultrasound probe is aligned with the lesion area to scan and obtain an ultrasound image;

[0011] The ultrasound images were analyzed to obtain the tissue infiltration depth and blood flow resistance index of the lesion area;

[0012] Cervical lesions are predicted based on the depth of tissue invasion and the blood flow resistance index.

[0013] In one embodiment of the present invention, the cervical surface is imaged using a colposcope to obtain a cervical surface image, including configuring a multispectral colposcope, imaging the cervical surface based on the multispectral colposcope, and simultaneously acquiring white light reflection images, narrow-band imaging blood vessel images, and fluorescent staining images of the cervical surface.

[0014] In one embodiment of the present invention, the cervical surface image is analyzed to locate the lesion area, including feature extraction from the white light reflection image to obtain surface texture features and edge features; feature extraction from the narrow-band imaging vascular image to obtain vascular density features and vascular morphology features; feature extraction from the fluorescently stained image to obtain fluorescence intensity distribution features; normalization and weighted fusion of the surface texture features, edge features, vascular density features, vascular morphology features, and fluorescence intensity distribution features to obtain a fused feature map; threshold segmentation of the fused feature map, and obtaining the lesion area based on the threshold segmentation result.

[0015] In one embodiment of the present invention, the multispectral colposcope is configured including an objective lens and a beam splitter prism group. The objective lens converges reflected light from the cervical surface into the beam splitter prism group. The beam splitter prism group divides the incident light into a first optical path, a second optical path, and a third optical path according to wavelength. An RGB image sensor is configured on the first optical path to acquire white light reflection images of the cervical surface. A CMOS image sensor is configured on the second optical path to acquire narrow-band imaging blood vessel images of the cervical surface. A solid-state image sensor is configured on the third optical path to acquire fluorescent staining images of the cervical surface.

[0016] In one embodiment of the present invention, the pose parameters of the ultrasound probe are configured according to the location of the lesion area, including extracting the two-dimensional pixel coordinates of the lesion area in the image coordinate system; converting the two-dimensional pixel coordinates into three-dimensional spatial coordinates in the spatial rectangular coordinate system according to the imaging geometric parameters of the optical system of the colposcope; and obtaining the pose parameters of the ultrasound probe according to the end coordinates of the ultrasound probe and the three-dimensional spatial coordinates.

[0017] In one embodiment of the present invention, analyzing the ultrasound image to obtain the tissue invasion depth of the lesion area includes using an image segmentation algorithm to extract the boundary of the cervical stromal layer, setting multiple measurement points at uniform intervals within the projection range of the lesion area, calculating the vertical distance from each measurement point to the nearest stromal layer boundary, and taking the maximum value of all measurement point distances as the tissue invasion depth.

[0018] In one embodiment of the present invention, analyzing the ultrasound image to obtain the blood flow resistance index of the lesion area includes performing color Doppler blood flow imaging processing on the ultrasound image to identify all blood flow signals; calculating the peak systolic velocity (PSV) and end-diastolic velocity (EDV) of each blood flow signal; calculating the resistance index RI of each blood flow signal according to the following method: RI = (PSV - EDV) / PSV; and taking the average of the resistance indices of all blood flow signals as the blood flow resistance index.

[0019] In one embodiment of the present invention, identifying the blood flow signal includes: extracting an initial set of blood flow signal pixels through color Doppler blood flow imaging processing; binarizing the initial set of blood flow signal pixels to obtain an initial blood flow signal binary image; denoising the initial blood flow signal binary image to obtain a denoised binary image; performing region growing processing on the denoised binary image and merging adjacent blood flow signals to obtain a set of blood flow connected regions; calculating the pixel area of ​​each blood flow connected region, and selecting blood flow connected regions with pixel areas greater than or equal to a threshold pixel area as valid blood flow regions; identifying blood flow signals for each of the valid blood flow regions to obtain all blood flow signals.

[0020] In one embodiment of the present invention, predicting cervical lesions based on the tissue invasion depth and blood flow resistance index includes performing a logarithmic transformation on the tissue invasion depth to obtain a tissue invasion characteristic index; performing a Z-score normalization transformation on the blood flow resistance index to obtain a blood flow resistance index; and obtaining a cervical lesion risk index S based on the following method:

[0021] S = 0.6 × ln(D + 1) + 0.4 × (1 - Z);

[0022] Where D represents the depth of tissue infiltration; Z represents the standardized result of the blood flow resistance index Z-score.

[0023] Secondly, based on the same inventive concept, the present invention also provides an image processing-based cervical lesion prediction device for implementing the aforementioned image processing-based cervical lesion prediction method, including...

[0024] The colposcope module is used to image the surface of the cervix and obtain an image of the cervical surface.

[0025] The image analysis module is used to analyze the cervical surface image and locate the lesion area;

[0026] A pose adjuster, equipped with an ultrasound probe, is used to adjust the pose parameters of the ultrasound probe so that the ultrasound probe is aligned with the lesion area.

[0027] An ultrasound probe is used to scan the lesion area and obtain an ultrasound image of the lesion area.

[0028] An ultrasound analysis module is used to analyze the ultrasound images to obtain the tissue infiltration depth and blood flow resistance index of the lesion area;

[0029] A lesion prediction unit is used to predict cervical lesions based on the depth of tissue invasion and the blood flow resistance index.

[0030] Compared with the prior art, the above-described technical solution of the present invention has the following advantages:

[0031] The image processing-based cervical lesion prediction method and device described in this invention combine colposcopy and ultrasound for simultaneous detection of cervical lesions, thereby improving detection accuracy and efficiency.

[0032] In this method, the lesion area on the surface of the cervix is ​​located by colposcopy imaging. The position parameters of the ultrasound probe are registered with the location of the lesion area as the scanning target. The ultrasound image containing the depth and structural information of the lesion area is obtained by scanning the lesion area. The cervical lesion is evaluated by combining the tissue invasion depth and blood flow resistance index, which improves the accuracy of detection and significantly reduces the false detection rate and the missed detection rate. In addition, the colposcopy and ultrasound detection are performed in the same process. The time from colposcopy to targeted ultrasound scanning is shortened to less than 2 minutes, avoiding the repetitive actions of traditional "blind scanning" and improving detection efficiency. Attached Figure Description

[0033] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings, wherein...

[0034] Figure 1 This is a flowchart of a cervical lesion prediction method based on image processing in a preferred embodiment of the present invention;

[0035] Figure 2 This is a flowchart of analyzing cervical surface images to locate lesion areas in a preferred embodiment of the present invention;

[0036] Figure 3 This is a structural block diagram of a multispectral colposcope in a preferred embodiment of the present invention;

[0037] Figure 4 This is a flowchart illustrating the process of obtaining the pose parameters of the ultrasonic probe in a preferred embodiment of the present invention.

[0038] Figure 5 This is a structural block diagram of an image processing-based cervical lesion prediction device according to a preferred embodiment of the present invention. Detailed Implementation

[0039] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0040] The purpose of this invention is to solve the problems of false positives, false negatives, and low detection efficiency caused by the difficulty in combining colposcopy and ultrasound in the same process for detecting cervical lesions in the prior art. It provides a cervical lesion prediction method and device based on image processing, which combines colposcopy and ultrasound in the same process to detect cervical lesions, thereby improving detection accuracy and efficiency.

[0041] Example 1: Refer to Figure 1 As shown, this invention discloses an image processing-based method for predicting cervical lesions, including:

[0042] S100. The cervical surface is imaged using a colposcope to obtain an image of the cervical surface.

[0043] S200. Analyze the cervical surface image to locate the lesion area;

[0044] S300. Configure the pose parameters of the ultrasound probe according to the location of the lesion area, so that the ultrasound probe is aligned with the lesion area to scan and obtain an ultrasound image.

[0045] S400. Analyze the ultrasound image to obtain the tissue infiltration depth and blood flow resistance index of the lesion area;

[0046] S500, predict cervical lesions based on the tissue infiltration depth and blood flow resistance index.

[0047] In specific applications, narrow-band imaging (NBI) or fluorescence colposcopy is used to enhance the contrast of lesion areas (such as acetic acid white epithelium and atypical blood vessels). Videos of the dynamic changes of the cervix under acetic acid / iodine solution reaction are recorded, and abnormal areas are extracted using inter-frame difference. A U-Net network is trained to segment the cervical transformation zone, and suspicious lesions (such as bright / dark spots) are identified using the HSV color space thresholding method. A lesion probability heatmap is generated by combining texture features, blood vessel morphology, and boundary irregularity, supporting manual annotation and correction, and outputting the center coordinates and boundaries of the lesion area. A colposcopy-ultrasound coordinate system transformation model is established, and the spatial relationship between the two is calibrated using a calibration plate. The robotic arm is driven to adjust the ultrasound probe according to the lesion area coordinates to ensure that the sound beam is perpendicularly incident on the lesion area. In the ultrasound image, a dynamic programming algorithm is used to track the basement membrane rupture point, calculate the deepest penetration distance of the lesion, and obtain the tissue invasion depth. Peak systolic velocity and end-diastolic velocity are extracted using Doppler spectrum, and the resistance index is calculated. Cervical lesions are assessed by combining the tissue invasion depth and the resistance index.

[0048] The cervical lesion prediction method based on image processing described in this invention combines colposcopy and ultrasound to detect cervical lesions in a single process, thereby improving detection accuracy and efficiency.

[0049] Among them, the lesion area on the surface of the cervix is ​​located by colposcopy imaging. The position and pose parameters of the ultrasound probe are registered with the location of the lesion area as the scanning target. The lesion area is scanned to obtain the depth and structural information of the lesion area. The cervical lesion is evaluated by combining the tissue invasion depth and blood flow resistance index, which improves the detection accuracy and significantly reduces the false detection rate and false negative rate.

[0050] In addition, colposcopy and ultrasound are performed in the same procedure, reducing the time from colposcopy to targeted ultrasound scanning to less than 2 minutes, avoiding the repetitive actions of traditional "blind scanning" and improving detection efficiency.

[0051] By combining "optical performance + deep function" in a multimodal synergy, it can achieve accurate stratified prediction of cervical lesions, which is especially suitable for the dual needs of rapid screening in primary hospitals and detailed assessment in tertiary hospitals.

[0052] Based on the above embodiments, the cervical surface is imaged using a colposcope to obtain a cervical surface image. This includes configuring a multispectral colposcope, imaging the cervical surface using the multispectral colposcope, and simultaneously acquiring white light reflection images, narrow-band imaging vascular images, and fluorescent staining images of the cervical surface.

[0053] In specific application scenarios, the hardware configuration of a multispectral colposcope includes:

[0054] The three-channel optical system integrates white LED (400-700nm), narrowband blue light (415nm) and green light (540nm) light sources, as well as a fluorescence excitation module (such as 405nm excitation light + fluorescent dye).

[0055] Time-sharing / optical-sharing synchronous acquisition:

[0056] Time-sharing mode: By switching the filter wheel at high speed (<100ms interval), white light reflection (RGB) images, narrow-band imaging (NBI) and fluorescence images are captured sequentially.

[0057] Beam splitting mode: A three-in-one prism beam splitter is used to simultaneously project light of different wavelengths onto three sets of image sensors to achieve true real-time synchronous imaging. Synchronous imaging means that three-modal data can be obtained in one exposure (traditional layer inspection takes 3-5 minutes, which is shortened to within 30 seconds).

[0058] Fluorescent staining aid: Before detection, spray fluorescent agents such as 5-aminolevulinic acid (5-ALA). Cancerous areas (such as hypermetabolic cells) show red fluorescence under blue light excitation.

[0059] Hardware trigger signals ensure time alignment of the three images, preventing misalignment caused by cervical movement.

[0060] White light imaging uses broadband white light to illuminate the surface of the cervix, and the captured reflected images provide natural-color anatomical images for observing superficial features such as mucosal color, borders, and erosion.

[0061] Narrow-band imaging enhances the contrast of microvessels and glandular structures on the mucosal surface by using specific narrow-band spectra (415nm blue light and 540nm green light). 415nm blue light shows superficial capillaries, while 540nm green light shows slightly deeper vascular networks.

[0062] Fluorescent staining imaging utilizes the selective binding properties of specific fluorescent dyes to cervical lesions. Under blue light or near-infrared light excitation, the lesion area emits fluorescent signals, thereby providing information on metabolic activity.

[0063] White light is sensitive to superficial structures (such as erosion), narrow band imaging (NBI) enhances the detection of vascular abnormalities, and fluorescence indicates metabolically active areas. The combination of these three factors results in a detection rate of cervical intraepithelial neoplasia grade 2 or higher (CIN2+) greater than 95%. Negative fluorescence can rule out false positives due to inflammation.

[0064] By using a three-dimensional imaging approach encompassing structure, blood vessels, and metabolism, this method addresses the issues of traditional colposcopy relying on subjective experience and missing minute invasive lesions, making it particularly suitable for rapid screening and precision medicine scenarios in primary hospitals.

[0065] Based on the above embodiments, referring to Figure 2 As shown, the cervical surface image is analyzed to locate the lesion area. This includes feature extraction from the white light reflection image to obtain surface texture and edge features; feature extraction from the narrow-band imaging vascular image to obtain vascular density and vascular morphology features; feature extraction from the fluorescently stained image to obtain fluorescence intensity distribution features; normalization and weighted fusion of the surface texture, edge, vascular density, vascular morphology, and fluorescence intensity distribution features to obtain a fused feature map; threshold segmentation of the fused feature map; and obtaining the lesion area based on the threshold segmentation result.

[0066] In specific application scenarios, the white light reflection image is preprocessed to adjust brightness and contrast, and noise reduction is performed to reduce image noise. Then, image analysis tools are used to identify the fine texture of the cervical surface. For example, the gray-level co-occurrence matrix method is used to statistically analyze the relationship between different brightness levels in the image, thereby analyzing the fine structure of the cervical surface tissue and obtaining texture features. Then, edge detection algorithms, such as the Canny algorithm or the Sobel algorithm, are used to find the boundaries with obvious brightness changes in the image and obtain edge features.

[0067] The NBI mode uses light of a specific wavelength to highlight superficial blood vessels, making the vascular network on the cervical surface more clearly visible. The acquired NBI image first undergoes vascular enhancement processing. Specific methods include using filtering algorithms (such as the Frangi vascular enhancement filter) to highlight vascular regions, suppress background tissue, and make the vascular structure more prominent; adjusting image contrast and brightness to further highlight small blood vessels. Next, an image segmentation algorithm is used to process the enhanced image, separating the blood vessels from the surrounding tissue. This includes setting pixel grayscale thresholds, separating blood vessels from the background based on threshold segmentation, outlining the blood vessels based on edge detection and region growing, and obtaining vascular density features by statistically analyzing the total length, number, or area occupied by blood vessels per unit area after segmenting the vascular region. Morphological analysis is then performed on the segmented vascular region: calculating the number of branches, branch angles, curvature, and other morphological features of the blood vessels; determining whether there are abnormal morphologies such as uneven thickness or disordered direction of the blood vessels; and obtaining vascular morphological features based on the morphological analysis results.

[0068] Fluorescent images of the cervical surface were acquired under specific excitation light. The images were then denoised and normalized to eliminate interference caused by uneven illumination or staining, resulting in clearer and more uniform images. A brightness threshold was set; pixels with brightness exceeding the threshold were classified as "abnormal areas," while the rest were considered normal areas, separating areas with strong fluorescence signals (bright spots) from the background. Within the segmented abnormal areas, the following features were extracted to obtain fluorescence intensity distribution characteristics: average fluorescence intensity: the average brightness of the entire lesion area; maximum / minimum intensity: identifying the fluorescence intensity of the brightest and darkest points; intensity distribution uniformity: statistically analyzing the amplitude of brightness variation within the area to determine if the fluorescence signal is evenly distributed; area ratio: calculating the proportion of high-intensity areas to the entire cervical surface.

[0069] Different features (such as texture, edge, blood vessel density, blood vessel morphology, and fluorescence intensity) may have completely different value ranges and scales. In order to make a unified comparison, normalization processing is required first: each feature value is converted to between 0 and 1 according to its maximum and minimum values; for example, if the original value range of a feature is 10 to 100, after normalization, 10 becomes 0, 100 becomes 1, and the remaining values ​​are mapped to between 0 and 1 proportionally.

[0070] Based on empirical data, each feature is assigned a weight (i.e., an importance coefficient), with the sum of the weights being 1. For example, surface texture features have a weight of 0.2, edge features have a weight of 0.2, blood vessel density has a weight of 0.2, blood vessel morphology has a weight of 0.2, and fluorescence intensity has a weight of 0.2 (each accounting for 20%); or, depending on actual needs, one feature may have a higher weight, such as fluorescence intensity having a weight of 0.3, and the others having a weight of 0.175, etc.; the weights can be optimized through statistical analysis of a large number of cases, or they can be adjusted by doctors based on experience.

[0071] The normalized features are weighted and superimposed according to their corresponding weights. After the calculation is completed, a "fusion feature map" of the same size as the original image is obtained. On this map, the higher the color or gray value, the more likely the area is to be a lesion after combining multiple features.

[0072] By normalizing and fusing multimodal features from white light, narrow-band imaging, and fluorescence staining images, the complementary information reflected by different imaging modalities on lesion tissue can be fully utilized, significantly improving the accuracy of lesion identification and localization. Feature complementarity under different modalities can effectively suppress interference and errors under a single modality, reducing the rate of missed and false detections caused by image noise, imaging conditions, and other factors, thereby improving the reliability of cervical lesion detection.

[0073] Based on the above embodiments, the multispectral colposcope is configured including an objective lens and a beam splitter prism group. The objective lens converges the reflected light from the cervical surface into the beam splitter prism group. The beam splitter prism group divides the incident light into a first optical path, a second optical path, and a third optical path according to wavelength. An RGB image sensor is configured on the first optical path to acquire white light reflection images of the cervical surface. A CMOS image sensor is configured on the second optical path to acquire narrow-band imaging blood vessel images of the cervical surface. A solid-state image sensor is configured on the third optical path to acquire fluorescent staining images of the cervical surface.

[0074] In specific application scenarios, refer to Figure 3 As shown, a high-quality objective lens assembly is installed at the tip of the colposcope to focus light onto the cervical surface, efficiently converging various reflected or emitted light rays to ensure that the subsequent optical system obtains a clear and bright image. Following the objective lens, a multi-layer beam splitter prism assembly is configured. The beam splitter prism assembly can precisely separate multiple light paths according to different wavelengths (colors):

[0075] First optical path: mainly used for the passage of visible light (RGB, red, green, and blue);

[0076] Second optical path: targeting specific narrowband wavelength regions, such as the blue-green band used to highlight vascular structures;

[0077] The third optical path: specifically separates fluorescence signals for the special excitation and emission bands used in fluorescence staining imaging.

[0078] A high-sensitivity RGB image sensor is configured in the first optical path. This sensor captures ordinary visible light images of the cervical surface, clearly recording tissue structure and surface condition. A CMOS image sensor optimized for specific wavelengths is configured in the second optical path to acquire images with enhanced blood vessels under narrow-band imaging, highlighting the distribution and morphology of microvessels. A dedicated solid-state image sensor is configured in the third optical path. This sensor has high sensitivity to fluorescence wavelengths (such as green or red fluorescence), enabling efficient capture of fluorescence signals from lesion areas. The three sets of image sensors simultaneously acquire three different types of image data from the same cervical region, transmitting them to the back-end analysis system via a high-speed data interface, achieving simultaneous acquisition and processing of multimodal images of the same location.

[0079] By using a beam-splitting prism array, light of different wavelengths from the same location is separated and fed into different sensors, simultaneously acquiring three images: white light, blood vessel, and fluorescence. This eliminates the need to repeatedly switch light sources or lenses, significantly improving image acquisition speed and detection efficiency. The three sets of sensors are synchronously aligned with the same area, ensuring natural registration (alignment) of the acquired multimodal images. This guarantees accurate information overlay during subsequent image fusion analysis, avoiding misjudgments caused by image misalignment and improving the accuracy of lesion localization and diagnosis.

[0080] Based on the above embodiments, the pose parameters of the ultrasound probe are configured according to the location of the lesion area, with reference to... Figure 4 As shown, it includes extracting the two-dimensional pixel coordinates of the lesion area in the image coordinate system; converting the two-dimensional pixel coordinates into three-dimensional spatial coordinates in the spatial rectangular coordinate system according to the imaging geometric parameters of the optical system of the colposcope; and obtaining the pose parameters of the ultrasound probe according to the end coordinates of the ultrasound probe and the three-dimensional spatial coordinates.

[0081] In specific application scenarios, an optical tracking system and a robotic arm system are configured. The optical tracking system uses an infrared optical positioning device (such as NDI Polaris), and reflective marker balls are installed on the colposcope body and ultrasound probe to capture their spatial position and attitude in real time. The robotic arm system uses a six-degree-of-freedom collaborative robotic arm (such as UR5e), with an ultrasound probe mounted at its end. The colposcope imaging system is equipped with a high-resolution digital camera, and the lens needs to be pre-calibrated using a checkerboard pattern to obtain the intrinsic parameter matrix, including focal length, principal point coordinates, and distortion coefficients. The ultrasound probe is a high-frequency linear array probe, and an optical marker ball is fixed to the probe handle to ensure that the geometric relationship between the marker point and the probe scanning plane is known.

[0082] Converting 2D pixel coordinates to 3D spatial coordinates:

[0083] Image coordinate system: with the top left corner of the image as the origin, the u-axis to the right and the v-axis to the bottom;

[0084] Camera coordinate system: with the optical center of the colposcope lens as the origin, and the Z-axis along the optical axis;

[0085] World coordinate system: A global coordinate system defined by the optical tracking system.

[0086] The largest connected component is extracted from the binary mask generated by the fused feature map, and the two-dimensional pixel coordinates (u, v) of the geometric center point of this connected component are calculated. Using colposcope intrinsic parameters (including focal length, principal point coordinates, and distortion coefficients), the pixel coordinates (u, v) are converted into three-dimensional ray directions in the camera coordinate system. Combined with colposcope extrinsic parameters (rotation matrix and translation vector) provided by the optical tracking system, the ray in the camera coordinate system is converted to the world coordinate system. Based on the preset initial value of lesion depth (usually set to 15 mm), the three-dimensional coordinates (X, Y, Z) of the lesion center in the world coordinate system are calculated. Based on the scanning plane geometric parameters of the ultrasound probe (the fixed relationship between the probe marker point and the scanning plane is known), the position and orientation of the probe are calculated to ensure that its scanning plane accurately passes through the lesion center point. The orientation angles (θx, θy, θz) are solved through spatial geometric relationships to ensure that the probe scanning plane is aligned with the normal vector of the lesion region.

[0087] By precisely locating lesions on vaginal images into actual space, the ultrasound probe can be accurately aligned and targeted, greatly improving the accuracy of detecting lesion infiltration depth and blood flow parameters. The deep integration of colposcopy and ultrasound imaging enables comprehensive, one-stop detection from surface images to internal structures, providing clinicians with more complete and objective information on cervical lesions and optimizing screening and diagnostic processes.

[0088] Based on the above embodiments, the ultrasound image is analyzed to obtain the tissue infiltration depth of the lesion area. This includes using an image segmentation algorithm to extract the boundary of the cervical stromal layer, setting multiple measurement points at uniform intervals within the projection range of the lesion area, calculating the vertical distance from each measurement point to the nearest stromal layer boundary, and taking the maximum value of all measurement point distances as the tissue infiltration depth.

[0089] In specific application scenarios, before analyzing the depth of tissue invasion, the ultrasound images are first preprocessed. This includes using methods such as Gaussian filtering and mean filtering to remove noise from the images and avoid noise interfering with boundary extraction. Histogram equalization or CLAHE (contrast-limited adaptive histogram equalization) is then used to enhance the contrast in the images, especially the contrast between the stromal layer and the lesion area. A U-Net network is then used to segment the preprocessed ultrasound images, obtaining three types of segmentation maps: epithelial layer, stromal layer, and lesion area. The lesion area is determined to be within the projection range of the stromal layer boundary. A minimum bounding rectangle ROI (with the long side along the cervical axis) is generated. A measurement line is set every 0.5 mm along the long side of the ROI (corresponding to a 10-pixel spacing in the ultrasound image). A measurement point is taken every 0.3 mm along each measurement line (6-pixel spacing). Example: 5mm × 3mm lesion area → 11 measurement lines × 11 points = 121 measurement points. For each measurement point, the stromal layer boundary is searched along the normal direction (the location where the gray value suddenly drops > 30 HU). The Bresenham algorithm is used to achieve fast linear traversal. The vertical distances of all measurement points {d1, d2, ..., dn} are recorded. The final invasion depth D = max(di) × calibration coefficient (probe frequency related, = 0.98 at 10MHz).

[0090] Based on the above embodiments, the ultrasound images are analyzed to obtain the blood flow resistance index of the lesion area. This includes performing color Doppler blood flow imaging processing on the ultrasound images to identify all blood flow signals; calculating the peak systolic velocity (PSV) and end-diastolic velocity (EDV) of each blood flow signal; calculating the resistance index RI of each blood flow signal according to the following method: RI = (PSV - EDV) / PSV; and taking the average of the resistance indices of all blood flow signals as the blood flow resistance index.

[0091] In specific applications, color Doppler flow imaging (CFM) is performed on ultrasound images. CFM processing generates a color-coded representation of blood flow in the image based on the velocity and direction of blood flow. During this process, the velocity and direction of blood flow are encoded with different colors (e.g., red indicates blood flowing towards the probe, blue indicates blood moving away from the probe), making the blood flow signal more prominent in the image. Colored regions in the image are identified, and blood flow signals are separated. Blood flow signals typically appear within blood vessels, distinguishing them from surrounding static tissue. For each identified blood flow signal, its time-velocity curve is further analyzed, calculating two key points of the flow velocity signal: peak systolic velocity (PSV) and end-diastolic velocity (EDV). PSV represents the highest velocity of the blood flow signal during cardiac systole, and the highest point is automatically identified through the detection of the blood flow velocity waveform. EDV represents the blood flow velocity at the end of cardiac diastole; the end of diastole is identified to obtain the velocity value at that moment. For each identified blood flow signal, the resistance index RI is calculated based on RI = (PSV - EDV) / PSV, reflecting the resistance characteristics of blood flow. Areas with high blood flow resistance are usually associated with lesions (such as tumors, vascular abnormalities, etc.). For all detected blood flow signals, their resistance indices RI are calculated, and the average of the resistance indices of all signals is taken. This average is the blood flow resistance index of the lesion area, used to assess the hemodynamic characteristics of that area. By accurately calculating PSV, EDV, and RI, the resistance characteristics of blood flow can be quantitatively analyzed. Color Doppler images are generated in real time, and changes in blood flow velocity and resistance index can be dynamically observed, enabling physicians to obtain blood flow information instantly during actual operations and quickly determine lesion progression.

[0092] Based on the above embodiments, identifying the blood flow signal includes: extracting an initial set of blood flow signal pixels through color Doppler blood flow imaging; binarizing the initial set of blood flow signal pixels to obtain an initial blood flow signal binary image; denoising the initial blood flow signal binary image to obtain a denoised binary image; performing region growing on the denoised binary image and merging adjacent blood flow signals to obtain a set of blood flow connected regions; calculating the pixel area of ​​each blood flow connected region and selecting blood flow connected regions with pixel areas greater than or equal to a threshold pixel as valid blood flow regions; and identifying blood flow signals for each valid blood flow region to obtain all blood flow signals.

[0093] In specific applications, color Doppler technology is used to process ultrasound images, generating color-coded images of blood flow velocity and direction. The color of each pixel represents the velocity and direction of blood flow, typically using red and blue to indicate flow towards and away from the probe, respectively. Pixels in all blood flow signal regions are extracted by setting an appropriate threshold. Blood flow signals are usually bright areas representing dynamic changes in blood flow; the set of these pixels forms the initial blood flow signal pixel set. Setting an appropriate threshold, pixels below this threshold are classified as background, and pixels above the threshold are classified as foreground (blood flow signal). This effectively reduces interference from low-signal areas, making the blood flow signal more prominent, resulting in a preliminary binary image of the blood flow signal. Since the image may contain noise, spurious signals, and other interference, the binarized blood flow signal image needs to be denoised to remove background noise and isolated noise points. The denoised binary image is subjected to region growing to further extract blood flow signal regions and merge adjacent blood flow signal regions to form complete blood flow connected regions. Region growing involves starting from one or more seed points and progressively expanding similar pixels (such as pixels with similar color or grayscale) to the surrounding area until a set similarity threshold or other stopping condition is met. Through region growing, connected regions of blood flow signals can be effectively identified, and scattered small regions can be merged into a large region, ensuring the integrity of the blood flow signal. For each blood flow connected region, the number of pixels it contains (i.e., the pixel area of ​​the region) is calculated. This area represents the size of the blood flow region, further helping to filter valid blood flow signal regions. Based on the pixel area of ​​the blood flow connected regions, a threshold is set, and only blood flow connected regions with an area greater than or equal to this threshold are selected as valid blood flow regions. For each selected valid blood flow region, its blood flow signal features are further calculated, and these blood flow signals are identified as the final valid blood flow signals.

[0094] Color Doppler blood flow imaging processing allows for the extraction of all preliminary blood flow signal pixels from ultrasound images. This process ensures the identification of all blood flow signal regions and provides fundamental data for subsequent processing steps. Region growing is then used to process the denoised binary image, merging adjacent blood flow signal regions to obtain a set of connected blood flow regions. This step ensures complete identification of blood flow signal regions, avoiding missed or false detections due to irregular blood flow signal distribution. In particular, it effectively addresses the problem of discontinuity or dispersion of blood flow signals in the image.

[0095] Based on the above embodiments, cervical lesions are predicted according to the tissue invasion depth and blood flow resistance index, including performing a logarithmic transformation on the tissue invasion depth to obtain tissue invasion characteristic indicators; performing Z-score normalization on the blood flow resistance index to obtain a blood flow resistance index; and obtaining the cervical lesion risk index S based on the following method:

[0096] S = 0.6 × ln(D + 1) + 0.4 × (1 - Z);

[0097] Where D represents the depth of tissue infiltration; Z represents the standardized result of the blood flow resistance index Z-score.

[0098] Tissue invasion depth D is usually a continuous value representing the depth of the lesion area. Since tissue invasion depth has a skewed distribution, logarithmic transformation can make the data more consistent with a normal distribution, making it more suitable for subsequent analysis. Adding 1 to the invasion depth D and performing a natural logarithmic transformation avoids the problems caused by zero and negative values. This can effectively compress large values ​​and amplify small values, facilitating a unified analysis of characteristics across different depth ranges. The blood flow resistance index RI reflects the degree of abnormality in blood flow in the lesion area. Considering that the blood flow resistance index may vary significantly among different patients, Z-score standardization is performed on the blood flow resistance index to ensure its effectiveness in the predictive model.

[0099] Z-score normalization formula:

[0100] ;

[0101] This represents the mean; The standard deviation is represented by 1. Through standardization, the blood flow resistance index is converted into a distribution with a mean of 0 and a standard deviation of 1, making it comparable across different cases.

[0102] Example 2: Based on the same inventive concept, this embodiment of the invention provides an image processing-based cervical lesion prediction device for implementing the image processing-based cervical lesion prediction method, referring to... Figure 5 As shown, the device includes,

[0103] The colposcope module is used to image the surface of the cervix and obtain an image of the cervical surface.

[0104] The image analysis module is used to analyze the cervical surface image and locate the lesion area;

[0105] A pose adjuster, equipped with an ultrasound probe, is used to adjust the pose parameters of the ultrasound probe so that the ultrasound probe is aligned with the lesion area.

[0106] An ultrasound probe is used to scan the lesion area and obtain an ultrasound image of the lesion area.

[0107] An ultrasound analysis module is used to analyze the ultrasound images to obtain the tissue infiltration depth and blood flow resistance index of the lesion area;

[0108] A lesion prediction unit is used to predict cervical lesions based on the depth of tissue invasion and the blood flow resistance index.

[0109] The embodiments of the present invention are based on the same inventive concept as Embodiment 1, and both have the same technical effects, which will not be repeated here.

[0110] The image processing-based cervical lesion prediction method and device described in this invention combine colposcopy and ultrasound for simultaneous detection of cervical lesions, thereby improving detection accuracy and efficiency.

[0111] In this method, the lesion area on the surface of the cervix is ​​located by colposcopy imaging. The position parameters of the ultrasound probe are registered with the location of the lesion area as the scanning target. The ultrasound image containing the depth and structural information of the lesion area is obtained by scanning the lesion area. The cervical lesion is evaluated by combining the tissue invasion depth and blood flow resistance index, which improves the accuracy of detection and significantly reduces the false detection rate and the missed detection rate. In addition, the colposcopy and ultrasound detection are performed in the same process. The time from colposcopy to targeted ultrasound scanning is shortened to less than 2 minutes, avoiding the repetitive actions of traditional "blind scanning" and improving detection efficiency.

[0112] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0113] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 Devices or equipment that provide the functions specified in one or more boxes.

[0114] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including an instruction means device, the instruction means device being implemented in a process Figure 1One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0115] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0116] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A cervical lesion prediction method based on image processing, characterized in that: include, The cervical surface is imaged using a colposcope to obtain an image of the cervical surface. The cervical surface image was analyzed to locate the lesion area; The position parameters of the ultrasound probe are configured according to the location of the lesion area, so that the ultrasound probe is aligned with the lesion area to scan and obtain an ultrasound image; The ultrasound images were analyzed to obtain the tissue infiltration depth and blood flow resistance index of the lesion area; Cervical lesions can be predicted based on the depth of tissue invasion and the blood flow resistance index. The pose parameters of the ultrasound probe are configured according to the location of the lesion area, including extracting the two-dimensional pixel coordinates of the lesion area in the image coordinate system; converting the two-dimensional pixel coordinates into three-dimensional spatial coordinates in the spatial rectangular coordinate system according to the imaging geometric parameters of the optical system of the colposcope; and obtaining the pose parameters of the ultrasound probe according to the end coordinates of the ultrasound probe and the three-dimensional spatial coordinates. Predicting cervical lesions based on the tissue invasion depth and blood flow resistance index includes performing a logarithmic transformation on the tissue invasion depth to obtain tissue invasion characteristic indicators; and performing a Z-score normalization transformation on the blood flow resistance index to obtain blood flow resistance indicators. The cervical lesion risk indicator S was obtained using the following methods: S = 0.6 × ln(D + 1) + 0.4 × (1 - Z); Where D represents the depth of tissue infiltration; Z represents the standardized result of the blood flow resistance index Z-score.

2. The cervical lesion prediction method based on image processing according to claim 1, characterized in that: Imaging the cervical surface using a colposcope to obtain cervical surface images includes configuring a multispectral colposcope, imaging the cervical surface based on the multispectral colposcope, and simultaneously acquiring white light reflection images, narrow-band imaging vascular images, and fluorescent staining images of the cervical surface.

3. The cervical lesion prediction method based on image processing according to claim 2, characterized in that: The cervical surface image was analyzed to locate the lesion area, including, Feature extraction is performed on the white light reflection image to obtain surface texture features and edge features; Feature extraction is performed on the narrow-band imaging vascular image to obtain vascular density features and vascular morphology features; Feature extraction is performed on the fluorescently stained image to obtain the fluorescence intensity distribution characteristics; The surface texture features, edge features, blood vessel density features, blood vessel morphology features, and fluorescence intensity distribution features are normalized and then weighted and fused to obtain a fused feature map; The fused feature map is segmented by a threshold, and the lesion region is obtained based on the threshold segmentation result.

4. The cervical lesion prediction method based on image processing according to claim 2 or 3, characterized in that: The configuration of the multispectral colposcope includes, An objective lens and a beam splitter assembly are configured. The objective lens converges reflected light from the cervical surface into the beam splitter assembly. The beam splitter assembly divides the incident light into a first optical path, a second optical path, and a third optical path according to wavelength. An RGB image sensor is configured on the first optical path to acquire white light reflection images of the cervical surface. A CMOS image sensor is configured on the second optical path to acquire narrow-band imaging blood vessel images of the cervical surface. A solid-state image sensor is configured on the third optical path to acquire fluorescent staining images of the cervical surface.

5. The cervical lesion prediction method based on image processing according to claim 1, characterized in that: Analyzing the ultrasound image to obtain the tissue invasion depth of the lesion area includes using an image segmentation algorithm to extract the boundary of the cervical stromal layer, setting multiple measurement points at uniform intervals within the projection range of the lesion area, calculating the vertical distance from each measurement point to the nearest stromal layer boundary, and taking the maximum value of all measurement point distances as the tissue invasion depth.

6. The cervical lesion prediction method based on image processing according to claim 1 or 5, characterized in that: Analyzing the ultrasound images, the blood flow resistance index of the lesion area is obtained, including, The ultrasound images were processed using color Doppler blood flow imaging to identify all blood flow signals; Calculate the peak systolic velocity (PSV) and end-diastolic velocity (EDV) for each of the blood flow signals; The resistance index RI for each blood flow signal is calculated as follows: RI = (PSV - EDV) / PSV The average of the resistance indices of all blood flow signals is taken as the blood flow resistance index.

7. The cervical lesion prediction method based on image processing according to claim 6, characterized in that: Identifying the blood flow signal includes, The initial set of blood flow signal pixels was extracted by color Doppler blood flow imaging processing; The initial blood flow signal pixel set is binarized to obtain a binary image of the initial blood flow signal; The initial blood flow signal binary image is denoised to obtain a denoised binary image. The denoised binary image is subjected to region growing processing, and adjacent blood flow signals are merged to obtain a set of blood flow connected regions; Calculate the pixel area of ​​each blood flow connected region, and filter the blood flow connected regions whose pixel area is greater than or equal to a threshold pixel as valid blood flow regions; Blood flow signals are identified for each of the effective blood flow regions to obtain all blood flow signals.

8. A cervical lesion prediction device based on image processing, used to implement the cervical lesion prediction method based on image processing as described in any one of claims 1-7, characterized in that: include, The colposcope module is used to image the surface of the cervix and obtain an image of the cervical surface. The image analysis module is used to analyze the cervical surface image and locate the lesion area; A pose adjuster, equipped with an ultrasound probe, is used to adjust the pose parameters of the ultrasound probe so that the ultrasound probe is aligned with the lesion area. An ultrasound probe is used to scan the lesion area and obtain an ultrasound image of the lesion area. An ultrasound analysis module is used to analyze the ultrasound images to obtain the tissue infiltration depth and blood flow resistance index of the lesion area; A lesion prediction unit is used to predict cervical lesions based on the depth of tissue invasion and the blood flow resistance index.

Images