Colposcope image highlight detection removal method and device based on improved Criminisi algorithm

Abstract

The invention discloses a colposcope image highlight detection and removal method and device based on an improved Criminisi algorithm, and belongs to the technical field of image processing. The method comprises the following steps: S1, acquiring an original colposcope image; s2, performing brightness and contrast enhancement on the original colposcope image to obtain an enhanced colposcope image; s3, performing highlight detection on the enhanced colposcope image to obtain a highlight mask image; s4, the contour of the obtained highlight mask image is detected, and all independent highlight areas are found; s5, according to the obtained contour areas from small to large, reading independent highlight areas in sequence, calculating texture complexity, constructing a priority function according to the texture complexity, and calculating edge pixel point repair priorities of the highlight areas; and based on a particle swarm optimization algorithm, adaptively adjusting a search range according to texture complexity, searching to obtain an optimal matching pixel, and repairing each highlight part one by one to obtain a colposcope image after highlight removal.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and specifically to a method and apparatus for detecting and removing highlights in colposcopy images based on an improved Criminisi algorithm. Background Technology

[0002] Colposcopy, as an intermediate step in cervical cancer precancerous screening, plays a crucial role in the prevention and screening of cervical cancer lesions. Colposcopy uses colposcopic images to obtain the cervical transformation zone, which is then analyzed to detect cervical intraepithelial neoplasia and invasive carcinoma. Colposcopy typically involves staining the cervical region with acetic acid and iodine solution. Under the illumination of the instrument's beam, the colposcopic probe magnifies the cervical mucosa to observe minute lesions on the cervical surface. During this process, due to the presence of mucus on the cervical surface and the influence of the staining solution, large areas of specular reflection often appear. From a clinical perspective, specular highlights can distract the doctor from the actual clinical target and easily cause visual fatigue, affecting the accuracy and efficiency of the doctor's judgment. From an image processing perspective, specular highlight areas do not represent the actual color and texture structure of the cervical region, posing challenges to the performance of subsequent computer vision algorithms. Therefore, specular highlight removal from colposcopy images is essential.

[0003] Currently, methods for specular highlight removal in images mainly include filtering-based methods, methods based on duochromatic reflectance models, low-rank + sparse matrix factorization, deep learning methods, and image inpainting methods. Filtering-based methods typically require designing a complex filter to remove specular reflections. The filter parameters need to be manually tuned, cannot be adaptively adjusted for different scenarios, are time-consuming, and their performance deteriorates when dealing with images with large specular areas. Duochromatic reflectance models propose that an image can be decomposed into diffuse and specular reflections. The diffuse reflection part contains the image's texture and color information, while the specular reflection part represents the highlights. By calculating and removing the specular reflection part from the image, an image without specular interference can be obtained. However, this method is not suitable for all specular images. For images with strong highlights, the highlight part is already nearly saturated, making it impossible to extract effective information, and removing the highlights can result in "black holes" in the image. Low-rank + sparse matrix factorization, a popular method for specular highlight removal in medical images, can decompose the sparse result into sharp and important information, and the low-rank result into stable features. However, this method does not work well with large areas of specular reflection. Deep learning methods have seen significant applications in image feature extraction and image enhancement in recent years. However, these methods are influenced by the quantity and quality of the dataset, and network training is time-consuming and requires high-performance equipment. Currently, image restoration algorithms are widely used in various fields such as cultural relic preservation, medical assistance, and image editing. Highlight removal methods for image restoration first detect the highlight areas in the image and then fill them with the most similar regions found in the known datasets. This method can effectively improve image quality and ensure the validity of information, but it also suffers from problems such as an excessively large search range and long computation time.

[0004] For example, the invention patent application with publication number CN119784646A discloses a method and apparatus for removing highlights from endoscopic images, which overcomes the problems of excessively large global search range and mismatch in the prior art. However, this method optimizes the repair order by detecting the contour length. For large-area highlight areas, the repair effect may be limited by the availability of local textures. Furthermore, this method does not provide a detailed description of the processing speed of the algorithm in real endoscopic images.

[0005] For example, the invention patent application with publication number CN119488261A discloses a control method, device, electronic colposcope, medium and equipment for an electronic colposcope. It controls the colposcope lens through multi-directional trigger function keys to reduce the degree of lens shaking and offset. However, this method requires manual directional trigger command information, which requires a certain degree of operation difficulty, and cannot solve the problem of highlights in the captured image. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention proposes a method and apparatus for detecting and removing highlights in colposcopy images based on an improved Criminisi algorithm.

[0007] The objective of this invention can be achieved through the following technical solutions:

[0008] A first aspect of the present invention relates to a method for highlight detection and removal in colposcopy images based on an improved Criminisi algorithm, comprising the following steps:

[0009] S1. Obtain the original colposcopy image;

[0010] S2. Enhance the brightness and contrast of the original colposcopy image to obtain the enhanced colposcopy image;

[0011] S3. Perform highlight detection on the enhanced colposcopy image to obtain a highlight mask image;

[0012] S4. Detect the contours of the acquired specular mask image and find all independent specular regions;

[0013] S5. Based on the obtained contour area from small to large, read the independent highlight regions in sequence and calculate the texture complexity. Construct a priority function based on the texture complexity, calculate the repair priority of the edge pixels of the highlight region, and determine the pixel with the highest priority. Based on the particle swarm optimization algorithm, adaptively adjust the search range according to the texture complexity, search for the best matching pixel, repair each highlight part one by one, and obtain the colposcopic image after removing the highlights.

[0014] This invention combines the particle swarm optimization algorithm with the Criminisi image restoration algorithm, and further improves the Criminisi algorithm to achieve highlight removal from colposcopic images. Specifically, an improved priority function is used to determine the pixel restoration priority for each highlight region, prioritizing restoration from areas with high texture complexity to ensure accuracy. The particle swarm optimization algorithm effectively accelerates the image restoration process by searching for the best matching pixels. The search range is adaptively adjusted based on texture complexity, decreasing the search range in areas of low texture complexity and increasing it in areas of high texture complexity, thus ensuring both restoration effectiveness and speed.

[0015] In this embodiment, the specific steps for enhancing the original colposcopy image in S2 include:

[0016] S21. Convert the original colposcopy image from RGB color space to HSV color space;

[0017] S22. Extract the V channel component, estimate the illumination component in the image using the center wrap function method and remove it to obtain the V channel reflection component after brightness and contrast enhancement.

[0018] S23. Convert the enhanced HSV color space image back to the RGB color space to obtain a colposcopy image with enhanced brightness and contrast.

[0019] In this embodiment, the specific steps for highlight detection in S3 of the brightened colposcopy image include:

[0020] S31. Obtain the enhanced colposcopy image;

[0021] S32. Perform morphological operations on the image, including high-hat filtering and low-hat filtering, and then apply an adaptive gray-scale thresholding method to threshold the processed image to obtain the first binary mask image of the highlight region.

[0022] S33. Use the Sobel operator to perform edge detection on the enhanced colposcopy image to obtain a second binary mask image of the highlight area;

[0023] S34. Perform a bitwise AND operation on the two binary mask images obtained in steps S32 and S33 to obtain the final specular mask image.

[0024] In this embodiment, the specific steps for repairing the highlight areas of the colposcopy image based on the improved Criminisi algorithm in S5 include:

[0025] S51. Convert the colposcopy image from RGB color space to LAB color space;

[0026] S52. Read each independent highlight area in sequence according to the outline area from smallest to largest;

[0027] S53. After reading a highlight region, create a blank binary image that is identical to the original image. Fill the blank binary image with pixels according to the position of the highlight region to obtain a binary mask image containing only the highlight region.

[0028] S54. Calculate the average amplitude of the Scharr response in the local region using the Scharr operator to obtain the texture complexity of the image. Dynamically adjust the weight values ​​based on the texture complexity and calculate the priority function.

[0029] S55. Calculate the edge pixel repair priority of the highlight area according to the improved priority function, and determine the pixel with the highest priority.

[0030] S56. Based on the particle swarm optimization algorithm, the search range is adaptively adjusted according to the texture complexity, the fitness function is minimized, and the best matching pixel is searched.

[0031] S57. Copy the best matching pixel to the unknown pixel in the corresponding target, update the confidence and the edge of the highlight area until the highlight area is completely repaired.

[0032] S58. Repeat the above steps to repair each individual highlight area in turn until the repair is complete;

[0033] In this embodiment, after acquiring the colposcopy image, the illumination component of the image is extracted using a method based on the center-around function. This enhances the brightness and contrast of the image, reduces the impact of uneven illumination on the original colposcopy image, and effectively improves the accuracy of subsequent highlight detection of the image.

[0034] The reflection component extracted in this embodiment is:

[0035] R(x,y)=logS(x,y)-log[G(x,y)*S(x,y)]

[0036] Where R(x,y) and S(x,y) are the estimated reflection and illumination components, respectively, * denotes the image convolution operation, and G(x,y) represents the center wrap function. Generally, the center wrap function is:

[0037]

[0038] Where σ1 is a scale parameter used to control the smoothness of the image.

[0039] In this embodiment, high-hat filtering and low-hat filtering are used to perform morphological operations on the image, highlighting the texture details in the image, reducing the impact of small noise on the image, and effectively improving the accuracy of highlight detection.

[0040] In this embodiment, the high-hat filtering and low-hat filtering are as follows:

[0041]

[0042] L = A - (A·B)

[0043] Where A represents the original image and B represents the structural element. A represents the morphological opening operation performed on the original image, A·B represents the morphological closing operation performed on the original image, and H and L are the images after high-hat filtering and low-hat filtering, respectively.

[0044] In this embodiment, an adaptive grayscale threshold method is used to obtain the first specular mask image, which effectively detects the specular region in the image.

[0045] In this example, the threshold for obtaining the first specular mask image is:

[0046]

[0047] T1 = gray max -α·σ2

[0048] Among them, I gray (x,y) represents the grayscale image of the input image, and L(xy) is the binary output. max This represents the maximum gray value in the image, α is the weight, which is chosen to be 1, and σ2 is the standard deviation of the gray image;

[0049] In this embodiment, the Sobel operator is used to perform edge detection on the image, refine the highlight areas, and ensure that all highlight areas are completely covered.

[0050] In this embodiment, the threshold for obtaining the second specular mask image is:

[0051] T2 = T1 - 20

[0052] In this embodiment, texture complexity calculation is incorporated as an important criterion for dynamically adjusting the priority function weights and the particle swarm search range.

[0053] The texture complexity in this embodiment is:

[0054]

[0055] Where c represents texture complexity, and N are the number of pixels in the x and y directions of the local region, respectively. scharr(x i ) and scharr(y i ) represent the values ​​of the i-th pixel in the x and y directions within this region, respectively;

[0056] In this embodiment, the dynamic weights in the priority function are:

[0057]

[0058] In this embodiment, the priority function is dynamically adjusted by adding texture complexity to increase the priority of areas with higher texture complexity, so as to achieve the effect of prioritizing the repair of these areas.

[0059] In this embodiment, the improved priority function is:

[0060] P(p)=ωC(p)+D(p)

[0061] Wherein, P(p) is the priority weight function; C(p) is the confidence term, used to reflect the reliability of the surrounding area; and D(p) is the data term, used to measure the correlation between the isolines and the normal vector, guiding the repair order.

[0062] In this embodiment, the formula for calculating the confidence term C(p) is as follows:

[0063]

[0064] Ψp is the patch area centered at point p, mask(q) is the mask value of pixel q (1 represents to be repaired, 0 represents known), and C(q) represents the confidence level of point q.

[0065] In this embodiment, the formula for calculating data item D(p) is:

[0066]

[0067] in, The direction of the isolux lines is calculated from the gradient; n p α is the unit normal vector pointing towards the front edge of the target region; α is the normalization constant.

[0068] In this embodiment, the preset conditions for the particle swarm optimization algorithm are: a basic search range of 5k×5k, a patch size of 9, a particle number of 40 according to the parameter requirements of the particle swarm optimization algorithm, acceleration coefficients c1 and c2 of 2, an inertia weight ω of 0.5, random numbers r1 and r2 randomly generated within the range of 0 to 1, and an upper limit velocity v for particle updates. max and lower limit speed v min The values ​​are 3 and -3 respectively, and the number of iterations is 80.

[0069] In this embodiment, the adaptive search range rule of the particle swarm optimization algorithm is as follows:

[0070]

[0071] search_size=c_factor×basic_size

[0072] Where c_factor is the texture complexity factor, search_size is the actual search range size, and basic_size is the basic search range.

[0073] In this embodiment, the optimal pixel match is found by minimizing the fitness function. The fitness function is based on the sum of squared differences to calculate the difference between the target patch and the source patch in the known region. The calculation formula is as follows:

[0074]

[0075] Where SSD is the sum of squared differences, TargetPatch is the target patch, i.e. the area surrounding the pixel to be repaired, and SourcePatch is the area surrounding the candidate matching pixel, which is the same size as the target patch area; c is the channel index, used to distinguish the three channels.

[0076] This embodiment is configured in such a way that it can effectively and accurately remove highlights from images.

[0077] A second aspect of the present invention relates to a computer-readable storage medium storing instructions that, when executed, enable the above-described method for detecting and removing highlights in colposcopy images based on the improved Criminisi algorithm.

[0078] A third aspect of the present invention relates to a portable handheld colposcope device based on an improved Criminisi algorithm, comprising:

[0079] The optical acquisition module is used to acquire colposcopy images;

[0080] The core processing module is equipped with the aforementioned computer-readable storage medium, and is capable of receiving colposcopy images acquired by the optical module, and executing instructions in the storage medium to perform a colposcopy image highlight detection and removal method based on the improved Criminisi algorithm on the colposcopy images.

[0081] In addition, there is a handle interaction module used to stabilize the optical acquisition module to reduce unwanted shaking.

[0082] Preferably, the optical acquisition module includes a magnifying lens and a variable LED light source;

[0083] The magnifying lens is located in the center of the optical acquisition module, and the LED variable light source is distributed in a ring around the lens. The optical acquisition module is connected to the core processing module through the magnetic unit on the back.

[0084] Preferably, the core processing module includes:

[0085] The light source control module is used to control the color and brightness of the LED variable light source;

[0086] The image processing module is used to remove highlights from the acquired colposcopy images;

[0087] A wireless transmission module is used to transmit colposcopy images before and after processing to a mobile device;

[0088] The core processing module is connected to the handle interaction module via a plug-in device.

[0089] Preferably, the controller interaction module includes:

[0090] Stabilizing handle module for stabilizing the lens during shooting;

[0091] The physical button module is used for interaction and to control the core processing module.

[0092] Preferably, the stabilization handle module includes a yaw axis, a roll axis, a pitch axis, and a gyroscope sensor. The gyroscope sensor is used to sense and calculate the change in lens angle and is embedded inside the stabilization handle. The yaw axis, roll axis, and pitch axis work together to control the change in lens angle, thereby stabilizing the lens.

[0093] The beneficial effects of this invention are:

[0094] To address the problem of excessive computation time in traditional image restoration algorithms, the method described in this invention combines particle swarm optimization with the Criminisi algorithm, effectively accelerating the image restoration speed while ensuring the accuracy of the restoration.

[0095] To address the issue of poor restoration performance in large highlight areas during image inpainting, the method described in this invention incorporates texture complexity as a crucial criterion for calculating pixel restoration priority and adjusting the particle swarm optimization (PSO) search range. The weight of regions with higher texture complexity is increased to prioritize their restoration. Furthermore, when using the PSO algorithm to search for the best-matching pixel block, the search range is adaptively adjusted based on texture complexity, ensuring both the accuracy of image inpainting and effectively improving the speed of search and computation.

[0096] To address the image blurring issue caused by environmental factors and physician operation during actual imaging with portable handheld colposcopes, this invention proposes a stabilizer in the handle to effectively improve lens stability and prevent lens shake due to handheld operation. The handheld colposcope device of this invention adopts a modular design, with the optical acquisition module, core processing module, and interactive handle module all detachable, further facilitating portability and effectively reducing maintenance and repair costs. Furthermore, the device can connect to mobile devices via a wireless communication module, allowing users to directly view and obtain image information through their mobile devices, reducing operational difficulty and facilitating timely access to examination information and results. Attached Figure Description

[0097] The invention will now be further described with reference to the accompanying drawings.

[0098] Figure 1 This is a flowchart illustrating the highlight removal method for colposcopy images based on the improved Criminisi algorithm proposed in this invention.

[0099] Figure 2This is a schematic diagram of the colposcopy image highlight removal device based on the improved Criminisi proposed in this invention;

[0100] Figure 3 The images show the results of highlight removal using the method of the present invention; where (a) is a colposcopy image after contrast and brightness enhancement, (b) is the highlight area marked by highlight detection, and (c) is a colposcopy image after highlight removal processing.

[0101] In the diagram: 1. Magnifying glass; 2. LED variable light source; 3. Core processing module; 4. Plug-in device; 5. Gyroscope sensor; 6. Stabilizer pitch axis; 7. Stabilizer roll axis; 8. Stabilizer yaw axis; 9. Physical button module. Detailed Implementation

[0102] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0103] like Figure 1 As shown, in some embodiments of the present invention, a method for highlight removal from colposcopy images based on an improved Criminisi algorithm is disclosed, comprising:

[0104] S1. Obtain the original colposcopy image;

[0105] S2. Enhance the brightness and contrast of the original colposcopy image to obtain the enhanced colposcopy image;

[0106] S3. Perform highlight detection on the enhanced colposcopy image to obtain a highlight mask image;

[0107] S4. Detect the contour of the acquired specular mask image and find all independent specular parts;

[0108] S5. Based on the obtained contour area from small to large, each highlight part is repaired one by one according to the improved Criminisi algorithm to obtain the colposcopy image after removing the highlights.

[0109] This invention combines the particle swarm optimization algorithm with the Criminisi image restoration algorithm, and further improves the Criminisi algorithm to achieve highlight removal from colposcopic images. Specifically, an improved priority function is used to determine the pixel restoration priority for each highlight region, prioritizing restoration from areas with high texture complexity to ensure accuracy. The particle swarm optimization algorithm effectively accelerates the image restoration process by searching for the best matching pixels. The search range is adaptively adjusted based on texture complexity, decreasing the search range in areas of low texture complexity and increasing it in areas of high texture complexity, thus ensuring both restoration effectiveness and speed.

[0110] In this embodiment, the specific steps for enhancing the original colposcopy image in S2 include:

[0111] S21. Convert the original colposcopy image from RGB color space to HSV color space;

[0112] S22. Extract the V channel component, estimate the illumination component in the image using the center wrap function method and remove it to obtain the V channel reflection component after brightness and contrast enhancement.

[0113] S23. Convert the enhanced HSV color space image back to the RGB color space to obtain a colposcopy image with enhanced brightness and contrast.

[0114] The reflection component is:

[0115] R(x,y)=logS(x,y)-log[G(x,y)*S(x,y)]

[0116] Where R(x,y) and S(x,y) are the estimated reflection and illumination components, respectively, * denotes the image convolution operation, and G(x,y) represents the center wrap function. Generally, the center wrap function is:

[0117]

[0118] Where σ1 is a scale parameter used to control the smoothness of the image.

[0119] In this embodiment, the specific steps for highlight detection in S3 of the brightened colposcopy image include:

[0120] S31. Obtain the enhanced colposcopy image;

[0121] S32. Perform morphological operations on the image, including high-hat filtering and low-hat filtering, and then apply an adaptive gray-scale thresholding method to threshold the processed image to obtain the first binary mask image of the highlight region.

[0122] S33. Use the Sobel operator to perform edge detection on the enhanced colposcopy image to obtain a second binary mask image of the highlight area;

[0123] S34. Perform a bitwise AND operation on the two binary mask images obtained in steps S32 and S33 to obtain the final specular mask image.

[0124] The high-hat filtering and low-hat filtering are as follows:

[0125]

[0126] L = A - (A·B)

[0127] Where A represents the original image and B represents the structural element. A represents the morphological opening operation performed on the original image, A·B represents the morphological closing operation performed on the original image, and H and L represent the images after high-hat filtering and low-hat filtering, respectively.

[0128] In step S32 above, the threshold of the first specular mask image obtained by the adaptive grayscale thresholding method is:

[0129]

[0130] T1 = gray max -α·σ2

[0131] Among them, I gray (x,y) represents the grayscale image of the input image, and L(xy) is the binary output. max This represents the maximum gray value in the image, α is the weight, which is chosen to be 1, and σ2 is the standard deviation of the gray image;

[0132] In step S33 above, the Sobel operator is used to perform edge detection on the image, refining the highlight regions to ensure that all highlight regions are completely covered. The threshold for the obtained second highlight mask image is:

[0133] T2 = T1 - 20

[0134] In this embodiment, the specific steps for repairing the highlight areas of the colposcopy image based on the improved Criminisi algorithm in S5 include:

[0135] S51. Convert the colposcopy image from RGB color space to LAB color space;

[0136] S52. Read each independent highlight area in sequence according to the outline area from smallest to largest;

[0137] S53. After reading a highlight region, create a blank binary image that is identical to the original image. Fill the blank binary image with pixels according to the position of the highlight region to obtain a binary mask image containing only the highlight region.

[0138] S54. Calculate the average amplitude of the Scharr response in the local region using the Scharr operator to obtain the texture complexity of the image. Dynamically adjust the weight values ​​based on the texture complexity and calculate the priority function.

[0139] S55. Calculate the edge pixel repair priority of the highlight area according to the improved priority function;

[0140] S56. Based on the particle swarm optimization algorithm, the search range is adaptively adjusted according to the texture complexity to search for the best matching pixel;

[0141] S57. Copy the best matching pixel to the unknown pixel in the corresponding target, update the confidence and the edge of the highlight area until the highlight area is completely repaired.

[0142] S58. Repeat the above steps to repair each individual highlight area in turn until the repair is complete.

[0143] In step S53, the rule for pixel filling in the binary mask image is as follows:

[0144]

[0145] Where l(x,y) is the output binary mask image, I(x,y) represents the input image, and S i This represents the i-th independent highlight region detected;

[0146] In step S54 above, the calculation of texture complexity is added as an important criterion for dynamically adjusting the priority function weight and the particle swarm search range.

[0147] Specifically, the texture complexity is:

[0148]

[0149] Where c represents texture complexity, and N are the number of pixels in the x and y directions of the local region, respectively. scharr(x i ) and scharr(y i ) represent the values ​​of the i-th pixel in the x and y directions within this region, respectively;

[0150] In this embodiment, the weight for dynamically adjusting texture complexity is:

[0151]

[0152] In this embodiment, the priority function is dynamically adjusted by adding texture complexity to increase the priority of areas with higher texture complexity, so as to achieve the effect of prioritizing the repair of these areas.

[0153] The improved priority function is:

[0154] P(p)=ωC(p)+D(p)

[0155] Wherein, P(p) is the priority weight function, C(p) is the confidence term, which reflects the reliability of the surrounding area; D(p) is the data term, which measures the correlation between the isolines and the normal vectors and guides the repair order.

[0156] In this embodiment, the formula for calculating the confidence term C(p) is as follows:

[0157]

[0158] Ψp is the patch area centered at point p, mask(q) is the mask value of pixel q (1 represents to be repaired, 0 represents known), and C(q) represents the confidence level of point q.

[0159] In this embodiment, the formula for calculating data item D(p) is:

[0160]

[0161] in, The direction of the isolux lines is calculated from the gradient; n p α is the unit normal vector pointing towards the front edge of the target region; α is the normalization constant.

[0162] In step S56, the preset conditions for the particle swarm optimization algorithm are: the basic search range is 5k×5k, the patch size k is 9, the number of particles is 40 according to the parameter requirements of the particle swarm optimization algorithm, the acceleration coefficients c1 and c2 are 2, the inertia weight ω is 0.5, the random numbers r1 and r2 are randomly generated in the range of 0 to 1, and the upper limit velocity v of particle update is... max and lower limit speed v min The values ​​are 3 and -3 respectively, and the number of iterations is 80.

[0163] The adaptive search range rule for the particle swarm optimization algorithm is as follows:

[0164]

[0165] search_size=c_factor×basic_size

[0166] Where c_factor is the texture complexity factor, search_size is the actual search range size, and basic_size is the basic search range.

[0167] In this embodiment, the optimal pixel match is found by minimizing the fitness function. The fitness function is based on the sum of squared differences to calculate the difference between the target patch and the source patch in the known region. The calculation formula is as follows:

[0168]

[0169] Where SSD stands for Sum of Squared Differences, TargetPatch is the target patch, i.e., the area surrounding the pixel to be repaired, and SourcePatch is the area surrounding the candidate matching pixel, which is the same size as the target patch area; c is the channel index, used to distinguish the three channels.

[0170] In this embodiment, after acquiring the colposcopy image, the illumination component of the image is extracted using a method based on the center-around function. This enhances the brightness and contrast of the image, reduces the impact of uneven illumination on the original colposcopy image, and effectively improves the accuracy of subsequent highlight detection of the image.

[0171] In some embodiments of the present invention, a computer-readable storage medium is also involved, storing instructions capable of executing the colposcopy image highlight removal method based on the improved Criminisi algorithm described in the above embodiments. The storage medium may be a physical memory capable of storing code or programs.

[0172] Furthermore, some embodiments of the present invention also involve a device configured with the aforementioned storage medium, which, when in operation, is capable of implementing the colposcopy image highlight removal method based on the improved Criminisi algorithm described in the above embodiments, and outputting or displaying the colposcopy image after highlight removal.

[0173] In some embodiments, the device configured with the above-described storage medium may be, but is not limited to, an image processing workstation, a server, a host computer, a laptop computer, a mobile device (such as a tablet computer, a smartphone), etc.

[0174] like Figure 2 As shown, this invention also proposes a portable handheld colposcope device based on an improved Criminisi algorithm, including an optical acquisition module, a core processing module, and an interactive handle module:

[0175] The optical acquisition module is used to acquire colposcopy images;

[0176] The core processing module is equipped with the aforementioned computer-readable storage medium, and can then execute a colposcopy image highlight removal method based on the improved Criminisi algorithm to remove highlights from the acquired colposcopy image.

[0177] The handle interaction module is used to stabilize the lens and simultaneously control the optical acquisition module;

[0178] In this embodiment, the optical acquisition module includes a magnifying lens 1 and an LED variable light source 2; the magnifying lens 1 is located in the center of the optical acquisition module, and the LED variable light source 2 is distributed in a ring around the lens. The optical acquisition module is connected to the core processing module through a magnetic unit on the back.

[0179] In this embodiment, a magnifying glass 1 is used to meet the shooting needs during the process of acquiring colposcopy images. At the same time, the LED variable light source 2 can adjust the warmth and intensity of the light source to meet the lighting needs in different environments and ensure the clarity of the acquired images.

[0180] In this embodiment, the core processing module 3 includes: a light source control module, an image processing module, and a wireless transmission module; the light source control module is used to control the color and brightness of the LED variable light source 2; the image processing module is used to remove highlights from the acquired colposcopy image using the algorithm described in this invention; the wireless transmission module is used to transmit the colposcopy images before and after processing to the mobile device; the core processing module 3 is connected to the handle interaction module through a plug-in device 4.

[0181] The handle interaction module in this embodiment includes a stabilized handle module and a physical button module: the stabilized handle module is used to automatically adjust the lens angle and stabilize the lens during shooting; the physical button module 9 is used for interaction and control of the core processing module 3.

[0182] In a further embodiment, the stabilization handle module includes a yaw axis 8, a roll axis 7, a pitch axis 6, and a gyroscope sensor 5. The gyroscope sensor 5 is used to sense and calculate the change in lens angle and is embedded inside the stabilization handle. The three axes of yaw axis 8, roll axis 7, and pitch axis 6 work together to control the change in lens angle, thereby stabilizing the lens.

[0183] In one specific embodiment, when using the colposcopy image highlight removal method and apparatus based on the improved Criminisi algorithm proposed in this invention for highlight removal, the specific operation process includes:

[0184] S1. Turn on the device and adjust the device light source: Call the light source control module in the core processing module 3 through the physical button module 9 to adjust the temperature and intensity of the LED variable light source 2.

[0185] S2. Adjusting the device status and taking a colposcopy image: First, hold the device and aim it at the shooting area. Use the physical button module 9 to call the magnifying lens 1 to zoom in or out, find the most suitable shooting angle, and then call the magnifying lens to take an image. This image is the original colposcopy image, which will be stored in the core processing module.

[0186] S3. Highlight Removal: The wireless transmission module in the core processing module 3 is invoked, and the original colposcopy image can be viewed through a mobile device. The highlight removal method for colposcopy images based on the improved Criminisi algorithm proposed in this invention is used to remove the highlights in the original colposcopy image.

[0187] S4. Viewing and sharing results: After highlight removal is completed, the processed colposcopy image will be displayed on the mobile device, allowing you to view the images before and after processing.

[0188] Experimental results of highlight removal using the colposcopy image highlight removal method based on the improved Criminisi algorithm described in this invention are as follows: Figure 3 As shown. The experimental images used in this embodiment are medical images acquired by the hospital with the patient's consent.

[0189] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0190] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention.

Claims

1. A method for highlight detection and removal in colposcopy images based on an improved Criminisi algorithm, characterized in that, Includes the following steps: S1. Obtain the original colposcopy image; S2. Enhance the brightness and contrast of the original colposcopy image to obtain the enhanced colposcopy image; S3. Perform highlight detection on the enhanced colposcopy image to obtain a highlight mask image; S4. Detect the contours of the acquired specular mask image and find all independent specular regions; S5. Based on the obtained contour area from small to large, read the independent highlight regions in sequence and calculate the texture complexity. Construct a priority function based on the texture complexity and calculate the repair priority of the edge pixels of the highlight region. Based on the particle swarm optimization algorithm, adaptively adjust the search range according to the texture complexity, search for the best matching pixel, repair each highlight part one by one, and obtain the colposcopic image after removing the highlights.

2. The method for highlight detection and removal in colposcopy images based on the improved Criminisi algorithm according to claim 1, characterized in that, The specific steps for enhancing the brightness and contrast of the original colposcopy image include: S21. Convert the original colposcopy image from RGB color space to HSV color space; S22. Extract the V channel component, estimate the illumination component in the image using the center wrap function method and remove it to obtain the V channel reflection component after brightness and contrast enhancement. S23. Convert the enhanced HSV color space image back to the RGB color space to obtain a colposcopy image with enhanced brightness and contrast.

3. The method for highlight detection and removal in colposcopy images based on the improved Criminisi algorithm according to claim 2, characterized in that, The reflection component is: R(x,y)=logS(x,y)-log[G(x,y)*S(x,y)] Where R(x,y) and S(x,y) are the estimated reflection and illumination components, respectively, * denotes the image convolution operation, and G(x,y) represents the center wrap function. Generally, the center wrap function is: Where σ1 is a scale parameter used to control the smoothness of the image.

4. The method for highlight detection and removal in colposcopy images based on the improved Criminisi algorithm according to claim 1, characterized in that, The specific steps for highlight detection in colposcopy images after brightness enhancement include: S31. Obtain the enhanced colposcopy image; S32. Perform morphological operations on the image, including high-hat filtering and low-hat filtering, and then apply an adaptive gray-scale thresholding method to threshold the processed image to obtain the first binary mask image of the highlight region. S33. Use the Sobel operator to perform edge detection on the enhanced colposcopy image to obtain a second binary mask image of the highlight area; S34. Perform a bitwise AND operation on the two binary mask images obtained in steps S32 and S33 to obtain the final specular mask image.

5. The method for highlight detection and removal in colposcopy images based on the improved Criminisi algorithm according to claim 4, characterized in that, The high-hat filtering and low-hat filtering are as follows: L = A - (A·B) Where A represents the original image and B represents the structural element. A represents the morphological opening operation performed on the original image, A·B represents the morphological closing operation performed on the original image, and H and L represent the images after high-hat filtering and low-hat filtering, respectively.

6. The method for highlight detection and removal in colposcopy images based on the improved Criminisi algorithm according to claim 1, characterized in that, The specific steps for repairing highlight areas in colposcopic images based on the improved Criminisi algorithm include: S51. Convert the colposcopy image from RGB color space to LAB color space; S52. Read each independent highlight area in sequence according to the outline area from smallest to largest; S53. After reading a highlight region, create a blank binary image that is identical to the original image. Fill the blank binary image with pixels according to the position of the highlight region to obtain a binary mask image containing only the highlight region. S54. Calculate the average amplitude of the Scharr response in the local region using the Scharr operator to obtain the texture complexity of the image. Dynamically adjust the weight values ​​based on the texture complexity and calculate the priority function. S55. Calculate the edge pixel repair priority of the highlight area according to the improved priority function; S56. Based on the particle swarm optimization algorithm, the search range is adaptively adjusted according to the texture complexity, the fitness function is minimized, and the best matching pixel is searched. S57. Copy the best matching pixel to the unknown pixel in the corresponding target, update the confidence and the edge of the highlight area until the highlight area is completely repaired. S58. Repeat the above steps to repair each individual highlight area in turn until the repair is complete.

7. The method for highlight detection and removal in colposcopy images based on the improved Criminisi algorithm according to claim 1, characterized in that, The texture complexity is: Where c represents texture complexity, and N are the number of pixels in the x and y directions of the local region, respectively. scharr(x i ) and scharr(y i ) represent the values ​​of the i-th pixel in the x and y directions within this region, respectively; The weights for dynamically adjusting texture complexity are: The improved priority function is: P(p)=ωC(p)+D(p) Where P(p) is the priority function, C(p) is the confidence term, and D(p) is the data term.

8. The method for highlight detection and removal in colposcopy images based on the improved Criminisi algorithm according to claim 1, characterized in that, The preset conditions for the particle swarm optimization algorithm are: a basic search range of 5k×5k, a patch size of 9, a particle number of 40 according to the parameter requirements of the particle swarm optimization algorithm, acceleration coefficients c1 and c2 of 2, an inertia weight ω of 0.5, random numbers r1 and r2 randomly generated within the range of 0 to 1, and an upper limit velocity v for particle updates. max and lower limit speed v min The values ​​are 3 and -3 respectively, and the number of iterations is 80. The adaptive search range rule for the particle swarm optimization algorithm is as follows: search_size=c_factor×basic_size Where c_factor is the texture complexity factor, search_size is the actual search range size, and basic_size is the basic search range.

9. A computer-readable storage medium storing instructions, characterized in that, When the instruction is executed, it can realize the colposcopy image highlight detection and removal method based on the improved Criminisi algorithm as described in any one of claims 1 to 8.

10. A portable handheld colposcope device based on an improved Criminisi algorithm, comprising: The optical acquisition module is used to acquire colposcopy images; The core processing module is equipped with the computer-readable storage medium as described in claim 9, and is capable of receiving colposcopy images acquired by the optical module, and performing a colposcopy image highlight detection and removal method based on the improved Criminisi algorithm on the colposcopy images by executing the instructions in the storage medium. In addition, there is a handle interaction module used to stabilize the optical acquisition module to reduce unwanted shaking.

Images