Water surface inverted image removing method based on K-Means clustering

Abstract

The invention relates to a water surface inverted image removing method based on K-Means clustering, and belongs to the field of computer vision and image processing. The method comprises the following steps: S1, data preparation: extracting a feature vector of each pixel point from an image to form a data set for clustering analysis; s2, clustering and distinguishing an inverted image region and a non-inverted image region in the image by using a K-Means clustering algorithm; s3, creating a mask to remove an inverted image; and S4, image restoration: restoring the image after the inverted image is removed by adopting a Telea algorithm. According to the method, inverted image region division and image restoration can be realized, the detection and segmentation precision of objects on the water surface is improved, and an image analysis result is more accurate and reliable.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of computer vision and image processing, and relates to a method for removing water surface reflection based on K-Means clustering. BACKGROUND

[0002] In the application of computer vision, image analysis of scenes containing water bodies (such as rivers, lakes, oceans or post-rain water) is a common and important task. However, water surface reflection, as a common optical phenomenon, poses a serious challenge to the analysis of such scenes, especially to the target detection task, seriously affecting the accuracy and reliability of the algorithm.

[0003] The core of the target detection algorithm is to accurately locate and identify the preset target in the image, such as ships, pedestrians, vehicles or water surface obstacles, etc. Water surface reflection is highly similar to the real object in visual features, and is easily misidentified as an independent, real target by existing algorithms. The reflection is very close to the source object in color, shape, texture, etc., resulting in a large number of false positives for the detection algorithm, which not only reduces the accuracy of the detection, but also brings potential risks to the subsequent decision and control system. Therefore, before target identification and analysis, image preprocessing to effectively remove water surface reflection is a key step to improve the overall system performance.

[0004] Currently, developing an accurate and effective water surface reflection removal algorithm mainly faces two technical difficulties: 1) High similarity between reflection and entity: the reflection largely replicates the visual features of the real object, including color distribution, internal texture and basic outline. This high degree of similarity makes it difficult for the algorithm to effectively distinguish the reflection from the real object through low-level or intermediate image features (such as color histogram, gradient information, etc.), and the difficulty of distinguishing is much greater than that of general image segmentation tasks.

[0005] 2) Fuzziness and uncertainty of edge and contour: the dividing line between the real object and its water surface reflection, i.e. the water line, is often not clear and sharp. Affected by water surface fluctuations, ripples, changes in lighting and perspective effects, the edges and contours of the reflection often exhibit features of fuzziness, gradual change and even fusion with the real object or background. This uncertainty of the boundary brings great difficulty to the accurate segmentation of the reflection area.

[0006] Currently, the research on water surface reflection removal by the academic and industrial communities is still insufficient, and there are relatively few public technical documents. Existing related research mainly focuses on some specific scenes or assumptions.

[0007] For example, Pan Hanfeng proposed a process for detecting water surface obstacles by removing the reflection area in his doctoral thesis "Research on Water Surface Obstacle Detection Method Based on Reflection Removal" (Harbin Engineering University, 2024). This method combines automatic multi-seed region growing algorithm, mean shift filtering, Laplacian pyramid decomposition and reorganization, and various image processing techniques such as image segmentation, dilation, erosion, and pixel filling in the OpenCV library to achieve reflection area detection and removal. The method is relatively complex, involving the combination and debugging of multiple algorithms, and has relatively high computational cost.

[0008] In addition, some researches, although not directly aimed at water surface reflection, have certain reference value in their shadow processing ideas. For example, Chai Dong proposed a shadow removal algorithm based on spectral features in "Research on Control River Ship Visual Monitoring and Illegal Navigation Detection Method" (Chongqing University, 2012). This algorithm uses the difference in spectral features (especially color and brightness) between target pixels and their shadow area pixels to distinguish them. However, this method is more suitable for scenes where the color difference between the target and the shadow is significant, and its application effect may be limited for reflections and entities with very similar color features.

[0009] In summary, the existing technology has the problems of complex method, high computational cost, or limited application scenarios in dealing with water surface reflection. Therefore, it is urgent to develop a new type of water surface reflection removal method with simple principle, high computational efficiency and strong robustness to meet the application requirements of computer vision systems in complex water environments. SUMMARY

[0010] Therefore, the purpose of the present application is to provide a K-Means clustering-based water surface reflection removal method. In target detection and image segmentation tasks, removing reflections can avoid misjudgment of reflections as targets or interference with target recognition and segmentation, improve the detection and segmentation accuracy of objects on the water surface such as ships and buoys, and make the image analysis results more accurate and reliable. The following two problems are mainly solved: First, reflection area division. The K-means algorithm is used to divide the image pixels into two categories (K = 2), representing the reflection area and the non-reflection area, respectively. Based on the characteristics of image pixels (such as color, brightness, etc.), clustering is performed to distinguish the reflection and non-reflection parts.

[0011] Secondly, image restoration. After removing reflections, blank or incomplete areas often remain in the image. To restore the image to a natural and complete state, effective image restoration processing is required. Here, the Telea algorithm from Inpaint is used to achieve this goal. The Telea algorithm is based on a fast-moving algorithm and can fill in missing areas in a reasonable way based on known information about the surrounding image. It can accurately simulate the texture and structure of the image, allowing the restored area to blend naturally with the surrounding image, ultimately resulting in a visually appealing and complete restored image.

[0012] To achieve the above objectives, the present invention provides the following technical solution: A method for removing water reflections based on K-Means clustering includes the following steps: S1: Data Preparation: Extract the feature vector of each pixel from the image to form a dataset for cluster analysis. These feature vectors will serve as input to the clustering algorithm to determine the category to which each pixel belongs.

[0013] S2: Use the K-Means clustering algorithm to cluster and distinguish between reflection and non-reflection regions in the image; S3: Create a mask to remove the reflection; S4: Image Restoration: After removing reflections using a series of techniques, a large number of blank areas are generated in the image due to the removal of reflections. To restore the image to its natural state and maintain its integrity, this invention uses the Telea algorithm to restore the image after reflection removal. This algorithm meticulously calculates the information diffusion of boundary pixels, smoothly integrating the color, texture, and other features contained in surrounding pixels into the blank areas.

[0014] Furthermore, step S1 specifically includes: extracting the RGB value of each pixel in the image to form a data set. ,in, It is the total number of pixels in the image. Indicates the first The RGB values ​​of each pixel. .

[0015] Furthermore, step S2 specifically includes: randomly selecting two data points from the dataset X as initial cluster centers, denoted as... and ; Calculate the feature vector of each pixel in X and the two cluster centers. and The distance between pixels is typically measured using Euclidean distance; within image pixels, Representing the three RGB channels, for pixels and cluster center , the Euclidean distance between them is:

[0016] Let , if , the pixel is assigned to cluster , otherwise to cluster .

[0017] Each pixel is assigned to the class represented by the cluster center closest to its feature vector, completing a clustering iteration; after completing the assignment of all pixels, the mean of all pixels in the two clusters and is calculated respectively, and is taken as the new cluster center; for cluster , let the set of pixels it contains be , where is the number of pixels in cluster , then the new cluster center is:

[0018] The above steps are repeated until the cluster centers no longer change significantly, i.e., the convergence condition of the algorithm is reached.

[0019] After clustering is completed, the two classes divided are discriminated to discriminate the reflection region and the non-reflection region, specifically: according to the label of each pixel, the corresponding RGB value is obtained from the cluster center to obtain the segmented pixel value, and the RGB value of the first pixel is , , , In the RGB color space, the human visual system has different sensitivities to different colors, and the weight of green is usually the highest, followed by red, and blue is the lowest. Reflection is usually darker than the solid color, and there is a loss in the process of light reflection, so the class with a smaller average brightness value is the reflection;

[0020] where N represents the number of cluster centers; Further, in step S3, creating a mask to remove the reflection specifically includes: creating a mask to mark the reflection region. The reflected region after being distinguished is set to have a pixel label of 0; when creating the mask, the pixel points in the mask that are determined to be the reflection region are set to white. Conversely, the pixel points in the non-reflection region are set to have a label of 1, and are represented by black in the mask.

[0021] Furthermore, in step S4, the Telea algorithm uses the fast travel method to solve the partial differential equations. In the fast travel method, the points in the region to be repaired are divided into three categories: known points (repaired points), undetermined points (points being processed), and unknown points (points to be repaired). The algorithm starts with the known points, gradually updates the values ​​of the undetermined points according to the velocity function, and adds them to the set of known points until all unknown points are repaired.

[0022] For each undetermined point Its value Update using the following formula:

[0023] in, It is a point The neighborhood, Neighboring points The velocity function value; this formula indicates that the point... The value is obtained by weighting the values ​​of its neighboring points according to the velocity function. Through continuous iteration of this process, the area to be repaired is gradually filled until the entire area is repaired.

[0024] The beneficial effects of this invention are as follows: This invention addresses the problem of water reflections affecting target recognition by utilizing the K-Means clustering algorithm and image processing techniques to remove water reflections. Specifically, the following effects are achieved: 1) Optimizing Computer Vision Tasks: In computer vision tasks such as object detection, image segmentation, and image recognition, the presence of reflections can lead to algorithmic misjudgments, identifying reflections as independent targets or interfering with the recognition of real targets. Removing reflections can significantly improve the accuracy and reliability of these tasks, enabling algorithms to more accurately identify and locate target objects, and providing more accurate data support for related applications.

[0025] 2) Enhanced visual effects: After removing the reflection, the main subject of the image stands out more, the picture is simpler and clearer, and the reflection avoids interfering with the subject, allowing the viewer to focus more directly on the key information.

[0026] 3) Promoting the improvement of image processing algorithms: Removing reflections from water surfaces is an important research topic in the field of image processing. Solving this problem requires continuous exploration and improvement of existing image processing algorithms. Research on reflection removal techniques can promote the development and innovation of related algorithms, providing new ideas and methods for other image processing tasks.

[0027] 4) Expanding the scope of application: Although this invention is mainly aimed at removing reflections on the water surface, the technology and methods used are also applicable to other fields that require removal, such as watermark removal, and have broad application prospects.

[0028] Additional advantages, objects, and features of the application will be apparent to those skilled in the art upon examination of the following specification. It is intended that the application not be limited by any of the details of the specification. Instead, the true scope of the application is to be determined by the full width of the claims, along with their equivalents. BRIEF DESCRIPTION OF DRAWINGS

[0029] In order to make the objectives, technical solutions and advantages of the present application clearer, the preferred embodiments of the present application will be described in detail below with reference to the accompanying drawings, in which: Figure 1 Flow chart of the method for removing water surface reflection based on K-Means clustering; Figure 2 Flow chart of K-Means clustering for dividing reflection area and non-reflection area; Figure 3 Schematic diagram for dividing reflection area based on K-Means clustering; Figure 4 Process diagram for removing water surface reflection and restoring the original image. DETAILED DESCRIPTION

[0030] The other advantages and effects of the present application can be easily understood by those skilled in the art from the content disclosed in the specification. The present application can also be implemented or applied by different specific embodiments, and the details in the specification can be modified or changed in different ways without departing from the spirit of the present application. It should be noted that the diagrams provided in the following embodiments only illustrate the basic concept of the present application in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0031] In various scenarios involving image acquisition and analysis, water surface reflection is a ubiquitous and extremely challenging interference source that continues to pose significant obstacles to image processing. The reflection often makes it difficult to distinguish the actual object from the reflection in the picture captured by the camera near the water area, leading to frequent misjudgment and missed judgment, seriously affecting the reliability of the security system. The present application removes water surface reflection based on K-Means clustering, which accurately classifies pixels in the image according to features related to reflection. By constructing a feature vector containing color, texture, gradient, and other information, the K-Means algorithm is used to divide the pixel points into reflection area and non-reflection area. After clearly distinguishing the two areas, subsequent removal operations can be performed on the reflection area. This method can provide pure and high-quality image data for image restoration, target recognition, and other downstream tasks, greatly improving the accuracy and efficiency of image processing and meeting the urgent need for clear and accurate images in different fields.

[0032] Please see Figures 1-4 This invention proposes a method for removing water reflections based on K-Means clustering, such as... Figure 1 As shown, the main contents of this method are as follows: 1) Data Preparation: Extract the feature vector of each pixel from the image to form a dataset for clustering analysis. These feature vectors will serve as input to the clustering algorithm to determine the category to which each pixel should belong. Extract the RGB values ​​of each pixel in the image to form a dataset. ,in, It is the total number of pixels in the image. Indicates the first The RGB values ​​of each pixel. .

[0033] 2) Use the K-Means clustering algorithm to cluster and distinguish between reflection and non-reflection regions: Randomly select two data points from the dataset X as the initial cluster centers, denoted as... and ; Calculate the feature vector of each pixel in X and its two cluster centers. and The distance is measured using Euclidean distance as the metric. Within image pixels, (RGB three channels), for pixels and cluster center , The Euclidean distance between them for:

[0034] set up , ,if Then the pixels Assigned to cluster Conversely, they are assigned to clusters. .

[0035] Each pixel is assigned to the category represented by the cluster center closest to its feature vector, completing one clustering iteration. After all pixels have been assigned, the two clusters are calculated separately. and The mean of all pixels within the cluster is used as the new cluster center. For each cluster... Let its set of pixels be... ,in It is a cluster The number of pixels in the middle determines the new cluster center. for:

[0036] The above steps are repeated until the cluster centers no longer change significantly, i.e., the convergence condition of the algorithm is reached. Then the entire clustering process is completed. The specific process is as shown in Figure 2 .

[0037] After clustering is completed, the two categories divided are discriminated to discriminate the mirroring region and the non-mirroring region. According to the label of each pixel, the corresponding RGB value is obtained from the cluster center to obtain the segmented pixel value. The RGB value of the first pixel is , , , In the RGB color space, the sensitivity of the human visual system to different colors is different, and generally the weight of green is the highest, the weight of red is the second, and the weight of blue is the lowest. The mirroring is usually darker than the color of the entity, and there is a certain loss in the light reflection process, so the category with a smaller average brightness value is the mirroring.

[0038]

[0039] wherein N represents the number of cluster centers.

[0040] 3) Create a mask to remove the mirroring: A mask is created to mark the reflection region. The mirroring region after division is set to have a pixel label of 0; when the mask is created, the pixel points in the mask are set to white in the mask which are determined to be the mirroring region. Conversely, the pixel points in the non-mirroring region are set to have a label of 1, and are represented by black in the mask.

[0041] 4) Image restoration: after a series of technical means are used to remove the mirroring, a large number of blank areas will be generated in the image after the mirroring is removed, so as to make the image return to the natural state and keep the image complete. The Telea algorithm is used for image restoration. The algorithm diffuses the color, texture and other characteristics contained in the surrounding pixels to the blank area in a smooth transition manner through careful calculation of the information diffusion of the boundary pixels.

[0042] The Telea algorithm uses the fast marching method to solve the partial differential equation. In the fast marching method, the points in the to-be-repaired region are divided into three categories: known points (repaired points), pending points (points being processed) and unknown points (to-be-repaired points). The algorithm starts from the known points, updates the value of the pending points according to the speed function, and adds them to the known point set until all unknown points are repaired. For each pending point , its value is updated by the following formula:

[0043] wherein is the neighborhood of point , is the value of the speed function at the neighboring point ; this formula states that the value of the point is a weighted average of the values of its neighboring points according to the speed function. By continuously iterating this process, the area to be repaired is gradually filled until the entire area is repaired.

[0044] Embodiment 1: The specific implementation steps of the above method are as follows: Step one: Configure the Python related environment, including code editor, etc. Configure the related libraries including opencv, numpy, etc. Install them by executing the pip install numpy command. After installation, the Python library path where they are located will also be automatically integrated into the environment variable, so you can successfully call these libraries in the code to carry out development work.

[0045] Step two: Create a.py file to write and run Python scripts. Use the import keyword to import related library files such as opencv, numpy, etc.

[0046] Step three: Define the K-means segmentation function kmeans_segmentation( ) to perform K-Means clustering segmentation on the input image. The function receives the image and the number of clusters (default value is 2) as parameters.

[0047] ① Use image.reshape( ) to reshape the input image into a two-dimensional array, where each row represents the red, green, and blue channel values of a pixel. Use the np.float32( ) function to convert the data type to float32 to meet the requirements of the cv2.kmeans function in opencv.

[0048] ② Define the termination condition of K-Means clustering, which combines the two conditions of the distance of cluster center movement being less than a threshold and reaching the maximum number of iterations. Call the cv2.kmeans ( ) function to perform clustering, randomly initialize the cluster centers, run 20 times repeatedly and select the optimal result, get the cluster label of each pixel and the center pixel value of each cluster.

[0049] ③ Call the np.uint8 ( ) function to convert the data type of the cluster center to uint8 to conform to the representation of image pixel values. Then select the corresponding pixel value from the cluster center according to the pixel label to generate a one-dimensional array, reshape it to the same shape as the input image to get the segmented image. Finally, the function returns 2 values, which are the segmented image and the cluster label of each pixel respectively.

[0050] According to the label of each pixel, the corresponding RGB value is obtained from the cluster center, and the segmented pixel value is obtained, and the RGB value of the first pixel is The average brightness value is calculated: The one with a smaller is the reflection.

[0051] The effect of K-Means clustering reflection area division is shown in Figure 3 , where Non-Reflection Area is the non-reflection area and Reflection Area is the reflection area. Figure 3

[0052] Step four: define the function remove_reflection() to remove the reflection area, and the function receives the clustered segmented image as an argument.

[0053] ① Create a single-channel mask reflection_mask with the same shape as the labels array and uint8 data type using the np.zeros_like function in numpy, and the initial value is all 0.

[0054] ② Based on the result of K-means segmentation, assume that the area with label 0 represents the reflection area, assign the value of 255 to the position of reflection_mask corresponding to labels 0, that is, mark the reflection area in the mask as white.

[0055] ③ To ensure that the mask size matches the original image size, check whether the shape of reflection_mask is consistent with the first two dimensions of the original image height and width, if not, use the cv2.resize function to adjust the mask to the same width and height as the original image.

[0056] ④ Define a variable radius, which represents the neighborhood radius used in the image inpainting operation (in this embodiment, radius = 5), that is, when repairing a pixel, it will consider the pixel information within a certain range.

[0057] ​​​​⑤ Use cv2.inpaint() of opencv to repair the original image image, the second parameter reflection_mask of the function is the mask created before for marking the reflection area, which indicates which area of the image needs to be repaired; the third parameter is the neighborhood radius radius defined before; the flags parameter is set to cv2.INPAINT_TELEA, which indicates that the image repair method based on the Fast Marching Method is adopted. The repaired image is stored in the final_image variable.

[0058] Step five: read the image using the cv2.imread() function, and call remove_reflection() to get the final image after removing the reflection.

[0059] Figure 4 The whole process of removing the water surface reflection from the original image using the method of the embodiment is shown, Figure 4 In the method, OrignalImage is the original image, Segmented Image is a result image after K-Means aggregation, Reflection Mask is a result image of creating a mask, and Final Image without Reflection is the final image generated after the above operations and image restoration.

[0060] Finally, it should be pointed out that the above embodiments are only used to illustrate the technical solutions of the present application and not to limit it. Although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present application can be modified or replaced equivalently without departing from the purpose and scope of the technical solutions, and they should be covered in the scope of the claims of the present application.

Claims

1. A method for removing water surface reflections based on K-Means clustering, characterized in that, The method specifically comprises the following steps: S1: data preparation: extracting the feature vector of each pixel point from the image to form a data set for clustering analysis; S2: using K-Means clustering algorithm to cluster and distinguish the reflection area and non-reflection area in the image; S3: creating a mask to remove the reflection; S4: image restoration: using the Telea algorithm to restore the image after removing the reflection.

2. The method of claim 1, wherein, Step S1 specifically includes: extracting the RGB value of each pixel of the image to form a data wherein, is the total number of pixels in the image, denotes the RGB value of the i th pixel, .

3. The method of claim 2, wherein, Step S2 specifically includes: randomly selecting 2 data points from the data set X as initial cluster centers, denoted as and ; calculating the distance between the feature vector of each pixel point in X and the two cluster centers and , using Euclidean distance as the measurement standard; in the image pixels, representing the three channels of RGB, for the pixel and the cluster center , , the Euclidean distance between them is: Set , if , then pixel is assigned to cluster , otherwise to cluster ; Each pixel is assigned to the category represented by the cluster center closest to its feature vector, completing one clustering iteration; after all pixels have been assigned, the two clusters are calculated separately. and The mean of all pixels within the cluster is used as the new cluster center; for each cluster... Let its set of pixels be... ,in It is a cluster The number of pixels in the middle determines the new cluster center. for: The above steps are repeatedly performed until the cluster center no longer changes significantly, that is, the convergence condition of the algorithm is reached. After clustering, the two categories divided are discriminated to discriminate the inverted region and the non-inverted region, specifically: according to the label of each pixel, the corresponding RGB value is obtained from the cluster center to obtain the segmented pixel value, and the RGB value of the first pixel is , , , ; in the RGB color space, the sensitivity of the human visual system to different colors is different, the weight of green is the highest, the weight of red is the second, and the weight of blue is the lowest; the inverted image is darker than the solid color, and there is loss in the light reflection process, so the category with a smaller average brightness value is the inverted image; Wherein, N represents the number of cluster centers.

4. The method of claim 3, wherein, In step S3, the mask is created to remove the reflection, specifically including: creating a mask to mark the reflection area; setting the pixel label of the distinguished reflection area to 0; when creating the mask, the pixel points determined as the reflection area are set to white in the mask; otherwise, the pixel label of the non-reflection area is set to 1, and the mask is represented by black.

5. The method of claim 1, wherein, In step S4, the Telea algorithm is used to restore the image after removing the reflection, specifically by carefully calculating the information diffusion of the boundary pixels, and the features contained in the surrounding pixels are merged into the blank area in a smooth transition manner.

6. The method of claim 1 or 5, wherein, In step S4, the Telea algorithm uses the fast marching method to solve the partial differential equation; in the fast marching method, the points in the to-be-repaired area are divided into three categories: known points (i.e. repaired points), pending points (i.e. points being processed) and unknown points (i.e. to-be-repaired points); the algorithm starts from the known points, updates the value of the pending points according to the speed function, and adds them to the known point set until all unknown points are repaired.

7. The method of claim 6, wherein the mirror image is removed from the water surface. In step S4, for each pending point whose value is updated by the following formula: wherein is a point of the neighborhood, is a velocity function value of the neighborhood point ; this formula indicates that the value of the point is a weighted average of the values of its neighborhood points according to the velocity function; by continuously iterating this process, the to-be-repaired region is gradually filled until the entire region is repaired.

Images