A method for image region segmentation suitable for dynamic scenes

Abstract

The present invention provides an image region segmentation method suitable for dynamic scenes, which belongs to the field of image processing. The dynamic scene refers to a scene in which a moving target exists in the field of view. The method of the present invention comprises the following steps: using a camera to capture a visual image in the scene; using an optical flow estimation network PCW‑Net to calculate the dense optical flow between adjacent camera frames; using color coding to visualize the dense optical flow; using a k‑means algorithm to classify the pixels in the optical flow visualization image into K categories, and the k‑means algorithm is an unsupervised clustering algorithm that divides groups according to distance; extracting a mask for each type of pixel, performing morphological processing on the mask and judging the connectivity between different regions; dividing the unconnected pixel regions in the same pixel type into new classes, thereby obtaining a final region segmentation image. Compared with traditional methods, the image region segmentation results of the method of the present invention for dynamic scenes are more in line with the research needs of dynamic scenes, and can accurately divide the regions in different motion states in the scene, thereby providing strong support for subsequent image processing.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an image region segmentation method suitable for dynamic scenes. Background Art

[0002] Image region segmentation is an important research topic in computer vision. Its goal is to divide pixels in an image into regions with similar characteristics to achieve semantic understanding and analysis of the image. However, in dynamic scenes, accurately segmenting regions with different motion states is crucial for understanding and processing motion information.

[0003] Traditional image region segmentation methods, typically based on features such as color, texture, and edges, can effectively segment image regions based on the characteristics of objects in a scene. However, in motion analysis of dynamic scenes, these methods often fail to accurately segment regions in different motion states. Therefore, a method for image region segmentation suitable for dynamic scenes is needed that can accurately delineate regions in different motion states within a moving scene, thereby providing strong support for subsequent image processing and analysis tasks.

[0004] This invention provides an image region segmentation method suitable for dynamic scenes. An optical flow estimation network calculates dense optical flow between camera frames to measure the motion state of objects. A k-means + connectivity judgment method is used to segment regions in the optical flow visualization image, achieving precise segmentation of regions in different motion states in dynamic scenes. This method can be widely applied in fields such as dynamic SLAM (Simultaneous Localization and Mapping). Summary of the Invention

[0005] In view of the shortcomings of existing traditional image segmentation methods in dynamic scenes, the present invention proposes an image region segmentation method suitable for dynamic scenes, which realizes the effective segmentation of regions in different motion states in dynamic scenes.

[0006] In order to achieve the above purpose, the technical solution adopted by the present invention is:

[0007] This solution provides an image region segmentation method suitable for dynamic scenes, characterized by comprising the following steps:

[0008] S1, using a camera to capture a visual image F of a dynamic scene;

[0009] S2, using the optical flow estimation network PCW-Net to calculate the adjacent camera frames F t and F t+1 The dense optical flow matrix M between them;

[0010] S3, performing color-coded visualization processing on the dense optical flow matrix M to generate an optical flow visualization image I;

[0011] S4. Use the k-means method to classify the pixels in the optical flow visualization image into K categories according to color;

[0012] S5. Extract the mask for each type of pixel, perform morphological processing on the mask, and classify the pixels according to the connectivity relationship of each pixel in the mask:

[0013] S6. Combine the pixel mask classification results of K classes to generate the final region segmentation image.

[0014] Furthermore, the data of the dense optical flow matrix M obtained in step S2 is as follows:

[0015] The size of M is consistent with F, and the optical flow vector (u, v) of each pixel is represented in F t The pixel point at this position in F t+1 The offset of the corresponding pixel point in the x and y coordinates is:

[0016] (u, v) = (x t+1 -x t ,y t+1 -y t ).

[0017] Furthermore, step S3 includes the following steps:

[0018] S301, according to the predefined color coding rules, the colors are stored in the colorwheel array, and different colors correspond to different optical flow directions and sizes;

[0019] S302, traversing the dense optical flow matrix M, calculating the modulus of the optical flow vector, and determining the maximum modulus value;

[0020] S303, mapping the direction of each optical flow vector to a color coding range to obtain a corresponding color;

[0021] S304, performing color normalization according to the modulus of the optical flow vector, so that the color corresponding to the larger modulus is more vivid;

[0022] S305 : Assign the calculated color values ​​to the pixels of the output image to obtain the final color optical flow visualization image.

[0023] Furthermore, step S4 includes the following steps:

[0024] S401, converting the optical flow visualization image into a data set of color feature vectors, where the color feature vector of each pixel includes the value of the RGB color channel;

[0025] S402, determining the number of classification categories K, and randomly selecting K pixels as initial cluster centers;

[0026] S403, repeatedly calculating the distance between each pixel and each cluster center, selecting the cluster center with the closest distance as the category of the pixel, and updating each cluster center to the average value of all pixels belonging to the category, until the algorithm converges or reaches the maximum number of iterations;

[0027] S404: Divide each pixel into a corresponding category to generate a classified image.

[0028] Furthermore, step S5 includes the following steps:

[0029] S501: For each type of pixel, assign a value of 1 to the pixel points belonging to the type and 0 to the other pixel points, thereby obtaining a binary mask corresponding to the pixel points of the type;

[0030] S502, performing morphological processing on the mask, including filling mask holes, removing noise and isolated points, etc.;

[0031] S503 : According to the pixel connectivity of each mask, the connected pixels are classified as one category, and the disconnected pixels are classified into different categories. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 Flow chart of the method of the present invention. DETAILED DESCRIPTION

[0033] The specific embodiments of the present invention are described below to facilitate understanding of the present invention by those skilled in the art. However, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the appended claims, these changes are obvious, and all inventions and creations utilizing the concepts of the present invention are protected.

[0034] Example

[0035] The present invention proposes a method for image region segmentation suitable for dynamic scenes. The method uses an optical flow estimation network to estimate the dense optical flow in dynamic scenes to accurately measure the motion state of different regions. Subsequently, the k-means clustering method is used to classify the pixels in the optical flow visualization image into K categories based on their color features. By extracting a mask for each category of pixels and using a connectivity judgment method, the region is re-segmented. Finally, the region segmentation results of the K categories of pixels are integrated to obtain an image region segmentation image suitable for dynamic scenes. The present invention proposes a method for image region segmentation suitable for dynamic scenes, and its implementation method is as follows:

[0036] S1. Use a camera to capture a visual image F of a dynamic scene. Because it is a dynamic scene, continuous camera frames need to be captured.

[0037] S2, using the optical flow estimation network PCW-Net to calculate the adjacent camera frames F t and F t+1 The dense optical flow matrix M between them has the following data form:

[0038] The size of M is consistent with F, and the optical flow vector (u, v) of each pixel is represented in F t The pixel point at this position in F t+1 The offset of the corresponding pixel point in the x and y coordinates is:

[0039] (u, v) = (x t+1 -x t ,y t+1 -y t ).

[0040] S3. Perform color-coded visualization processing on the dense optical flow matrix M to generate an optical flow visualization image I. The implementation method is as follows:

[0041] S301, according to the predefined color coding rules, the colors are stored in the colorwheel array, and different colors correspond to different optical flow directions and sizes;

[0042] S302, traversing the dense optical flow matrix M, calculating the modulus of the optical flow vector, and determining the maximum modulus value;

[0043] S303, mapping the direction of each optical flow vector to a color coding range to obtain a corresponding color;

[0044] S304, performing color normalization according to the modulus of the optical flow vector, so that the color corresponding to the larger modulus is more vivid;

[0045] S305 : Assign the calculated color values ​​to the pixels of the output image to obtain the final color optical flow visualization image.

[0046] S4. Use the k-means method to classify the pixels in the optical flow visualization image into K categories according to color. The implementation method is as follows:

[0047] S401, converting the optical flow visualization image into a data set of color feature vectors, where the color feature vector of each pixel includes the value of the RGB color channel;

[0048] S402, determining the number of classification categories K, and randomly selecting K pixels as initial cluster centers;

[0049] S403, repeatedly calculating the distance between each pixel and each cluster center, selecting the cluster center with the closest distance as the category of the pixel, and updating each cluster center to the average value of all pixels belonging to the category, until the algorithm converges or reaches the maximum number of iterations;

[0050] S404: Divide each pixel into a corresponding category to generate a classified image.

[0051] S5. Extract a mask for each type of pixel, perform morphological processing on the mask, and classify the pixels according to the connectivity relationship of each pixel in the mask. The implementation method is as follows:

[0052] S501: For each type of pixel, assign a value of 1 to the pixel points belonging to the type and 0 to the other pixel points, thereby obtaining a binary mask corresponding to the pixel points of the type;

[0053] S502, performing morphological processing on the mask, including filling mask holes, removing noise and isolated points, etc.;

[0054] S503 : According to the pixel connectivity of each mask, the connected pixels are classified as one category, and the disconnected pixels are classified into different categories.

[0055] S6. Combine the pixel mask classification results of K classes to generate the final region segmentation image.

[0056] In this example, we developed an image region segmentation method for dynamic scenes. This method uses continuously acquired camera frames as input and performs dense optical flow estimation with the assistance of an optical flow estimation network (PCW-Net). We then employ a k-means + connectivity detection method to segment the optical flow visualization image, classifying pixels in regions with different motion states in dynamic scenes to support subsequent image processing and analysis tasks.

[0057] To validate this image region segmentation method for dynamic scenes, we integrated it into existing dynamic SLAM methods. Dynamic SLAM methods address the shortcomings of SLAM methods in dynamic scenes. Accurately detecting areas in motion is crucial for improving their positioning accuracy. Existing dynamic SLAM methods, such as Dyna-SLAM, typically use instance segmentation methods like MASK-RCNN to directly segment objects in dynamic scenes that may be in motion, such as people and cars. The segmented images are then used as the basis for the SLAM process to reduce the system's impact on dynamic objects. However, this approach is significantly affected by the presence of non-rigid objects and undefined moving targets in the scene. Instance segmentation methods can only remove pre-defined moving targets and are unable to segment undefined moving targets. Furthermore, when dealing with non-rigid targets such as people that only have localized motion, instance segmentation methods will directly remove the entire human body, resulting in the inadvertent removal of a large number of non-moving regions. Our method replaces the instance segmentation method in existing dynamic SLAM methods, can accurately segment the moving areas in the image, and effectively deal with non-rigid objects and undefined moving targets in various dynamic scenes.

[0058] To evaluate our method, we compared it with Dyna-SLAM, DS-SLAM, DGS-SLAM, and the general-purpose SLAM method ORB-SLAM2 on the commonly used dynamic SLAM test dataset TUM. Table 1 shows the localization accuracy of each method in terms of absolute trajectory error, measured using root mean square error. The best results are marked in bold black.

[0059] Table 1 Experimental results

[0060]

[0061]

[0062] Experimental results show that the method of the present invention achieves the highest positioning accuracy in most sequences of the TUM dataset. Compared with other dynamic SLAM methods and general SLAM methods, our method can effectively segment the areas in dynamic scenes that are truly in motion, achieve accurate removal, and achieve excellent results in dynamic scenes of the TUM dataset. Therefore, the method proposed by the present invention has broad application prospects in the field of image processing, especially in fields such as dynamic SLAM.

Claims

1. A method for image region segmentation suitable for dynamic scenes, characterized in that The following steps are involved: S1, using a camera to capture a visual image F of a dynamic scene; S2, using the optical flow estimation network PCW-Net to calculate the adjacent camera frames F t and F t+1 The dense optical flow matrix M between them; S3, performing color-coded visualization processing on the dense optical flow matrix M to generate an optical flow visualization image I; S4. Use the k-means method to classify the pixels in the optical flow visualization image into K categories according to color; S5. Extracting a mask for each type of pixel, performing morphological processing on the mask, and classifying the pixels according to the connectivity relationship of each pixel in the mask; S6. Combine the pixel mask classification results of K classes to generate the final region segmentation image; Furthermore, the data of the dense optical flow matrix M obtained in step S2 is as follows: The size of M is consistent with F, and the optical flow vector (u, v) of each pixel is represented in F t The pixel point at this position in F t+1 The offset of the corresponding pixel point in the x and y coordinates is: (u,v)=(x t+1 -x t ,and t+1 -and t ) The step S3 comprises the following steps: S301, according to the predefined color coding rules, the colors are stored in the colorwheel array, and different colors correspond to different optical flow directions and sizes; S302, traversing the dense optical flow matrix M, calculating the modulus of the optical flow vector, and determining the maximum modulus value; S303, mapping the direction of each optical flow vector to a color coding range to obtain a corresponding color; S304, performing color normalization according to the modulus of the optical flow vector, so that the color corresponding to the larger modulus is more vivid; S305, assigning the calculated color value to the pixel of the output image to obtain the final color optical flow visualization image; The step S4 comprises the following steps: S401, converting the optical flow visualization image into a data set of color feature vectors, where the color feature vector of each pixel includes the value of the RGB color channel; S402, determining the number of classification categories K, and randomly selecting K pixels as initial cluster centers; S403, repeatedly calculating the distance between each pixel and each cluster center, selecting the cluster center with the closest distance as the category of the pixel, and updating each cluster center to the average value of all pixels belonging to the category, until the algorithm converges or reaches the maximum number of iterations; S404, dividing each pixel into a corresponding category to generate a classified image; The step S5 includes the following steps: S501: For each type of pixel, assign a value of 1 to the pixel points belonging to the type and 0 to the other pixel points, thereby obtaining a binary mask corresponding to the pixel points of the type; S502, performing morphological processing on the mask, including filling mask holes, removing noise and isolated points, etc.; S503 : According to the pixel connectivity of each mask, the connected pixels are classified as one category, and the disconnected pixels are classified into different categories.

Images