Image difference region detection method based on superpixel segmentation

Abstract

The application discloses an image difference area detection method based on superpixel segmentation, comprising the following steps: S1, using a sift algorithm to register images to be compared, so that the images are changed to the same plane, and coordinates of corresponding points are one-to-one corresponding; S2, constructing image pyramids of a source image and a target image through a Gaussian function; S3, extracting the first image of the top layer of the target image pyramid, performing SLIC processing on the first image, and dividing the image into a plurality of sub-regions with relatively consistent information; and S4, performing difference discrimination on each sub-region, calculating the size, direction and color information of the gradient of each pixel of each sub-region, and comparing the three-dimensional information with the corresponding pixel. The application uses the SLIC algorithm to perform regional clustering on the image, uses the clustered regions as image blocks to calculate the difference, and makes the information in the blocks more consistent.

Description

Technical Field

[0001] This invention belongs to the field of image discrimination and relates to a method for detecting image difference regions based on superpixel segmentation. Background Technology

[0002] Image discrimination is used to calculate the regions of difference between images, providing more useful information in the temporal and spatial domains, and has an important impact on fields such as detection and tracking.

[0003] Image discrimination algorithms vary. Some algorithms calculate differences between pixels, while others calculate differences between image blocks. Each approach has its advantages and disadvantages. Pixel-based algorithms offer more precise calculations and better reflect subtle changes, while block-based algorithms are more robust and resistant to noise and other interference. However, pixel-based algorithms produce noisier information and are more susceptible to noise and image offsets. Block-based algorithms, on the other hand, suffer from greater information variability due to inconsistent composition within regions, and the filtering process can cause some useful information to be missed, resulting in less precise detection of discrepancies. Summary of the Invention

[0004] To address the aforementioned problems, this invention proposes an image difference region detection method based on superpixel segmentation, comprising the following steps:

[0005] S1. Use the SIFT algorithm to register the images to be compared, so that they are transformed to the same plane, and match the coordinates of the corresponding points one by one;

[0006] S2, constructs image pyramids for the source and target images using a Gaussian function;

[0007] S3: Extract the first image at the top of the target image pyramid, perform SLIC processing on it, and divide the image into multiple sub-regions with relatively consistent information.

[0008] S4. Differentiate each sub-region: For each pixel in the sub-region, calculate the magnitude, direction, and color information of its gradient, and compare the difference between these three dimensions and the corresponding pixel. When the difference value of a certain dimension is greater than the preset threshold, the difference degree of the point is increased by 1; when the difference values ​​of all three dimensions are greater than the threshold, the difference degree of the point will be increased by an additional 1. That is, the difference value of each pixel is between 0 and 4.

[0009] Preferably, the image pyramid in S2 includes two parameters: layer height and number of layers. The number of layers determines how many image resolutions are in the pyramid, and the layer height determines the number of images after filtering at a single resolution. By adjusting the shape of the pyramid using these two parameters, the ratio of the detection area located at the difference to the total detection area is maximized in subsequent processing.

[0010] Preferably, step S3 further includes using an interpolation algorithm to extend the segmentation information of the first image at the top of the pyramid to each layer of the pyramid, so that all images of the pyramid obtain a corresponding and consistent image region distribution.

[0011] Preferably, step S4 includes the following steps:

[0012] S41, use the Sobel operator to calculate the gradient magnitude and direction of the pixel:

[0013] I x =G x I (1)

[0014] I y =G y I (2)

[0015]

[0016]

[0017] Where I is the input image, G x and G y Sobel operators in the x and y directions, respectively, I x and I y For the gradient map in the corresponding direction, I I Let I be the magnitude of the gradient. θ The direction of the gradient;

[0018] S42 converts the image to the HSV color space and obtains the value of its H dimension, which is the color information I of the pixel. H Then, the I values ​​of the source image pixels and the target image pixels are obtained respectively. G I θ and I H After obtaining the 3-dimensional information, calculate the difference values ​​corresponding to the 3 dimensions:

[0019]

[0020]

[0021]

[0022] Among them, I′ G 、I′ θ 、I′ H The three-dimensional information values ​​corresponding to the source image. For the 3-dimensional information value corresponding to the target image, when S G S θ S HEach greater than the gradient threshold τ G τ θ τ H At that time, the total difference D of the pixel p Add 1 to each, otherwise do not add, when S G S θ S H When all three are greater than their corresponding thresholds, D p Then add 1 extra;

[0023] S43, after summing and normalizing the differences of all pixels within the region, we obtain the overall regional difference D. t :

[0024]

[0025] Where n is the number of pixels in the region, D pi D is the i-th pixel p Value, when the overall variability D of the region t Greater than the threshold τ t When the number of images in a region marked as 1 is greater than the total number of images τ, the region is marked as 1, indicating a possible difference; otherwise, it is marked as 0. After marking all image regions in the pyramid, if the number of images marked as 1 in a certain region is greater than the total number of images τ, the region is considered a potential difference. r If so, then it is determined that there are differences in the region;

[0026]

[0027] In equation (9), M k Here, k represents the label value for the region with index k in the final difference map, and m is the total number of pyramid images. Let be the label value of the k-th region in the i-th image.

[0028] The beneficial effects of this invention include at least the following:

[0029] This application employs a patch-based detection algorithm, but unlike simple image segmentation, it utilizes the SLIC algorithm to cluster regions within the image. These clustered regions are then used as image patches for difference calculation, resulting in more consistent information within each patch. Furthermore, pixel-by-pixel calculations are performed within each patch, incorporating gradient magnitude and angle information, and supplemented with hue information. Compared to histogram quantization, this pixel-by-pixel calculation method does not discard spatial information of individual pixels, leading to more accurate and reliable results. Attached Figure Description

[0030] Figure 1 This is a flowchart illustrating the steps of the image difference region detection method based on superpixel segmentation according to an embodiment of the present invention.

[0031] Figure 2This is an image pyramid diagram in the image difference region detection method based on superpixel segmentation according to an embodiment of the present invention;

[0032] Figure 3 This is the clustering input image of the image difference region detection method based on superpixel segmentation in this embodiment of the invention;

[0033] Figure 4 This is a clustering segmentation effect diagram of the image difference region detection method based on superpixel segmentation according to an embodiment of the present invention;

[0034] Figure 5 This is a diagram showing the difference detection effect of the image difference region detection method based on superpixel segmentation according to an embodiment of the present invention. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0036] Conversely, this invention encompasses any substitutions, modifications, equivalent methods, and solutions made within the spirit and scope of the invention as defined in the claims. Furthermore, to provide a better understanding of the invention, certain specific details are described in detail below. However, those skilled in the art will fully understand the invention even without these detailed descriptions.

[0037] See Figure 1 This includes the following steps:

[0038] S1. The SIFT algorithm is used to register the images to be compared, so that they are transformed to the same plane, and the coordinates of the corresponding points are matched one by one to eliminate the influence of different shooting poses.

[0039] S2 constructs the image pyramids for the source and target images using a Gaussian function. (See [link]) Figure 2 ;

[0040] S3 extracts the first image at the top of the target image pyramid and performs SLIC processing on it to segment the image into multiple sub-regions with relatively consistent information. SLIC (simple linear iterative cluster) processing improves the running speed and clustering effect of the k-means algorithm.

[0041] See Figure 3 , 4As can be seen, after SLIC processing, the input image is segmented into image blocks with similar image information. Then, the segmentation information of the first image at the top of the pyramid is expanded to each layer of the pyramid using an interpolation algorithm, so that all images in the pyramid can obtain the corresponding image region distribution.

[0042] S4. Differentiate each sub-region: For each pixel in the sub-region, calculate the magnitude, direction, and color information of its gradient, and compare the difference between these three dimensions and the corresponding pixel. When the difference value of a certain dimension is greater than the preset threshold, the difference degree of the point is increased by 1; when the difference values ​​of all three dimensions are greater than the threshold, the difference degree of the point will be increased by an additional 1. That is, the difference value of each pixel is between 0 and 4.

[0043] The S2 image pyramid includes two parameters: layer height and number of layers. The number of layers determines how many image resolutions are in the pyramid, and the layer height determines the number of filtered images at a single resolution. By adjusting the shape of the pyramid using these two parameters, the ratio of the detection area at the difference point to the total detection area is maximized in subsequent processing, and is as close to 100% as possible.

[0044] S4 includes the following steps:

[0045] S41, use the Sobel operator to calculate the gradient magnitude and direction of the pixel:

[0046] I x =G x I (1)

[0047] I y =G y I (2)

[0048]

[0049]

[0050] Where I is the input image, G x and G y Sobel operators in the x and y directions, respectively, I x and I y For the gradient map in the corresponding direction, I G Let I be the magnitude of the gradient. θ The direction of the gradient;

[0051] S42 converts the image to the HSV color space and obtains the value of its H dimension, which is the color information I of the pixel. H Then, the I values ​​of the source image pixels and the target image pixels are obtained respectively. G I θ and IH After obtaining the 3-dimensional information, calculate the difference values ​​corresponding to the 3 dimensions:

[0052]

[0053]

[0054]

[0055] Among them, I′ G 、I′ θ 、I′ H The three-dimensional information values ​​corresponding to the source image. For the 3-dimensional information value corresponding to the target image, when S G S θ S H Each greater than the gradient threshold τ G τ θ τ H At that time, the total difference D of the pixel p Add 1 to each, otherwise do not add, when S G S θ S H When all three are greater than their corresponding thresholds, D p Then add 1 extra;

[0056] S43, after summing and normalizing the differences of all pixels within the region, we obtain the overall regional difference D. t :

[0057]

[0058] Where n is the number of pixels in the region, D pi D is the i-th pixel p Value, when the overall variability D of the region t Greater than the threshold τ t When the number of images in a region marked as 1 is greater than the total number of images τ, the region is marked as 1, indicating a possible difference; otherwise, it is marked as 0. After marking all image regions in the pyramid, if the number of images marked as 1 in a certain region is greater than the total number of images τ, the region is considered a potential difference. r If so, then it is determined that there are differences in the region;

[0059]

[0060] In equation (9), M k Here, k represents the label value for the region with index k in the final difference map, and m is the total number of pyramid images. Let be the label value of the k-th region in the i-th image.

[0061] See Figure 5As can be seen, the effect of merging the difference regions in the pyramid image is that, since the image blocks are obtained through clustering rather than simple division, the boundaries of the difference regions are more closely aligned and accurate, making the difference mask more precise.

[0062] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An image difference region detection method based on superpixel segmentation, characterized in that, Includes the following steps: S1. Use the SIFT algorithm to register the images to be compared, so that they are transformed to the same plane, and match the coordinates of the corresponding points one by one; S2, constructs image pyramids for the source and target images using a Gaussian function; S3: Extract the first image at the top of the target image pyramid, perform SLIC processing on it, and divide the image into multiple sub-regions with relatively consistent information. S4. Differentiate each sub-region: For each pixel in the sub-region, calculate the magnitude, direction, and color information of its gradient, and compare the difference of these three dimensions of information of corresponding pixels in the source image and the target image. When the difference value of a certain dimension is greater than the preset threshold, the difference degree of the point is increased by 1; when the difference values ​​of all three dimensions are greater than the threshold, the difference degree of the point will be increased by an additional 1, that is, the difference value of each pixel is between 0 and 4. The image pyramid of S2 includes two parameters: layer height and number of layers. The number of layers determines how many image resolutions are in the pyramid, and the layer height determines the number of images after filtering at a single resolution. By adjusting the shape of the pyramid through these two parameters, the ratio of the detection area at the difference point to the total detection area is maximized in subsequent processing. The S3 section further includes using an interpolation algorithm to diffuse the segmentation information of the first image at the top of the pyramid to every layer of the pyramid, so that all images of the pyramid obtain a corresponding and consistent image region distribution. S4 includes the following steps: S41, use the Sobel operator to calculate the gradient magnitude and direction of the pixel: (1)； (2)； (3)； (4)； in, For the input image, and These are the Sobel operators in the x and y directions, respectively. and This is the gradient map in the corresponding direction. The magnitude of the gradient. The direction of the gradient; S42 converts the image to the HSV color space and obtains the value of its H dimension, which is the color information of the pixel. Then, the source image pixels and target image pixels are obtained separately. , and After obtaining the 3-dimensional information, calculate the difference values ​​corresponding to the 3 dimensions: (5)； (6)； (7)； in, , , The three-dimensional information values ​​corresponding to the source image. , , For the 3-dimensional information value corresponding to the target image, when , , Each greater than a preset threshold , , At that time, the total difference of the pixel Add 1 to each case, otherwise leave it blank. , , When all three are greater than the corresponding thresholds Then add 1 extra; S43, after summing and normalizing the differences of all pixels within the region, we obtain the overall regional difference. : (8)； Where n is the number of pixels in the region. For the i-th pixel Value, when the overall variability of the region Greater than the threshold When the number of images in a region marked as 1 is greater than the total number of images, the region is marked as 1, indicating a potential difference; otherwise, it is marked as 0. After marking all image regions in the pyramid, if the number of images marked as 1 in a certain region exceeds the total number of images... If so, then it is determined that there are differences in the region; (9)； In equation (9), Here, k represents the label value for the region with index k in the final difference image, and m is the total number of pyramid images. Let be the label value of the k-th region in the i-th image.

Images