A fast method for locating abnormal visual information in mosaic images

Abstract

The present invention relates to the field of image processing, and provides a method for quickly locating abnormal visual information in a stitched image, by marking all stitched areas in the stitched image, dividing each stitched area into a plurality of sub-areas in turn, locating abnormal image blocks according to the pixel distribution in the sub-areas, and highlighting all abnormal image blocks in the stitched image. The method can effectively identify and locate abnormal visual information in the stitched image, reduce the need for manual participation in the identification and screening process, and can process a larger amount of image data in a shorter time by analyzing the pixel distribution in the sub-areas in parallel, significantly improving the overall visual quality of the stitched image, and can also be widely used in various images that need to be stitched, and has strong versatility.

Description

Technical Field

[0001] The invention relates to the field of image processing, and in particular to a method for quickly locating abnormal visual information in a spliced ​​image. Background Art

[0002] Image stitching and fusion, as an indispensable step in modern visual information processing technology, is widely used in many image processing-related fields such as panoramic stitching and satellite image synthesis.

[0003] During the acquisition process of stitched images, due to multi-angle framing, uneven exposure caused by different lighting conditions or differences in shooting equipment parameters, abnormal visual information often appears in the stitched images, including brightness or geometric distortion, discontinuity in stitching seams or blurred objects, which has a great impact on the stitching quality.

[0004] Geometric distortion usually comes from the wide-angle lens or camera equipment with different viewing angles used in the image capture process. During the stitching process, the edges of different images cannot be accurately aligned due to distortion, resulting in stretching, compression or irregular deformation of the stitched image, making it difficult to accurately match the shape and position of the target object, which in turn affects the visual coherence of the image and causes deviations in subsequent detection algorithms based on related image features. At the same time, the stitching seam problem is another common problem in image stitching. Although the current mainstream stitching algorithms can better align the geometric features of the image, due to the differences in color, brightness or texture between image blocks, there are still a small number of relatively obvious seams in the stitching results. These seams are visually manifested as discontinuous areas in the image, especially in locations with large differences in color or brightness.

[0005] Therefore, for complex and large stitched images, a method that can quickly and accurately detect and locate abnormal visual information in the image is the key to improving the overall image stitching effect in scenarios such as panoramic image synthesis and high-resolution remote sensing image applications. Summary of the invention

[0006] The purpose of the present invention is to provide a method for quickly locating abnormal visual information in a spliced ​​image to solve one or more technical problems existing in the prior art and at least provide a beneficial choice or create conditions.

[0007] The present invention provides a method for quickly locating abnormal visual information in a stitched image, by marking all stitched areas in the stitched image, dividing each stitched area into a plurality of sub-areas in turn, locating abnormal image blocks according to the pixel distribution in the sub-areas, and highlighting all abnormal image blocks in the stitched image. The method can effectively identify and locate abnormal visual information in the stitched image, reduce the need for manual participation in the identification and screening process, and can process a larger amount of image data in a shorter time by analyzing the pixel distribution in the sub-areas in parallel, significantly improving the overall visual quality of the stitched image, and can also be widely used in various images that need to be stitched, with strong versatility.

[0008] In order to achieve the above object, according to one aspect of the present invention, a method for quickly locating abnormal visual information in a spliced ​​image is provided, the method comprising the following steps:

[0009] Mark all stitching regions in the stitching image, and divide each stitching region into multiple sub-regions in turn;

[0010] According to the pixel distribution in the sub-region, the abnormal image block is located;

[0011] Highlight all abnormal image blocks in the stitched image;

[0012] The stitched image is stitched together from multiple sub-images.

[0013] Furthermore, the stitched area in the stitched image refers to partial image areas in a plurality of sub-images that are aligned in the same coordinate system after geometric transformation, and these partial image areas are shared by different sub-images and contain the same or similar pixels.

[0014] Furthermore, the method of dividing the stitched area into multiple sub-areas is specifically: dividing the stitched area into multiple sub-areas by an image segmentation algorithm based on region growing, or by an image segmentation algorithm based on edge detection, where N represents the number of the multiple sub-areas (different stitched areas correspond to different numbers of sub-areas, that is, different N values).

[0015] Furthermore, the pixel distribution in the sub-region is defined as follows: the k-th sub-region among the N sub-regions is sbr(k), sbr(k) contains M(k) pixels, and each pixel corresponds to a unique pixel value; pb(i,k) represents the i-th pixel among the M(k) pixels in sbr(k), and Vpb(i,k) represents the pixel value corresponding to pb(i,k);

[0016] Taking the M(k) pixels contained in the sub-region sbr(k) and the M(k) pixel values ​​corresponding to these M(k) pixels as the pixel distribution of sbr(k), there are N kinds of pixel distributions corresponding to N sub-regions.

[0017] Furthermore, according to the pixel distribution in the sub-region, the method for locating the abnormal image block is specifically as follows:

[0018] S1, among the M(k) pixels in the sub-region sbr(k), mark the pixel with the largest pixel value pb(c1,k) and the pixel with the smallest pixel value pb(c2,k);

[0019] Initialize variable h0, the initial value of h0 is set to h0=sum(sbr(k))÷(Vpb(c1,k)×Vpb(c2,k)), sum(sbr(k)) represents the sum of the pixel values ​​of M(k) pixels in the sub-region sbr(k), and go to S2;

[0020] S2, record A0=Vpb(c1,k)-h0, record B0=Vpb(c2,k)+h0; among the M(k) pixels in the sub-region sbr(k), record the pixel with a pixel value less than A0 and closest to A0 as pb(x,k), and the pixel with a pixel value greater than B0 and closest to B0 as pb(y,k); mark pb(x,k) as a high visual pixel, mark pb(y,k) as a low visual pixel, and go to S3;

[0021] S3, increase the value of h0 by Vpb(x,k)÷Vpb(y,k), (that is, update the value of h0 to h0+Vpb(x,k)÷Vpb(y,k), expressed in code language as h0=h0+Vpb(x,k)÷Vpb(y,k)); if the first visual condition is met, go to S4; if the first visual condition is not met, go to S2;

[0022] S4, locates the abnormal image blocks according to all high visual pixels and all low visual pixels.

[0023] The beneficial effect of this step is: since the stitching areas in the stitching image often have alignment errors or visual inconsistencies caused by image edge effects, targeted local repair is required, and large-scale stitching images have more stitching areas, it is difficult to meet the needs of fast image processing by manual screening, identification and repair. Therefore, the method of this step utilizes the pixel distribution of the sub-area, gradually marks and updates the high and low visual pixels in the sub-area, and realizes the recognition of abnormal image blocks by comparing the degree of fusion of the two types of pixels, thereby reducing the misjudgment of non-abnormal image blocks and improving the accurate recognition performance of abnormal image blocks. When processing large-scale stitching images, the efficiency and quality of the abnormal visual processing process of the image are significantly improved by quickly locating the abnormal area.

[0024] The principle of this step is: it can be seen that in the process of screening high-visual pixels and low-visual pixels, h0 is a constant growth, and the values ​​of A0 and B0 will decrease and increase with the increase of h0, respectively. When A0 becomes smaller, the Vpb(x,k) corresponding to the screened pb(x,k) will be farther away from Vpb(c1,k) than the Vpb(x,k) screened in the previous round. Similarly, when B0 becomes larger, Vpb(y,k) will also be farther away from Vpb(c2,k), so that the two types of visual pixels are moving towards all The position of the median pixel value is close until the first visual condition is met and the traversal is completed. Obviously, since the screening conditions are set for pb(x, k) and pb(y, k), the screening process of high visual pixels and low visual pixels is an interval screening. Compared with the pixel-by-pixel screening method from high to low and finally close to the median, the interval screening method of this step can save more traversal cost overhead. Therefore, when facing the entire stitched image, it can quickly locate abnormal visual information by processing multiple sub-areas at the same time.

[0025] Furthermore, the method for determining whether the first visual condition is satisfied is as follows: if any high visual pixel among all high visual pixels has a pixel value less than h1, or any low visual pixel among all low visual pixels has a pixel value greater than h1, then the first visual condition is satisfied, where h1=(Vpb(c1,k)+Vpb(c2,k))÷2.

[0026] Furthermore, the method for locating abnormal image blocks according to all high-vision pixels and all low-vision pixels is specifically as follows: the minimum convex hull determined by all high-vision pixels is recorded as a high-vision area, and the minimum convex hull determined by all low-vision pixels is recorded as a low-vision area;

[0027] If there are less than M0 low-vision pixels in the high-vision area, or if there are less than M0 high-vision pixels in the low-vision area, the minimum convex hull jointly determined by all high-vision pixels and all low-vision pixels is recorded as the abnormal image block (or image abnormal area), and the value of M0 is equal to the absolute value of the difference between the number of all high-vision pixels and the number of all low-vision pixels.

[0028] The beneficial effect of this step is: by utilizing the two types of visual areas formed by high and low visual pixels, it is possible to avoid judging the abnormal image block as being too wide or too narrow, thereby making the positioning of the abnormal image block more accurate and reducing misjudgment. Using the difference in the number of high and low visual pixels as the judgment value can adapt to the pixel distribution of different sub-areas. The method has strong versatility. Once the abnormal image block is accurately located, the repair or processing of these abnormal visual information can be more direct and effective, thereby fully improving the processing efficiency of the image stitching quality optimization process.

[0029] The principle of this step is: the abnormal visual information in the stitching area is affected by many factors. In addition to the alignment error and image edge effect mentioned above, factors such as exposure difference, shooting angle, color difference, etc. will also lead to poor stitching effect. However, these abnormal visual information are often presented in the form of pixel segmentation. In addition to obvious segmentation (when the high and low visual areas do not contain each other's regional pixels, it is an obvious segmentation), some relatively less obvious visual information also has a certain probability of being abnormal visual information. Therefore, how to quickly and accurately identify this part of potential and less obvious abnormal visual information is the key to improving the stitching quality of the stitched image. This step In the first step, the abnormal image block is accurately located by the degree of intersection of the two types of visual areas determined by all high and low visual pixels, and M0 is used as the standard for judging whether the degree of intersection between the two reaches the standard for considering it as an abnormal image block. If one visual area contains a large number of visual pixels of the other visual area, it indicates that the degree of fusion is good. On the contrary, if one visual area contains a small number of visual pixels of the other type (M0 specifically defines the standard of very few), it indicates that although the two visual areas overlap, the visual information in one of the areas is not significant. Therefore, the area jointly determined by the two types of visual areas, that is, their union, is marked as an abnormal image block.

[0030] Furthermore, the method for highlighting all abnormal image blocks in the stitched image is specifically as follows: for any stitched area, all sub-areas in the stitched area are marked with abnormal image blocks, that is, the marking of all abnormal image blocks in the stitched area is completed; in the stitched image, all abnormal image blocks in all stitched areas are marked in turn, that is, the marking of all abnormal image blocks in the stitched image is completed; and all abnormal image blocks marked in the stitched image are highlighted.

[0031] Preferably, highlighting all abnormal image blocks in the stitched image also includes fusing all abnormal image blocks in the stitched image, specifically: in the stitched image, fusing all abnormal image blocks by a Poisson fusion algorithm, wherein the pixel selection range in the process of executing the Poisson fusion algorithm is all pixels in the entire stitched image.

[0032] The beneficial effects of the present invention are as follows: the method can effectively identify and locate abnormal visual information in a stitched image, reduce the need for manual participation in the identification and screening process, and by analyzing the pixel distribution in the sub-regions in parallel, can process a larger amount of image data in a shorter time, significantly improve the overall visual quality of the stitched image, and can also be widely used in various images that need to be stitched, with strong versatility. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The above and other features of the present invention will become more obvious by describing in detail the embodiments shown in the accompanying drawings. The same reference numerals in the accompanying drawings of the present invention represent the same or similar elements. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other accompanying drawings can be obtained based on these accompanying drawings without creative work. In the accompanying drawings:

[0034] Figure 1 Shown is a flow chart of a method for quickly locating abnormal visual information in a stitched image;

[0035] Figure 2 Shown is a schematic diagram of a stitching area in a stitching image;

[0036] Figure 3 Another schematic diagram of a stitched area within a stitched image is shown. DETAILED DESCRIPTION

[0037] The following will be combined with the embodiments and drawings to clearly and completely describe the concept, specific structure and technical effects of the present invention, so as to fully understand the purpose, scheme and effect of the present invention. It should be noted that the embodiments and features in the embodiments of this application can be combined with each other without conflict.

[0038] In the description of the present invention, "several" means one or more, "more" means more than two, "greater than", "less than", "exceed" etc. are understood as not including the number itself, and "above", "below", "within" etc. are understood as including the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0039] like Figure 1 The flowchart of the method for quickly locating abnormal visual information in a spliced ​​image according to the present invention is shown below. Figure 1 A method for quickly locating abnormal visual information in a stitched image according to an embodiment of the present invention is described below.

[0040] The present invention provides a method for quickly locating abnormal visual information in a spliced ​​image, the method comprising the following steps:

[0041] Mark all stitching regions in the stitching image, and divide each stitching region into multiple sub-regions in turn;

[0042] According to the pixel distribution in the sub-region, the abnormal image block is located;

[0043] Highlight all abnormal image blocks in the stitched image;

[0044] The stitched image is stitched together from multiple sub-images.

[0045] Furthermore, the stitched area in the stitched image refers to partial image areas in a plurality of sub-images that are aligned in the same coordinate system after geometric transformation, and these partial image areas are shared by different sub-images and contain the same or similar pixels.

[0046] Furthermore, the method of dividing the stitched area into multiple sub-areas is specifically: dividing the stitched area into multiple sub-areas by an image segmentation algorithm based on region growing, or by an image segmentation algorithm based on edge detection, where N represents the number of the multiple sub-areas (different stitched areas correspond to different numbers of sub-areas, that is, different N values).

[0047] Furthermore, the pixel distribution in the sub-region is defined as follows: let the k-th sub-region among N sub-regions be sbr(k), k is the serial number, k=1,2,…,N (equivalent to assigning numbers 1,2,…,N to N sub-regions), let sbr(k) contain M(k) pixels, and each pixel corresponds to a unique pixel value; let pb(i,k) represent the i-th pixel among the M(k) pixels in sbr(k), i is the serial number, i=1,2,…,M(k), i in pb(i,k) corresponds to M(k) pixels, k corresponds to sbr(k), Vpb(i,k) represents the pixel value corresponding to pb(i,k), then the sub-region contains M(k) pixels and M(k) pixel values ​​corresponding to M(k) pixels;

[0048] Taking the M(k) pixels contained in the sub-region sbr(k) and the M(k) pixel values ​​corresponding to these M(k) pixels as the pixel distribution of sbr(k), then N sub-regions correspond to N kinds of pixel distributions.

[0049] Furthermore, according to the pixel distribution in the sub-region, the method for locating the abnormal image block is specifically as follows:

[0050] S1, among the M(k) pixels in the sub-region sbr(k), mark the pixel with the largest pixel value pb(c1,k) and the pixel with the smallest pixel value pb(c2,k), c1, c2 are serial numbers, c1, c2∈[1,M(k)];

[0051] Initialize variable h0, the initial value of h0 is set to h0=sum(sbr(k))÷(Vpb(c1,k)×Vpb(c2,k)), where sum(sbr(k)) represents the sum of the pixel values ​​of M(k) pixels in the sub-region sbr(k), Vpb(c1,k) represents the pixel value corresponding to pb(c1,k), and Vpb(c2,k) represents the pixel value corresponding to pb(c2,k), and go to S2;

[0052] S2, record A0=Vpb(c1,k)-h0, record B0=Vpb(c2,k)+h0; among the M(k) pixels in the sub-region sbr(k), the pixel with a pixel value less than A0 and closest to A0 is recorded as pb(x,k), and the pixel with a pixel value greater than B0 and closest to B0 is recorded as pb(y,k), x and y are variables, x, y∈[1,M(k)]; mark pb(x,k) as a high visual pixel, mark pb(y,k) as a low visual pixel, and go to S3;

[0053] pb(x, k) and pb(y, k) are updated as the values ​​of A0 and B0 are continuously updated, thereby continuously marking more high visual pixels and low visual pixels. That is, the role of pb(x, k) and pb(y, k) is to update high and low visual pixels.

[0054] S3, increase the value of h0 by Vpb(x,k)÷Vpb(y,k), (that is, update the value of h0 to h0+Vpb(x,k)÷Vpb(y,k), expressed in code language as h0=h0+Vpb(x,k)÷Vpb(y,k)); if the first visual condition is met, go to S4; if the first visual condition is not met, go to S2; Vpb(x,k) represents the pixel value corresponding to pb(x,k), and Vpb(y,k) represents the pixel value corresponding to pb(y,k);

[0055] S4, locates the abnormal image blocks according to all high visual pixels and all low visual pixels.

[0056] Furthermore, the method for determining whether the first visual condition is satisfied is as follows: if any high visual pixel among all high visual pixels has a pixel value less than h1, or any low visual pixel among all low visual pixels has a pixel value greater than h1, then the first visual condition is satisfied, where h1=(Vpb(c1,k)+Vpb(c2,k))÷2.

[0057] Furthermore, the method for locating abnormal image blocks according to all high-vision pixels and all low-vision pixels is specifically as follows: the minimum convex hull determined by all high-vision pixels is recorded as a high-vision area, and the minimum convex hull determined by all low-vision pixels is recorded as a low-vision area;

[0058] If there are less than M0 low-vision pixels in the high-vision area, or if there are less than M0 high-vision pixels in the low-vision area, the minimum convex hull jointly determined by all high-vision pixels and all low-vision pixels is recorded as the abnormal image block (or image abnormal area), and the value of M0 is equal to the absolute value of the difference between the number of all high-vision pixels and the number of all low-vision pixels.

[0059] Furthermore, the method for highlighting all abnormal image blocks in the stitched image is specifically as follows: for any stitched area, all sub-areas in the stitched area are marked with abnormal image blocks, that is, the marking of all abnormal image blocks in the stitched area is completed; in the stitched image, all abnormal image blocks in all stitched areas are marked in turn, that is, the marking of all abnormal image blocks in the stitched image is completed; and all abnormal image blocks marked in the stitched image are highlighted.

[0060] Specifically, highlighting all abnormal image blocks in the stitched image also includes fusing all abnormal image blocks in the stitched image, specifically: in the stitched image, fusing all abnormal image blocks through a Poisson fusion algorithm, wherein the pixel selection range in the process of executing the Poisson fusion algorithm is all pixels in the entire stitched image.

[0061] The present invention provides a method for quickly locating abnormal visual information in a stitched image, by marking all stitched areas in the stitched image, dividing each stitched area into a plurality of sub-areas in turn, locating abnormal image blocks according to the pixel distribution in the sub-areas, and highlighting all abnormal image blocks in the stitched image. The method can effectively identify and locate abnormal visual information in the stitched image, reduce the need for manual participation in the identification and screening process, and can process a larger amount of image data in a shorter time by analyzing the pixel distribution in the sub-areas in parallel, significantly improve the overall visual quality of the stitched image, and can also be widely used in various images that need to be stitched, with strong versatility. Although the description of the present invention has been quite detailed and several of the embodiments have been described in particular, it is not intended to be limited to any of these details or embodiments or any special embodiments, so as to effectively cover the predetermined scope of the present invention. In addition, the above describes the present invention with the foreseeable embodiments of the inventor, the purpose of which is to provide a useful description, and those non-substantial changes to the present invention that have not yet been foreseen can still represent equivalent changes to the present invention.

Claims

1. A method for quickly locating abnormal visual information in a spliced ​​image, characterized in that: The method comprises the following steps: Mark all stitching regions in the stitching image, and divide each stitching region into multiple sub-regions in turn; According to the pixel distribution in the sub-region, the abnormal image block is located; Highlight all abnormal image blocks in the stitched image; Wherein, the stitched image is stitched together from a plurality of sub-images; The definition of pixel distribution in a sub-region is: let the k-th sub-region among N sub-regions be sbr(k), let sbr(k) contain M(k) pixels, and each pixel corresponds to a unique pixel value; let pb(i,k) represent the i-th pixel among the M(k) pixels in sbr(k), and let Vpb(i,k) represent the pixel value corresponding to pb(i,k); let the M(k) pixels contained in the sub-region sbr(k) and the M(k) pixel values ​​corresponding to these M(k) pixels be the pixel distribution of sbr(k), then N sub-regions correspond to N kinds of pixel distributions; According to the pixel distribution in the sub-region, the method for locating the abnormal image block is specifically as follows: S1, among the M(k) pixels in the sub-region sbr(k), mark the pixel with the largest pixel value pb(c1,k) and the pixel with the smallest pixel value pb(c2,k); Initialize variable h0, the initial value of h0 is set to h0=sum(sbr(k))÷(Vpb(c1,k)×Vpb(c2,k)), sum(sbr(k)) represents the sum of the pixel values ​​of M(k) pixels in the sub-region sbr(k), and go to S2; S2, record A0=Vpb(c1,k)-h0, record B0=Vpb(c2,k)+h0; among the M(k) pixels in the sub-region sbr(k), record the pixel with a pixel value less than A0 and closest to A0 as pb(x,k), and the pixel with a pixel value greater than B0 and closest to B0 as pb(y,k); mark pb(x,k) as a high visual pixel, mark pb(y,k) as a low visual pixel, and go to S3; S3, increase the value of h0 by (Vpb(x,k)÷Vpb(y,k)); if the first visual condition is met, go to S4; if the first visual condition is not met, go to S2; S4, locate the abnormal image block according to all high visual pixels and all low visual pixels; The method for locating abnormal image blocks according to all high-vision pixels and all low-vision pixels is specifically as follows: the minimum convex hull determined by all high-vision pixels is recorded as a high-vision area, and the minimum convex hull determined by all low-vision pixels is recorded as a low-vision area; If there are less than M0 low-vision pixels in the high-vision area, or if there are less than M0 high-vision pixels in the low-vision area, the minimum convex hull jointly determined by all high-vision pixels and all low-vision pixels is recorded as the abnormal image block, and the value of M0 is equal to the absolute value of the difference between the number of all high-vision pixels and the number of all low-vision pixels.

2. A method for quickly locating abnormal visual information in a spliced ​​image according to claim 1, characterized in that: The stitched area in the stitched image refers to partial image areas in multiple sub-images that are aligned in the same coordinate system after geometric transformation. These partial image areas are shared by different sub-images and contain the same or similar pixels.

3. The method for quickly locating abnormal visual information in a spliced ​​image according to claim 1, characterized in that: The method of dividing the stitched area into multiple sub-areas is specifically: dividing the stitched area into multiple sub-areas by an image segmentation algorithm based on region growing, or by an image segmentation algorithm based on edge detection, where N represents the number of the multiple sub-areas.

4. The method for quickly locating abnormal visual information in a spliced ​​image according to claim 1, characterized in that: The method for determining whether the first visual condition is satisfied is as follows: if any high visual pixel among all high visual pixels has a pixel value less than h1, or any low visual pixel among all low visual pixels has a pixel value greater than h1, then the first visual condition is satisfied, where h1=(Vpb(c1,k)+Vpb(c2,k))÷2.

5. The method for quickly locating abnormal visual information in a spliced ​​image according to claim 1, characterized in that: The method for highlighting all abnormal image blocks in a stitched image is specifically as follows: for any stitched area, all sub-areas in the stitched area are marked with abnormal image blocks, that is, the marking of all abnormal image blocks in the stitched area is completed; in the stitched image, all abnormal image blocks in all stitched areas are marked in turn, that is, the marking of all abnormal image blocks in the stitched image is completed; all abnormal image blocks marked in the stitched image are highlighted.

Images