Image fusion method

Abstract

The invention provides an image fusion method, which comprises the following steps of: aligning a plurality of images in a global coordinate system, traversing each pixel point in one of the images, recording the current traversed pixel point as a pixel point A, collecting gray information and binarization information at the pixel point A, and forming a gray information sequence and a binarization information sequence; according to the distribution condition of the background point binarization information and the foreground point binarization information in the binarization information sequence, gray values at the pixel point A are fused; obtaining a gray value of each pixel point under the global coordinate system; according to the method, binarization information is introduced, brightness changes reflected by different images are captured, and targeted fusion is carried out on the gray values of the pixel points in the overlapping area, so that the brightness of the fused image is uniform, detail information is richer, the accuracy of the gray information is improved, and the image quality is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an image fusion method. Background Art

[0002] In recent years, machine vision equipment has formed high-quality solutions in various industrial assembly lines with its speed, accuracy and high reliability. In various scenarios including measurement, detection and guidance, the images captured by industrial cameras are often the most important source of data analysis.

[0003] In the inspection scenario of large-size workpieces, due to the limited field of view of a single camera, it is impossible to obtain the full picture of the object to be tested. Therefore, it is often necessary to move the camera / increase the number of cameras, or move the workpiece to be tested, take multiple images of the surface of the object to be tested, and then convert the multiple images to a unified coordinate system to achieve image fusion. During the fusion process, the overlapping areas between the images will have the following phenomenon: multiple grayscale values ​​exist at the same pixel point at the same time; therefore, how to determine the final grayscale value of the pixel point has become a technical problem that needs to be solved urgently.

[0004] The existing fusion algorithm only performs simple grayscale mean fusion, maximum fusion or local interpolation fusion in the overlapping area. This algorithm cannot capture the illumination changes reflected in different images, and it is difficult to retain local features in a biased manner. The fused image has problems such as uneven brightness, faults, and loss of local details, which affects the image quality. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides an image fusion method. The method introduces binary information, captures the brightness changes reflected by different images, and performs targeted fusion of the grayscale values ​​of pixel points in the overlapping area, so that the brightness of the fused image is uniform and the detail information is richer, thereby improving the accuracy of the grayscale information and the image quality.

[0006] The technical solution is as follows:

[0007] An image fusion method aligns multiple images in a global coordinate system, wherein at least two images are continuous and partially overlapped; the image is a dual-channel image, and the pixel points in the image contain both binary information and grayscale information; the binary information is a foreground point binary value / background point binary value;

[0008] Use the following steps to perform grayscale fusion:

[0009] S1. In the global coordinate system, traverse each pixel in one of the images, record the currently traversed pixel as pixel A, and perform the following processing at pixel A:

[0010] Collect grayscale information and binary information at pixel point A to form a grayscale information sequence and a binary information sequence;

[0011] If the grayscale information sequence contains only one grayscale information, then keep the grayscale information. Otherwise, determine whether the binary information sequence contains both background point binary information and foreground point binary information:

[0012] If yes, assign weights to each grayscale value in the grayscale information sequence, and add the weighted grayscale values ​​of each overlapping image at pixel point A as the fused grayscale value;

[0013] If not, the grayscale mean or maximum value in the grayscale information sequence is used as the grayscale value after fusion, or weights are assigned to each grayscale value in the grayscale information sequence, and the grayscale values ​​are weighted and added as the grayscale value after fusion;

[0014] S2. Execute step S1 for each dual-channel image to obtain the grayscale value of each pixel in the global coordinate system.

[0015] Furthermore, in step S1, if the binary information sequence includes both background point binary information and foreground point binary information, the weights are assigned to the grayscale values ​​in the grayscale information sequence as follows:

[0016] The weight of the grayscale information corresponding to the background point binary information is set to 0;

[0017] According to the sorting interval between each foreground point binary information and the background point binary information, a weight is set for the grayscale information corresponding to each foreground point binary information, wherein the size of the weight is negatively correlated with the size of the sorting interval.

[0018] Preferably, in the binary information sequence, if there are multiple background point binary information;

[0019] According to the sorting interval between the foreground point binary information and the most adjacent background point binary information, a weight is set for the grayscale information corresponding to each foreground point binary information.

[0020] Furthermore, in step S1, if the binary information sequence only includes foreground point binary information, the weights are assigned to the grayscale values ​​in the grayscale information sequence as follows:

[0021] In the binary information sequence, two background point binary information are added, which are respectively located before the first binary information and after the last binary information in the binary information sequence;

[0022] According to the sorting interval between the foreground point binary information and the nearest background point binary information, a weight is set for the grayscale information corresponding to each foreground point binary information, wherein the size of the weight is negatively correlated with the size of the sorting interval.

[0023] Furthermore, the sorting interval between the binary information of the foreground points and the binary information of the background points is denoted as H j , j = 1, 2...M, M represents the number of foreground point binary information in the binary information sequence;

[0024]

[0025] Furthermore, in step S1, if the binary information sequence only includes background point binary information, the weights are assigned to each grayscale information in the grayscale information sequence as follows:

[0026] Subtract each grayscale information in the grayscale information sequence from the grayscale mean value corresponding to the grayscale information sequence to obtain a difference Si, i=1, 2...N, where N represents the number of grayscale information in the grayscale information sequence;

[0027] If the difference Si≤0, the weight corresponding to the i-th grayscale information is 0;

[0028] If the difference Si>0, then Where D is the sum of all differences greater than 0.

[0029] Preferably, in the global coordinate system, the multiple images are aligned in sequence according to the adjacency relationship between the acquisition timing / acquisition postures of each overlapping image;

[0030] If pixel A is in the non-overlapping area, there is only one grayscale information and one binary information at pixel A;

[0031] If the pixel point A is in the overlapping area, the images that overlap at the pixel point A are recorded as overlapping images;

[0032] According to the adjacency relationship between the acquisition timing / acquisition postures of each overlapping image, the overlapping images are sorted, and then the binary information and grayscale information of each overlapping image at the pixel point A are extracted in turn to form a binary information sequence and a grayscale information sequence.

[0033] Preferably, in step S1, the dual-channel image is obtained by:

[0034] Binarize the image to obtain the binary information of each pixel;

[0035] The binary information and grayscale information of each pixel are stored correspondingly to form a dual-channel image.

[0036] This method introduces binary information and uses dual-channel images for fusion. When processing pixels in overlapping areas, targeted fusion is performed based on the distribution (especially jump conditions) of foreground point binary information and background point binary information in the binary information sequence. If there is a jump condition in the sequence (both foreground point binary information and background point binary information exist at the same time), it means that at this position, the brightness and darkness corresponding to different images are different and need to be paid special attention to. In this solution, for such situations, a weighted fusion method is designed to make the grayscale value of the pixel point more restore the real brightness information and improve the image quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1a This is the processing effect diagram obtained by grayscale mean fusion;

[0038] Figure 1b This is the processing effect diagram obtained by grayscale mean fusion;

[0039] Figure 1c This is a processing effect diagram obtained by integrating the method in the specific implementation mode of the present invention. DETAILED DESCRIPTION

[0040] The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0041] An image fusion method aligns multiple images in a global coordinate system, wherein at least two images have continuous content and partially overlap;

[0042] In a specific implementation, the image is a dual-channel image, and the pixel points in the image contain both binary information and grayscale information; the binary information is a foreground point binary value (such as 255) / background point binary value (such as 0);

[0043] Use the following steps to perform grayscale fusion:

[0044] S1. In the global coordinate system, traverse each pixel in one of the images, record the currently traversed pixel as pixel A, and perform the following processing at pixel A:

[0045] Collect grayscale information and binary information at pixel point A to form a grayscale information sequence and a binary information sequence;

[0046] If the grayscale information sequence contains only one grayscale information, then keep the grayscale information. Otherwise, determine whether the binary information sequence contains both background point binary information and foreground point binary information:

[0047] If yes, assign weights to each grayscale value in the grayscale information sequence, and add the weighted grayscale values ​​of each overlapping image at pixel point A as the fused grayscale value;

[0048] If not, the grayscale mean or maximum value in the grayscale information sequence is used as the grayscale value after fusion, or weights are assigned to each grayscale value in the grayscale information sequence, and the grayscale values ​​are weighted and added as the grayscale value after fusion;

[0049] S2. Execute step S1 for each dual-channel image to obtain the grayscale value of each pixel in the global coordinate system.

[0050] Among them, the dual-channel image is obtained by the following method:

[0051] Binarize the image to obtain the binary information of each pixel;

[0052] The binary information and grayscale information of each pixel are stored correspondingly to form a dual-channel image.

[0053] In a specific implementation, before or after image alignment, each image is threshold segmented using a mean clustering or maximizing inter-class variance algorithm to obtain a binary image, and then morphological opening and closing operations are performed to denoise the binary image to obtain binary information of each pixel; the binary information of each pixel is stored correspondingly to the grayscale information to form a dual-channel image.

[0054] Specifically, the binary information sequence includes the following three situations. In this embodiment, exemplary explanations are given for different situations:

[0055] Scenario 1:

[0056] If the binary information sequence contains both background point binary information and foreground point binary information, the weights are assigned to each grayscale value in the grayscale information sequence as follows:

[0057] Set the weight of the grayscale information corresponding to the background point binary information (0) to 0;

[0058] According to the sorting interval between each foreground point binary information and the background point binary information, a weight is set for the grayscale information corresponding to each foreground point binary information, wherein the size of the weight is negatively correlated with the size of the sorting interval.

[0059] Specially, in the binary information sequence, if there are multiple background point binary information (0);

[0060] According to the sorting interval between the foreground point binary information and the most adjacent background point binary information, a weight is set for the grayscale information corresponding to each foreground point binary information.

[0061] For example, taking the case where 7 images overlap each other at pixel point A, there are 7 binary information in the binary information sequence, of which the third and sixth are both background point binary information 0, as shown in the following table:

[0062] Sorting number 1 2 3 4 5 6 7 Binarized information sequence 255 255 0 255 255 0 255 Sorting interval 2 1 / 1 1 / 1

[0063] Scenario 2:

[0064] If the binary information sequence only contains the foreground point binary information, the weights are assigned to each grayscale value in the grayscale information sequence as follows:

[0065] In the binary information sequence, two background point binary information (0) are added, which are respectively located before the first binary information and after the last binary information in the binary information sequence;

[0066] According to the sorting interval between the foreground point binary information and the nearest background point binary information, a weight is set for the grayscale information corresponding to each foreground point binary information, wherein the size of the weight is negatively correlated with the size of the sorting interval.

[0067] For example, taking the case where 7 images overlap at pixel point A, there are 7 binary information in total in the binary information sequence, all of which are foreground point binary information 255. After adding two background point binary information (0), the following table is shown:

[0068]

[0069] More specifically, in the above two cases, the weights are calculated as follows:

[0070] The sorting interval between the jth foreground point binary information and the background point binary information is H j , j = 1, 2...M, M represents the number of foreground point binary information in the binary information sequence;

[0071]

[0072] For example, taking the case where 7 images overlap each other at pixel point A, there are 7 binary information in the binary information sequence, of which the third and sixth are both background point binary information 0, as shown in the following table:

[0073] Sorting number 1 2 3 4 5 6 7 Binarized information sequence 255 255 0 255 255 0 255 Sorting interval 2 1 / 1 1 / 1 Weight 2 / 9 1 / 9 0 1 / 9 1 / 9 0 1 / 9

[0074] by

[0075] Scenario 3:

[0076] If the binary information sequence only contains background point binary information, the weights are assigned to each grayscale information in the grayscale information sequence as follows:

[0077] The i-th grayscale information in the grayscale information sequence is subtracted from the grayscale mean value corresponding to the grayscale information sequence to obtain a difference Si, i=1, 2...N, where N represents the number of grayscale information in the grayscale information sequence;

[0078] If the difference Si≤0, the weight corresponding to the i-th grayscale information is 0;

[0079] If the difference Si>0, then Where D is the sum of all differences greater than 0.

[0080] For example, taking the case where 7 images overlap each other at pixel point A, there are 7 binary information in the binary information sequence, of which the third and sixth are both background point binary information 0, as shown in the following table:

[0081] Sorting number 1 2 3 4 5 6 7 Binarized information sequence 0 0 0 0 0 0 0 Grayscale information sequence 20 20 20 15 15 20 30 Difference 0 0 0 -5 -5 0 10 Weight 0 0 0 0 0 0 1

[0082] In order to obtain a better fusion effect, as a preferred implementation:

[0083] In the global coordinate system, multiple images are aligned in sequence according to the adjacency relationship between the acquisition timing / image acquisition postures of each overlapping image;

[0084] If pixel A is in the non-overlapping area, there is only one grayscale information and one binary information at pixel A;

[0085] If the pixel point A is in the overlapping area, the images that overlap at the pixel point A are recorded as overlapping images;

[0086] According to the adjacency relationship between the acquisition timing / acquisition postures of each overlapping image, the overlapping images are sorted, and then the binary information and grayscale information of each overlapping image at the pixel point A are extracted in turn to form a binary information sequence and a grayscale information sequence.

[0087] In order to further illustrate the processing effect of this method, in this embodiment, the existing method and this method are used to process the image at the same time, and the processing effect diagram is as follows: Figure 1a to Figure 1c .

[0088] Among them, Figure 1a The grayscale mean of each pixel is calculated as the grayscale value after fusion, and the corresponding processing effect diagram is obtained; this method cannot retain the glossiness of the image itself, the contrast is poor, and there are obvious faults;

[0089] Figure 1b The maximum grayscale value of each pixel is used as the grayscale value after fusion, and the corresponding processing effect diagram is obtained; the noise of this method is obvious and it is easy to cause loss of details, such as the blurred text and missing details in Figure b;

[0090] Figure 1c In order to obtain the processing effect diagram by using the method fused in the specific implementation mode of the present invention, the method performs targeted fusion of the grayscale values ​​of the pixel points, so that the brightness of the fused image is uniform, there is no fault, and more details are retained, and the fusion result is more accurate.

[0091] The foregoing description of the specific exemplary embodiments of the present invention is presented for the purpose of illustration and description. The foregoing description is not intended to be exhaustive, nor is it intended to limit the present invention to the precise form disclosed, and it is apparent that many changes and variations are possible in light of the above teachings. The exemplary embodiments are selected and described to explain the specific principles of the present invention and its practical application, so that other technicians in the field can realize and utilize the various exemplary embodiments of the present invention and its different selected forms and modified forms. The scope of the present invention is intended to be limited by the appended claims and their equivalents.

Claims

1. An image fusion method aligns multiple images in a global coordinate system, where: The contents of at least two images are continuous and partially overlapped; the characteristics are: the image is a dual-channel image, and the pixel points in the image contain both binary information and grayscale information; the binary information is the foreground point binary value / background point binary value; Use the following steps to perform grayscale fusion: S1. In the global coordinate system, traverse each pixel in one of the images, record the currently traversed pixel as pixel A, and perform the following processing at pixel A: Collect grayscale information and binary information at pixel point A to form a grayscale information sequence and a binary information sequence; If the grayscale information sequence contains only one grayscale information, then keep the grayscale information. Otherwise, determine whether the binary information sequence contains both background point binary information and foreground point binary information: If yes, assign weights to each grayscale value in the grayscale information sequence, and add the weighted grayscale values ​​of each overlapping image at pixel point A as the fused grayscale value; If not, the grayscale mean or maximum value in the grayscale information sequence is used as the grayscale value after fusion, or weights are assigned to each grayscale value in the grayscale information sequence, and the grayscale values ​​are weighted and added as the grayscale value after fusion; S2. Execute step S1 for each dual-channel image to obtain the grayscale value of each pixel in the global coordinate system.

2. The image fusion method according to claim 1, characterized in that: In step S1, if the binary information sequence includes both background point binary information and foreground point binary information, the weights are assigned to the grayscale values ​​in the grayscale information sequence as follows: The weight of the grayscale information corresponding to the background point binary information is set to 0; According to the sorting interval between each foreground point binary information and the background point binary information, a weight is set for the grayscale information corresponding to each foreground point binary information, wherein the size of the weight is negatively correlated with the size of the sorting interval.

3. The image fusion method according to claim 2, characterized in that: In the binary information sequence, if there are multiple background point binary information; According to the sorting interval between the foreground point binary information and the most adjacent background point binary information, a weight is set for the grayscale information corresponding to each foreground point binary information.

4. The image fusion method according to claim 1, characterized in that: In step S1, if the binary information sequence only includes foreground point binary information, the weights are assigned to the grayscale values ​​in the grayscale information sequence as follows: In the binary information sequence, two background point binary information are added, which are respectively located before the first binary information and after the last binary information in the binary information sequence; According to the sorting interval between the foreground point binary information and the nearest background point binary information, a weight is set for the grayscale information corresponding to each foreground point binary information, wherein the size of the weight is negatively correlated with the size of the sorting interval.

5. The image fusion method according to claim 2 or 4, characterized in that: The sorting interval between the binary information of the foreground points and the binary information of the background points is H j , j = 1, 2...M, M represents the number of foreground point binary information in the binary information sequence; 6. The image fusion method according to claim 1, characterized in that: In step S1, if the binary information sequence only contains background point binary information, the weights are assigned to each grayscale information in the grayscale information sequence as follows: Subtract each grayscale information in the grayscale information sequence from the grayscale mean value corresponding to the grayscale information sequence to obtain a difference value Si, i=1, 2...N, where N represents the number of grayscale information in the grayscale information sequence; If the difference Si≤0, the weight corresponding to the i-th grayscale information is 0; If the difference Si>0, then Where D is the sum of all differences greater than 0.

7. The image fusion method according to claim 1, characterized in that: In the global coordinate system, multiple images are aligned in sequence according to the adjacency relationship between the acquisition timing / image acquisition postures of each overlapping image; If pixel A is in the non-overlapping area, there is only one grayscale information and one binary information at pixel A; If the pixel point A is in the overlapping area, the images that overlap at the pixel point A are recorded as overlapping images; According to the adjacency relationship between the acquisition timing / acquisition postures of each overlapping image, the overlapping images are sorted, and then the binary information and grayscale information of each overlapping image at the pixel point A are extracted in turn to form a binary information sequence and a grayscale information sequence.

8. The image fusion method according to claim 1, characterized in that: In step S1, the dual-channel image is obtained by: Binarize the image to obtain the binary information of each pixel; The binary information and grayscale information of each pixel are stored correspondingly to form a dual-channel image.

Images