Image fusion method and device and related equipment

Abstract

The invention relates to the technical field of image processing, in particular to an image fusion method and device and related equipment. The method comprises the following steps: respectively carrying out feature enhancement processing on a collected bright field image and a collected dark field image to obtain a feature bright field image corresponding to the bright field image and a feature dark field image corresponding to the dark field image; determining a first weight parameter of each first pixel and a second weight parameter of each second pixel according to a first pixel value of each first pixel in the feature bright field image and a second pixel value of each second pixel in the feature dark field image; according to the first weight parameter and the second weight parameter, performing image fusion processing on the bright field image and the dark field image to obtain a fused image; the technical problem that in the prior art, when a bright field image and a dark field image are used in a combined mode, the use efficiency is low can be solved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image fusion method, device and related equipment. Background Art

[0002] In the field of machine vision, the image obtained by allowing only the transmitted beam to pass through the objective aperture is called a bright field image, and the image obtained by allowing only the diffracted beam to pass through the objective aperture is called a dark field image.

[0003] In practical applications, since bright field images and dark field images have different imaging principles, unique image features can be observed in bright field images and dark field images respectively; for example, the overall morphology and outline of the crystal can be clearly observed through bright field images; crystal defects can be clearly observed through dark field images; in actual operations, bright field images and dark field images are usually used in combination; for example, crystal defects can be observed first through dark field images, and then the location, morphology, distribution and other information of the defects can be determined through bright field images.

[0004] According to practical experience, although bright field images and dark field images can be used in combination, the so-called "combined application" is just an offline summary of the unique image features observed from the two images. However, in actual operation, the two images are still observed and operated separately, resulting in low efficiency in the combined use of the two images. Summary of the invention

[0005] In view of this, the purpose of the present application is to provide an image fusion method, device and related equipment to solve the technical problem in the prior art that the bright field image and the dark field image are inefficient when used in combination.

[0006] In a first aspect, the present application provides an image fusion method, the method comprising: Performing feature enhancement processing on the collected bright-field image and dark-field image to obtain a feature bright-field image corresponding to the bright-field image and a feature dark-field image corresponding to the dark-field image; Determining a first weight parameter of each first pixel and a second weight parameter of each second pixel according to a first pixel value of each first pixel in the characteristic bright-field image and a second pixel value of each second pixel in the characteristic dark-field image; According to the first weight parameter and the second weight parameter, image fusion processing is performed on the bright field image and the dark field image to obtain a fused image.

[0007] In a second aspect, the present application provides an image fusion device, the device comprising: a feature enhancement processing module, a weight calculation module and an image fusion processing module; The feature enhancement processing module is used to perform feature enhancement processing on the collected bright field image and dark field image to obtain a feature bright field image corresponding to the bright field image and a feature dark field image corresponding to the dark field image; The weight calculation module is used to determine a first weight parameter of each first pixel and a second weight parameter of each second pixel according to a first pixel value of each first pixel in the characteristic bright-field image and a second pixel value of each second pixel in the characteristic dark-field image; The image fusion processing module is used to perform image fusion processing on the bright field image and the dark field image according to the first weight parameter and the second weight parameter to obtain a fused image.

[0008] In a third aspect, the present application provides an electronic device, comprising a processor and a memory, wherein the memory is used to store an application program, and the processor runs or executes a software program stored in the memory so that the electronic device implements the above-mentioned image fusion method.

[0009] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store program codes executed by a processor, wherein the program codes are used to implement the above-mentioned image fusion method.

[0010] Beneficial effects: The present application provides an image fusion method, comprising: performing feature enhancement processing on a collected bright field image and a dark field image respectively to obtain a feature bright field image corresponding to the bright field image and a feature dark field image corresponding to the dark field image; determining a first weight parameter of each first pixel and a second weight parameter of each second pixel according to a first pixel value of each first pixel in the feature bright field image and a second pixel value of each second pixel in the feature dark field image; performing image fusion processing on the bright field image and the dark field image according to the first weight parameter and the second weight parameter to obtain a fused image; In summary, the image fusion method provided in the present application uses the pixel value of each pixel in the characteristic bright-field image and the characteristic dark-field image as the basis for image fusion processing, and then quantifies the different image features of the bright-field image and the dark-field image based on this value, and finally performs image fusion processing on the bright-field image and the dark-field image based on the quantized image features to obtain a fused image. The fused image can simultaneously reflect the image content of the bright-field image and the dark-field image, and can also show the image features of the bright-field image and the dark-field image respectively. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. The following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0012] Figure 1 A schematic diagram of the process of the image fusion method provided in the embodiment of the present application; Figure 2 A logical schematic diagram of the image fusion method provided in an embodiment of the present application; Figure 3 A first example diagram of determining a sampling area of ​​a pixel provided in an embodiment of the present application; Figure 4 A second example diagram of determining a sampling area of ​​a pixel provided in an embodiment of the present application; Figure 5 A schematic diagram of determining a weighted sum of pixel values ​​based on pixel positions provided in an embodiment of the present application; Figure 6 A schematic diagram of the structure of an image fusion device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0013] In the prior art, electron microscopes can obtain bright field images and dark field images under different imaging conditions; the image obtained by allowing only the transmitted beam to pass through the objective lens aperture is called a bright field image, and the image obtained by allowing only the diffracted beam to pass through the objective lens aperture is called a dark field image.

[0014] In bright-field images, different areas have different diffraction capabilities for the transmitted electron beam. The difference in strength of the transmitted electron beam forms image contrast during imaging. Therefore, bright-field imaging is suitable for scenes with obvious surface features and high contrast, and is also suitable for precise measurement of the edges and shapes of objects.

[0015] In the dark field image, the background of the dark field image reflects the intensity of the diffraction beam at each point of the object, which is complementary to the bright field image. Therefore, the dark field image is suitable for detecting tiny surface defects or edge structures, as well as for contour edge detection, highlighting the edge of the object to make it contrast sharply with the background.

[0016] In actual use, since bright field images and dark field images have their own characteristics, they can be used in combination; however, the so-called "combined application" in the prior art is just to summarize and use the unique image features observed from the two images separately offline, but in actual operation, the two images are still observed and operated separately, resulting in low efficiency in the combined use of the two images.

[0017] In order to solve the above technical problems, the present application provides a technical solution for the image fusion of bright field images and dark field images. The technical solution uses the pixel value of each pixel in the characteristic bright field image and the characteristic dark field image as the basis for image fusion processing, and then quantifies the different image features of the bright field image and the dark field image based on this. Finally, based on the quantized image features, image fusion processing is performed on the bright field image and the dark field image to obtain a fused image. The fused image can simultaneously reflect the image content of the bright field image and the dark field image, and can also show the image features of the bright field image and the dark field image respectively.

[0018] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.

[0019] First, the present application provides an embodiment of an image fusion method, such as Figure 1 As shown, Figure 1 The following is a flow chart of the image fusion method provided in the embodiment of the present application. The method includes: S110 to S130, and the details are as follows: S110: performing feature enhancement processing on the collected bright field image and dark field image to obtain a feature bright field image corresponding to the bright field image and a feature dark field image corresponding to the dark field image.

[0020] Specifically, the bright field image and dark field image in the embodiments of the present application are obtained by shooting the same target sample at the same shooting distance and shooting angle using an electron microscope or other machine vision instruments (hereinafter referred to as shooting equipment).

[0021] The purpose of performing feature enhancement processing on the bright field image and the dark field image respectively is to further enhance the image features in the bright field image and the image features in the dark field image, and ensure that the fused image can show the image features of the bright field image and the dark field image respectively.

[0022] In one implementation, the bright field image and the dark field image are determined as target images; S110 includes: steps (1) to (3), the details of which are as follows: Step (1): Perform gradient processing on the target image to obtain the target gradient amplitude of each target pixel in the target image.

[0023] Specifically, the steps of determining the target gradient amplitude include: ①~②, the details are as follows: ①: Perform gradient processing on the target image to obtain the horizontal gradient and vertical gradient of each target pixel in the target image.

[0024] Among them, in actual operation, the method of gradient processing for the target image can be determined according to actual needs, such as laplacian, sobel operator, etc.; in the embodiment of the present application, the target gradient amplitude of each target pixel is calculated by the sobel operator ; Among them, use two The Sobel convolution kernel calculates the horizontal gradient of the target pixel in the target image respectively. and vertical gradient , the formulas for calculating horizontal and vertical gradients are as follows: ; ; In the formula, Represents the pixel position of the target pixel in the target image.

[0025] ②: Determine the target gradient amplitude of the target pixel based on the horizontal gradient and vertical gradient.

[0026] Among them, in determining the horizontal gradient of each target pixel and vertical gradient After that, according to the horizontal gradient and vertical gradient Calculate the target gradient magnitude of the target pixel , the formula for calculating the target gradient amplitude is as follows: .

[0027] Step (2): Determine the gradient image corresponding to the target image according to the target gradient amplitude.

[0028] Specifically, the gradient image is a gradient bright field image or a gradient dark field image; in the application embodiment, the objects to be feature enhanced are bright field images and dark field images, but the gradient bright field image and the gradient dark field image need to be assisted in the feature enhancement process, so it is necessary to determine the gradient bright field image corresponding to the bright field image and the gradient dark field image corresponding to the dark field image.

[0029] Step (3): using the gradient image as a guide image, performing guided filtering on the target image to obtain a feature image corresponding to the target image; The characteristic image is a characteristic bright field image or a characteristic dark field image.

[0030] Specifically, in the embodiment of the present application, guided filtering is used as a technical means of feature enhancement processing, and the formula for applying guided filtering is as follows: ; In the formula, Represents a feature image; represents guided filtering processing; represents the guide image, i.e., the gradient image; represents the target image; It should be emphasized that guided filtering is a processing operation that needs to be performed on the gradient bright field image and the gradient dark field image respectively. Although the same formula is used, the corresponding data needs to be used in the specific processing; for example, when When representing a characteristic bright field image, represents the gradient bright field image, Indicates bright field image.

[0031] In actual operation, guided filtering processing can emphasize the edges or fast-changing areas in the bright field image or the dark field image, and can further ensure that the fused image can reflect the image feature content of the bright field image and the dark field image.

[0032] In one implementation, before step (1), S110 further includes: step (4) to step (5), the details of which are as follows: Step (4): Collect the original bright field image and the original dark field image of the target sample.

[0033] Specifically, Figure 2 As shown, Figure 2 A logical schematic diagram of the image fusion method provided in an embodiment of the present application. In actual operation, the image fusion method is implemented by performing image registration processing on the original bright field image and the original dark field image; the original bright field image and the original dark field image refer to the bright field image and the dark field image initially obtained by the shooting device when shooting.

[0034] In actual operation, since there may be machine jitter between shooting actions at different times, the image position of the target sample in the bright field image and the dark field image is not the same; for example, when shooting the bright field image of the target sample, the target sample in the original bright field image taken by the camera device for the target sample is located in the center of the original bright field image, but when shooting the dark field image of the target sample, the shooting device moves due to factors such as accidents, resulting in the target sample in the original dark field image taken by the camera device for the target sample being located in the upper left of the original dark field image. If the above-mentioned original bright field image and original dark field image are directly used for image fusion, the resulting fused image will inevitably be inaccurate.

[0035] Step (5): Perform image registration processing on the original bright field image and the original dark field image to obtain the bright field image and the dark field image.

[0036] Specifically, in order to ensure the accuracy of the obtained fused image, it is necessary to perform image registration processing on the original bright field image and the dark field image so that the image position of the target sample in the registered bright field image and the dark field image is the same.

[0037] It should be emphasized that, in the process of image registration, only one image is needed to be used as a reference to register another image.

[0038] S120: Determine a first weight parameter of each first pixel and a second weight parameter of each second pixel according to a first pixel value of each first pixel in the characteristic bright-field image and a second pixel value of each second pixel in the characteristic dark-field image.

[0039] Specifically, in the embodiment of the present application, the image fusion processing performed on the bright field image and the dark field image is not just a simple "image superposition", but on the basis of fusing the image contents of the bright field image and the dark field image, it is also necessary to highlight the respective image features of the bright field image and the dark field image on the fused image obtained after the image fusion processing, so as to achieve the effect of "1+1>2". Therefore, in order to obtain a fused image that can show the respective image features of the bright field image and the dark field image, the embodiment of the present application uses pixels as the basic processing unit of image fusion and performs feature enhancement processing on the bright field image and the dark field image to obtain a feature bright field image and a feature dark field image; the feature bright field image and the feature dark field image emphasize the edges and textures of the bright field image and the dark field image, and therefore can serve as the basis for image fusion processing. In the embodiment of the present application, the weight parameter indicates the degree of contribution in image fusion; the larger the weight parameter, the greater the contribution of the pixel value of the pixel in image fusion, and the smaller the weight parameter, the smaller the contribution of the pixel value of the pixel in image fusion.

[0040] In one implementation, S120 includes: Step (6) to Step (8), the details of which are as follows: Step (6): for each first pixel, determine the first energy value of the first pixel according to the first pixel value.

[0041] Specifically, in the embodiment of the present application, the energy value is used to indicate the role of the pixel in displaying the image characteristics; the larger the energy value, the greater the role of the pixel value of the pixel in displaying the image characteristics, and the smaller the energy value, the smaller the role of the pixel value of the pixel in displaying the image characteristics.

[0042] In one implementation, step (6) includes: step (6.1) to step (6.2), the details of which are as follows: Step (6.1): Determine the first pixel in the characteristic bright field image The corresponding first sampling area ; in, Indicates the first pixel The pixel position in the characteristic bright field image; the sampling area includes a plurality of first pixels, the first pixels Relative to the sampling area The relative positions of the components meet the preset standards.

[0043] Specifically, in the example of the present application, the sampling area is an area constructed for determining the energy value of a pixel, which does not actually exist in the characteristic bright field image and the characteristic dark field image; Figure 3 As shown, Figure 3 The first example diagram of determining the sampling area of ​​a pixel provided in the embodiment of the present application is as follows: Figure 3 The first pixel in the characteristic bright field image shown Take the example to explain the steps of determining the sampling area of ​​a pixel; To determine the first pixel The first sampling area, first need to confirm the area shape of the first sampling area, Figure 3 The area shape of the first sampling area shown in FIG is a square, but in actual operation, other area shapes can be selected; after determining the area shape of the first sampling area, it is also necessary to determine the first pixel The relative position in its corresponding first sampling area, Figure 3 The first pixel in At the central position of its first sampling area, but in actual operation, other relative positions can be selected, that is, the "preset standard" can be determined according to actual needs.

[0044] exist Figure 3 The first pixel The first pixel is located closer to the center of the characteristic bright field image, so the other first pixels included in the corresponding first sampling area are actually present. However, in actual operation, for the first pixel located at the edge of the characteristic bright field image, its assignment element needs to be determined when determining its first sampling area; like Figure 4 As shown, Figure 4 A second example diagram for determining a sampling area of ​​a pixel provided in an embodiment of the present application, Figure 4 First Pixel Being at the edge position of the characteristic bright field image, the corresponding first sampling area lacks sufficient first pixels, so it is necessary to construct an assigned pixel in the first sampling area. The assigned pixel means an artificially constructed pixel; it can be the edge pixel value or other value of the characteristic image; in actual operation, the pixel value assigned to the assigned pixel can be determined according to actual needs.

[0045] Step (6.2): ​​The first sampling area The square sum of the first pixel values ​​of all the first pixels included is determined as the first pixel The first energy value of The formula for determining the first energy value is as follows: ; In the formula, Indicates the first pixel The first energy value of A first pixel value representing a first pixel in a first sampling region; Represents the first sampling area of ​​the square The side length of .

[0046] Specifically, in the embodiment of the present application, the first pixel The corresponding first sampling area The square sum of the first pixel values ​​of all the first pixels included is determined as the first pixel The first energy value.

[0047] In actual operation, steps (6.1) to (6.2) may be repeated along the arrangement direction of the plurality of first pixels in the characteristic bright field image to determine the first energy value of each first pixel in turn.

[0048] Step (7): for each second pixel, determine the second energy value of the second pixel according to the second pixel value.

[0049] Specifically, the step of determining the second energy value of the second pixel can refer to step (6.1) to step (6.2), which will not be repeated here.

[0050] Step (8): for a first pixel and a second pixel at the same pixel position, determining a first weight parameter of the first pixel and a second weight parameter of the second pixel according to a first energy value of the first pixel and a second energy value of the second pixel; The pixel position indicates the position of the pixel in the image.

[0051] Specifically, after respectively determining the first energy value of each first pixel in the characteristic bright field image and the second energy value of each second element in the characteristic dark field image, it is necessary to collaboratively determine the first weight parameter corresponding to the first pixel and the second weight parameter corresponding to the second pixel based on the pixel position and the first energy value and the second energy value; In the embodiment of the present application, the step of determining the weight parameter is performed based on the pixel position, such as Figure 5 As shown, Figure 5 A schematic diagram of determining a weighted sum of pixel values ​​based on pixel positions provided in an embodiment of the present application, Figure 5 The middle is a schematic diagram of pixel positions. Figure 5 The upper right center is a characteristic bright field image. Figure 5 The lower right middle image is a characteristic dark field image. When the first pixel needs to be determined The first weight parameter needs to be based on the same pixel position Determine the pixel position in the dark field image of the feature The second pixel The second energy value of the first pixel The first energy value and the second pixel The second energy value of the first pixel is determined The first weight parameter and the second pixel The second weight parameter of .

[0052] In one implementation, step (8) includes: step (8.1) to step (8.2), the details of which are as follows: Step (8.1): For a first pixel and a second pixel at the same pixel position, determine a ratio between the first energy value and the sum of the first energy value and the second energy value as a first weight parameter of the first pixel.

[0053] Specifically, in the embodiment of the present application, the sum of the first energy value of the first pixel and the second energy value of the second pixel at the same pixel position is first determined, and then the ratio between the first element value of the first pixel and the sum of the energy values ​​is determined as the first weight parameter of the first element; The formula for determining the first weight parameter is as follows: ; In the formula, in the formula, Indicates the first pixel The first energy value of Represents the second pixel The second energy value of Indicates the first pixel The corresponding first weight parameter.

[0054] Step (8.2): For a first pixel and a second pixel at the same pixel position, determine a ratio between the second energy value and the sum of the first energy value and the second energy value as a second weight parameter of the second pixel.

[0055] Specifically, the step of determining the second energy value of the second pixel can refer to step (8.1), which will not be repeated here; wherein, the formula for determining the second weight parameter is as follows: ; In the formula, Represents the second pixel The corresponding second weight parameter.

[0056] S130: performing image fusion processing on the bright field image and the dark field image according to the first weight parameter and the second weight parameter to obtain a fused image.

[0057] Specifically, in the embodiment of the present application, the weight parameters determined through the aforementioned steps are used to guide the image fusion processing of the bright field image and the dark field image; as mentioned above, the weight parameters indicate the contribution degree of the pixel in the image fusion, so the process of image fusion processing is the process in which the pixels at each pixel position in the bright field image and the dark field image contribute their pixel values ​​according to their own contribution degree.

[0058] In one implementation, S140 includes: Step (9) to Step (12), the details of which are as follows: Step (9): Determine the third pixel value of each third pixel in the bright field image and the fourth pixel value of each fourth pixel in the dark field image.

[0059] Specifically, it should be emphasized that since the bright field image and the characteristic bright field image are different images, and the dark field image and the characteristic dark field image are also different images, the pixel in the bright field image is defined as the third pixel, and the pixel in the dark field image is defined as the fourth pixel.

[0060] Step (10): for the first pixel, the second pixel, the third pixel and the fourth pixel at the same pixel position, determine the weighted sum of the pixel values ​​corresponding to the pixel position according to the first weight parameter and the third pixel value, the second weight parameter and the fourth pixel value.

[0061] Specifically, after determining the first weight parameter of each first pixel, the second weight parameter of each second pixel, the third pixel value of each third pixel, and the fourth pixel value corresponding to each fourth pixel, as follows: Figure 5 As shown, the weighted sum of the pixel values ​​corresponding to each pixel position can be determined in sequence according to the pixel position; wherein the formula for determining the weighted sum of the pixel values ​​is as follows: ; In the formula, Represents the weighted sum of pixel values; represents a third pixel value; Represents the fourth pixel value.

[0062] Step (11): Determine the pixel value of each pixel position in the fused image as the weighted sum of the pixel values ​​corresponding to each pixel position.

[0063] Step (12): Determine the fused image based on the pixel value of each pixel position.

[0064] Specifically, after the pixel value of each pixel position in the fused image is determined, imaging can be performed according to the pixel value to obtain the fused image.

[0065] In a second aspect, the present application provides an image fusion device, such as Figure 6 As shown, Figure 6This is a structural schematic diagram of an image fusion device provided in an embodiment of the present application, the device comprising: a feature enhancement processing module 310, a weight calculation module 320 and an image fusion processing module 330; The feature enhancement processing module 310 is used to perform feature enhancement processing on the collected bright field image and dark field image to obtain a feature bright field image corresponding to the bright field image and a feature dark field image corresponding to the dark field image; A weight calculation module 320, for determining a first weight parameter of each first pixel and a second weight parameter of each second pixel according to a first pixel value of each first pixel in the characteristic bright field image and a second pixel value of each second pixel in the characteristic dark field image; The image fusion processing module 330 is used to perform image fusion processing on the bright field image and the dark field image according to the first weight parameter and the second weight parameter to obtain a fused image.

[0066] In one implementation, the bright field image and the dark field image are determined as the target image; the feature enhancement processing module 310 is further used to perform gradient processing on the target image to obtain the target gradient amplitude of each target pixel in the target image; The feature enhancement processing module 310 is further used to determine a gradient image corresponding to the target image according to the target gradient amplitude; The feature enhancement processing module 310 is further used to use the gradient image as a guide image and perform a guide filtering process on the target image to obtain a feature image corresponding to the target image; The characteristic image is a characteristic bright field image or a characteristic dark field image.

[0067] In one implementation, the feature enhancement processing module 310 is further used to collect an original bright field image and an original dark field image of the target sample; The feature enhancement processing module 310 is further used to perform image registration processing on the original bright field image and the original dark field image to obtain the bright field image and the dark field image.

[0068] In one implementation, the weight calculation module 320 is further configured to determine, for each first pixel, a first energy value of the first pixel according to the first pixel value; The weight calculation module 320 is further used to determine, for each second pixel, a second energy value of the second pixel according to the second pixel value; The weight calculation module 320 is further used to determine, for a first pixel and a second pixel at the same pixel position, a first weight parameter of the first pixel and a second weight parameter of the second pixel according to a first energy value of the first pixel and a second energy value of the second pixel; The pixel position indicates the position of the pixel in the image.

[0069] In one implementation, the weight calculation module 320 is further configured to determine the first pixel in the characteristic bright field image. The corresponding first sampling area ; in, Indicates the first pixel The pixel position in the characteristic bright field image; the sampling area includes a plurality of first pixels, the first pixels Relative to the sampling area The relative position of meets the preset standard; The weight calculation module 320 is further configured to calculate the first sampling area The square sum of the first pixel values ​​of all the first pixels included is determined as the first pixel The first energy value of The formula for determining the first energy value is as follows: ; In the formula, Indicates the first pixel The first energy value of A first pixel value representing a first pixel in a first sampling region; Represents the first sampling area of ​​the square The side length of .

[0070] In one implementation, the weight calculation module 320 is further configured to determine, for a first pixel and a second pixel at the same pixel position, a ratio between the first energy value and the sum of the first energy value and the second energy value as a first weight parameter of the first pixel; The weight calculation module 320 is further configured to determine, for a first pixel and a second pixel at the same pixel position, a ratio between the second energy value and the sum of the first energy value and the second energy value as a second weight parameter of the second pixel; The formula for determining the weight parameter is as follows: ; ; In the formula, Indicates the first pixel The first energy value of Represents the second pixel The second energy value of Indicates the first pixel The first weight parameter of Represents the second pixel The second weight parameter of .

[0071] In one implementation, the image fusion processing module 330 is further used to determine a third pixel value of each third pixel in the bright field image and a fourth pixel value of each fourth pixel in the dark field image; The image fusion processing module 330 is further used to determine, for the first pixel, the second pixel, the third pixel and the fourth pixel at the same pixel position, a weighted sum of pixel values ​​corresponding to the pixel position according to the first weight parameter and the third pixel value, the second weight parameter and the fourth pixel value; The image fusion processing module 330 is further used to determine the pixel value of each pixel position in the fused image by weighting the pixel value corresponding to each pixel position; The fused image is determined according to the pixel value of each pixel position.

[0072] Third, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, steps S110 to S130 provided in the above embodiment are implemented.

[0073] Fourth, the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, steps S110 to S130 of the above embodiment are executed.

[0074] Fifth, the computer program product provided in the present application includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the method in the previous method embodiment. The specific implementation can refer to steps S110~S130 of the method embodiment, which will not be repeated here.

[0075] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0076] In addition, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0077] Furthermore, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0078] It should be noted that if the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0079] In this document, relational terms such as first and second, etc. are used merely to distinguish one entity or operation from another entity or operation, but do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0080] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An image fusion method, characterized in that: The method comprises: Performing feature enhancement processing on the collected bright-field image and dark-field image to obtain a feature bright-field image corresponding to the bright-field image and a feature dark-field image corresponding to the dark-field image; Determining a first energy value of each first pixel and a second energy value of each second pixel according to a first pixel value of each first pixel in the characteristic bright-field image and a second pixel value of each second pixel in the characteristic dark-field image; For the first pixel and the second pixel at the same pixel position in the characteristic bright-field image and the characteristic dark-field image, determine a first weight parameter of the first pixel and a second weight parameter of the second pixel according to the first energy value of the first pixel and the second energy value of the second pixel; Wherein, the pixel position indicates the position of the pixel in the image; According to the first weight parameter and the second weight parameter, image fusion processing is performed on the bright field image and the dark field image to obtain a fused image.

2. The method according to claim 1, characterized in that The bright field image and the dark field image are determined as target images; and the collected bright field image and dark field image are subjected to feature enhancement processing to obtain a feature bright field image corresponding to the bright field image and a feature dark field image corresponding to the dark field image, respectively, including: Performing gradient processing on the target image to obtain a target gradient amplitude of each target pixel in the target image; Determining a gradient image corresponding to the target image according to the target gradient amplitude; Using the gradient image as a guide image, performing a guide filtering process on the target image to obtain a feature image corresponding to the target image; Wherein, the characteristic image is the characteristic bright field image or the characteristic dark field image.

3. The method according to claim 2, characterized in that Before performing gradient processing on the target image to obtain a target gradient amplitude of each target pixel in the target image, the method further includes: Collecting an original bright field image and an original dark field image of a target sample; Image registration processing is performed on the original bright-field image and the original dark-field image to obtain the bright-field image and the dark-field image.

4. The method according to claim 1, characterized in that: The determining, for each of the first pixels, a first energy value of the first pixel according to the first pixel value includes: Determine the first pixel in the characteristic bright field image The corresponding first sampling area ; in, Indicates the first pixel The sampling area includes a plurality of the first pixels, the first pixels Relative to the sampling area The relative position of meets the preset standard; The first sampling area The square sum of the first pixel values ​​of all the first pixels included is determined as the first pixel The first energy value of The formula for determining the first energy value is as follows: ； In the formula, Indicates the first pixel The first energy value of a first pixel value representing the first pixel in the first sampling area; Said The first sampling area represents a square The side length of .

5. The method according to claim 4, characterized in that The determining, for the first pixel and the second pixel at the same pixel position, a first weight parameter of the first pixel and a second weight parameter of the second pixel according to the first energy value of the first pixel and the second energy value of the second pixel, comprises: For the first pixel and the second pixel at the same pixel position, determine a ratio between the first energy value and the sum of the first energy value and the second energy value as a first weight parameter of the first pixel; For the first pixel and the second pixel at the same pixel position, determine a ratio between the second energy value and the sum of the first energy value and the second energy value as a second weight parameter of the second pixel; The formula for determining the weight parameter is as follows: ； ； In the formula, Indicates the first pixel The first energy value of The second pixel The second energy value of Indicates the first pixel The first weight parameter of The second pixel The second weight parameter.

6. The method according to claim 1, characterized in that The performing image fusion processing on the bright field image and the dark field image according to the first weight parameter and the second weight parameter to obtain a fused image includes: Determining a third pixel value of each third pixel in the bright-field image and a fourth pixel value of each fourth pixel in the dark-field image; For the first pixel, the second pixel, the third pixel, and the fourth pixel at the same pixel position, determine a weighted sum of pixel values ​​corresponding to the pixel position according to the first weight parameter and the third pixel value, the second weight parameter, and the fourth pixel value; Determine the weighted sum of the pixel values ​​corresponding to each pixel position as the pixel value of each pixel position in the fused image; The fused image is determined according to the pixel values ​​of the respective pixel positions.

7. An image fusion device, characterized in that: The device comprises: a feature enhancement processing module, a weight calculation module and an image fusion processing module; The feature enhancement processing module is used to perform feature enhancement processing on the collected bright field image and dark field image to obtain a feature bright field image corresponding to the bright field image and a feature dark field image corresponding to the dark field image; The weight calculation module is used to determine a first weight parameter of each first pixel and a second weight parameter of each second pixel according to a first pixel value of each first pixel in the characteristic bright-field image and a second pixel value of each second pixel in the characteristic dark-field image; The image fusion processing module is used to perform image fusion processing on the bright field image and the dark field image according to the first weight parameter and the second weight parameter to obtain a fused image.

8. An electronic device, characterized in that: The electronic device comprises a processor and a memory, the memory is used to store an application program, and the processor runs or executes a software program stored in the memory so that the electronic device implements the image fusion method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store program codes executed by a processor, and the program codes are used to implement the image fusion method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The computer program product comprises program codes, and when a computer runs the computer program product, the computer executes the image fusion method according to any one of claims 1 to 6.

Images