Image fusion method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an image fusion method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a plurality of transmission images obtained by detecting a detected object by a plurality of detectors; for each transmission image, determining the thickness corresponding to each pixel point in the transmission image according to the gray value of each pixel point in the transmission image and a pre-established first corresponding relationship between the gray value and the thickness, wherein the first corresponding relationship corresponds to the transmission image and is used for representing the gray value and the thickness; for each fusion pixel point in a to-be-generated fusion image, determining a target thickness corresponding to the fusion pixel point based on the thickness corresponding to the basic pixel point; and for each fused pixel point, determining a target gray value corresponding to the fused pixel point according to the target thickness corresponding to the fused pixel point and a preset second corresponding relationship between the thickness and the gray value, and obtaining a fused image. Therefore, the image quality of the obtained fused image is improved, and detail information in various detected objects with different thicknesses can be clearly displayed.

Description

Technical Field

[0001] The present application relates to the field of image processing technologies, and in particular, to an image fusion method, apparatus, electronic device, and storage medium. Background Art

[0002] Using a security inspection machine to detect the items carried by personnel is an important measure to ensure public safety. The security inspection machine includes an X-ray source and a detector. For example, a dual-energy security inspection machine includes an X-ray source and a dual-energy detection device, and the dual-energy detection device can be composed of a low-energy detector, a copper sheet, and a high-energy detector.

[0003] During the detection process, first, the item is placed into the security inspection machine. When the item is conveyed to the detection area, the X-rays emitted by the X-ray source can pass through the item and be detected by the detector, and then a transmission image is generated based on the X-ray information detected by the detector. If there are multiple detectors, multiple transmission images will be detected. Furthermore, the multiple transmission images are fused, and the internal detailed information of the item is viewed using the fused image to eliminate potential public safety hazards.

[0004] However, currently, when using an X-ray detection device (including but not limited to a security inspection machine) equipped with multiple detectors to scan an item to be inspected, the fused image of the item to be inspected obtained finally has the problem of low fused image quality, and thus the internal detailed information of the item cannot be clearly displayed. Summary of the Invention

[0005] The purpose of the embodiments of the present application is to provide an image fusion method, apparatus, electronic device, and storage medium to improve the image fusion quality and clearly display the internal detailed information of various items to be inspected with different thicknesses. The specific technical solutions are as follows:

[0006] In a first aspect, the embodiments of the present application provide an image fusion method, and the method includes:

[0007] Obtain multiple transmission images obtained by detecting an item to be inspected by multiple detectors;

[0008] For each transmission image, determine the thickness corresponding to each pixel point in the transmission image according to the gray value of each pixel point in the transmission image and a first correspondence relationship established in advance for the transmission image to represent the relationship between the gray value and the thickness, where the first correspondence relationship is determined in advance according to the detection performance of the detector that detects the transmission image;

[0009] For each fusion pixel point in the fusion image to be generated, determine the target thickness corresponding to the fusion pixel point based on the thickness corresponding to the basic pixel point, where the basic pixel point is the pixel point with the same image position as the fusion pixel point in the multiple transmission images;

[0010] For each of the fused pixel points, according to the target thickness corresponding to the fused pixel point and a preset second correspondence between thickness and gray value, determine the target gray value corresponding to the fused pixel point, and obtain the fused image.

[0011] In a second aspect, an embodiment of the present application provides an image fusion device, the device includes:

[0012] An image acquisition module, configured to acquire a plurality of transmission images obtained by a plurality of detectors detecting an object to be inspected;

[0013] A first thickness determination module, configured to, for each transmission image, according to the gray value of each pixel point in the transmission image and a preset first correspondence between the transmission image and the thickness used to characterize the relationship between the gray value and the thickness, determine the thickness corresponding to each pixel point in the transmission image, where the first correspondence is determined in advance according to the detection performance of the detector that detects the transmission image;

[0014] A second thickness determination module, configured to, for each fused pixel point in the to-be-generated fused image, based on the thickness corresponding to the base pixel point, determine the target thickness corresponding to the fused pixel point, where the base pixel point is the pixel point with the same image position as the fused pixel point in the plurality of transmission images;

[0015] A gray value determination module, configured to, for each of the fused pixel points, according to the target thickness corresponding to the fused pixel point and a preset second correspondence between thickness and gray value, determine the target gray value corresponding to the fused pixel point, and obtain the fused image.

[0016] In a third aspect, an embodiment of the present application provides an image fusion system, the system includes an X-ray security inspection machine and an image processing device; the X-ray security inspection machine includes an X-ray source and a detector;

[0017] The X-ray source is configured to emit X-rays;

[0018] The detector is configured to detect an object to be inspected and obtain a transmission image;

[0019] The image processing device is used to obtain a plurality of transmission images obtained by a plurality of detectors detecting the object to be detected; for each transmission image, according to the gray value of each pixel point in the transmission image and a pre-established first correspondence relationship corresponding to the transmission image for characterizing the relationship between the gray value and the thickness, determine the thickness corresponding to each pixel point in the transmission image, where the first correspondence relationship is determined in advance according to the detection performance of the detector that detects the transmission image; for each fusion pixel point in the fusion image to be generated, based on the thickness corresponding to the base pixel point, determine the target thickness corresponding to the fusion pixel point, where the base pixel point is the pixel point with the same image position as the fusion pixel point in the plurality of transmission images; for each fusion pixel point, according to the target thickness corresponding to the fusion pixel point and a preset second correspondence relationship between the thickness and the gray value, determine the target gray value corresponding to the fusion pixel point, and obtain the fusion image.

[0020] In a fourth aspect, an embodiment of the present application provides an electronic device, including:

[0021] A memory for storing a computer program;

[0022] A processor, when executing the program stored in the memory, implements the method steps described in the first aspect above.

[0023] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, it implements the method steps described in the first aspect above.

[0024] Advantageous effects of the embodiments of the present application:

[0025] In the solution provided by the embodiments of the present application, the electronic device can obtain a plurality of transmission images obtained by a plurality of detectors detecting the object to be detected. For each transmission image, according to the gray value of each pixel point in the transmission image and a pre-established first correspondence relationship corresponding to the transmission image for characterizing the relationship between the gray value and the thickness, determine the thickness corresponding to each pixel point in the transmission image, where the first correspondence relationship is determined in advance according to the detection performance of the detector that detects the transmission image; for each fusion pixel point in the fusion image to be generated, based on the thickness corresponding to the base pixel point with the same image position as the fusion pixel point in the plurality of transmission images, determine the target thickness corresponding to the fusion pixel point. Furthermore, for each fusion pixel point, according to the target thickness corresponding to the fusion pixel point and a preset second correspondence relationship between the thickness and the gray value, determine the target gray value corresponding to the fusion pixel point, and obtain the fusion image.

[0026] Since the detection performances of different detectors for different thicknesses are different, the accuracy of the thickness of the object under inspection represented by the gray value of each pixel in the transmission image of each detector is also different. During the image fusion process, based on the first correspondence and the gray value of each pixel, the thickness corresponding to the position of this pixel is determined. Then, using the preset second correspondence, the thickness is mapped back to the gray space. The pixels at the same position in each transmission image can be flexibly fused according to the determined thickness and the detection performances of each detector. Compared with the fusion method of pre-assigning fixed weights to multiple transmission images, the image quality of the obtained fused image is better, and the internal detail information of different thickness regions of the object under inspection can be clearly displayed. Moreover, by flexibly customizing the second correspondence, different thicknesses can be mapped to the gray values expected to be displayed. In this way, without performing additional image processing, the distribution of regions with different thicknesses of the object under inspection can be relatively intuitively displayed in the fused image.

[0027] Of course, it is not necessary for any product or method implementing this application to simultaneously achieve all the above-mentioned advantages. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following-described drawings are only some embodiments of this application, and those of ordinary skill in the art can also obtain other embodiments based on these drawings.

[0029] Figure 1 It is a flowchart of an image fusion method provided by an embodiment of this application;

[0030] Fig. 2(a) is a schematic structural diagram of a dual-energy security inspection machine provided by an embodiment of this application;

[0031] Fig. 2(b) is a schematic diagram of an arrangement manner of multiple detectors provided by an embodiment of this application;

[0032] Figure 3 It is a curve graph showing the change of the signal-to-noise ratio of the transmission images detected by the low-energy detector and the high-energy detector provided by an embodiment of this application with the penetration thickness;

[0033] Fig. 4(a) is a schematic diagram of a fused image obtained by using the image fusion method provided by an embodiment of this application;

[0034] Fig. 4(b) is a schematic diagram of a corrected image obtained by using ordinary linear correction;

[0035] Figure 5 For Figure 1A specific flowchart of step S104 in the illustrated embodiment;

[0036] FIG. 6(a) is a schematic diagram of a fused image obtained using the second correspondence;

[0037] FIG. 6(b) is another schematic diagram of a fused image obtained using the second correspondence;

[0038] Figure 7 For Figure 1 A specific flowchart of step S103 in the illustrated embodiment;

[0039] Figure 8 Based on Figure 1 A flowchart of a method for establishing the first correspondence in the illustrated embodiment;

[0040] Figure 9 Based on Figure 1 A schematic diagram of the first correspondence corresponding to the low-energy detector and the high-energy detector in the illustrated embodiment;

[0041] Figure 10 Based on Figure 1 A flowchart schematic diagram of an image fusion method in the illustrated embodiment;

[0042] FIG. 11(a) is Figure 1 A schematic diagram of a transmission image in the illustrated embodiment;

[0043] FIG. 11(b) is Figure 1 Another schematic diagram of a transmission image in the illustrated embodiment;

[0044] FIG. 11(c) is a schematic diagram of a fused image of the transmission images shown in FIGS. 11(a) and 11(b);

[0045] Figure 12 A schematic diagram of the structure of an image fusion device provided by an embodiment of the present application;

[0046] Figure 13 A schematic diagram of the structure of an image fusion system provided by an embodiment of the present application;

[0047] Figure 14 A schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0048] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art based on the present application belong to the scope of protection of the present application.

[0049] In order to improve the image quality of the fused image and clearly display the detailed information inside the inspected objects with various different thicknesses, the embodiments of the present application provide an image fusion method, device, electronic device, computer-readable storage medium, and computer program product. First, an image fusion method provided by the embodiments of the present application will be introduced below.

[0050] The image fusion method provided by the embodiments of the present application can be applied to any electronic device capable of performing image fusion. For example, it can be an image processing device in an image fusion system, a processor in an X-ray security inspection machine, a server for performing image fusion, etc., which are not specifically limited herein. For the sake of clear description, it will be referred to as an electronic device hereinafter.

[0051] As Figure 1 shown, an image fusion method, the method includes:

[0052] S101: Obtain a plurality of transmission images obtained by a plurality of detectors detecting an inspected object.

[0053] The image fusion method provided by the embodiments of the present application can be applied to X-ray detection devices equipped with a plurality of detectors, including but not limited to security inspection machines, industrial detection devices, etc. Further, for example, dual-energy security inspection machines or dual-energy industrial detection devices (such as defect detection devices or food foreign object detection devices) equipped with dual-energy detectors, etc.

[0054] S102: For each transmission image, determine the thickness corresponding to each pixel point in the transmission image according to the gray value of each pixel point in the transmission image and the pre-established first correspondence relationship between the gray value and the thickness corresponding to the transmission image.

[0055] Among them, the first correspondence relationship is determined in advance according to the detection performance of the detector that detects the transmission image.

[0056] S103: For each fusion pixel point in the to-be-generated fused image, determine the target thickness corresponding to the fusion pixel point based on the thickness corresponding to the base pixel point.

[0057] Among them, the base pixel point is the pixel point with the same image position as the fusion pixel point in the plurality of transmission images.

[0058] S104: For each of the fused pixel points, determine the target gray value corresponding to the fused pixel point according to the target thickness corresponding to the fused pixel point and a preset second correspondence between thickness and gray value, to obtain the fused image.

[0059] It can be seen that in the solution provided by the embodiments of the present application, an electronic device can obtain multiple transmission images obtained by multiple detectors detecting a detected object. For each transmission image, according to the gray value of each pixel point in the transmission image and a preset first correspondence between the gray value and the thickness corresponding to the transmission image, determine the thickness corresponding to each pixel point in the transmission image, where the first correspondence is determined in advance according to the detection performance of the detector that detects the transmission image; for each fused pixel point in the to-be-generated fused image, the target thickness corresponding to the fused pixel point can be determined based on the thicknesses corresponding to the base pixel points with the same image position as the fused pixel point in multiple transmission images. Furthermore, for each fused pixel point, determine the target gray value corresponding to the fused pixel point according to the target thickness corresponding to the fused pixel point and a preset second correspondence between thickness and gray value, to obtain the fused image.

[0060] Since the detection performances of different detectors for different thicknesses are different, the accuracy of the thickness of the detected object represented by the gray value of each pixel point in the transmission image of each detector is also different. During the image fusion process, based on the first correspondence and the gray value of each pixel point, determine the thickness corresponding to the position of the pixel point, and then use the preset second correspondence to map the thickness back to the gray space. The pixel points with the same position in each transmission image can be flexibly fused according to the determined thickness and the detection performance of each detector. Compared with the fusion method of pre-assigning fixed weights to multiple transmission images, the image quality of the obtained fused image is better, and the detailed information inside different thickness regions of the detected object can be clearly displayed. Moreover, by flexibly customizing the second correspondence, different thicknesses can be mapped to the desired displayed gray values. In this way, without performing additional image processing, the distribution of regions with different thicknesses of the detected object can be relatively intuitively displayed in the fused image.

[0061] When using an X-ray source and a detector to detect a detected object, the X-ray emitted by the X-ray source passes through the detected object and is incident on the detector, and the detector can detect the X-ray to obtain a transmission image of the detected object. Then, the electronic device uses the transmission image to calculate the equivalent atomic number of the detected object to obtain the detection result of the detected object.

[0062] To improve the accuracy of the detection result, an X-ray source and multiple detectors can be used to detect the detected object, and the transmission images detected by the multiple detectors are fused, and the detection result of the detected object is calculated using the fused image.

[0063] In currently commonly used transmission image fusion methods, fixed weights are pre-assigned to each transmission image. However, due to the fact that the stability of each detector may change under the influence of various factors. For example, under the influence of various reasons such as X-ray hardening effect, non-uniform X-ray intensity, and inconsistency of device response, there is inconsistency in the image quality of the transmission images detected by each detector. In this way, the image quality of the fused image obtained by using fixed weights for image fusion is not high, it is difficult to see the details of each thickness region of the item, and further leads to low accuracy of the detection result.

[0064] Taking the dual-energy security inspection machine shown in Fig. 2(a) as an example, the dual-energy security inspection machine includes an X-ray source and a dual-energy detection device. The dual-energy detection device is composed of a low-energy detector, a copper sheet, and a high-energy detector. The over-pack direction is the direction of the X-axis in the world coordinate system (XYZ coordinate system). When using the dual-energy X-ray detector to detect the object to be inspected, the object to be inspected is conveyed into the detection area along the over-pack direction. The X-rays emitted by the X-ray source pass through the object to be inspected and first enter the low-energy detector. The copper sheet located between the low-energy detector and the high-energy detector will block a part of the low-energy X-rays that penetrate the low-energy detector. Among them, the process in which the above copper sheet blocks part of the low-energy X-rays and reduces the low-energy X-rays in the X-ray beam that penetrates the copper sheet is the process of X-ray hardening. The hardened X-rays continue to enter the high-energy detector. In this way, the low-energy detector and the high-energy detector can detect different X-rays and respectively obtain the transmission image of the object to be inspected based on the detected X-rays. Then, fixed weights are respectively assigned to the transmission image detected by the low-energy detector and the transmission image detected by the high-energy detector, and the two transmission images are fused using the fixed weights.

[0065] However, the detection performances of the low-energy detector and the high-energy detector are different, and moreover, the stabilities of the low-energy detector and the high-energy detector are different. Furthermore, the detection capabilities of the low-energy detector and the high-energy detector for different thickness regions of the object to be inspected are different, and the accuracies of the thicknesses of the object to be inspected characterized by the gray values of each pixel point in the obtained transmission images are also different. For the same region, the stronger the detection capability of the detector, the higher the accuracy of the gray values of each pixel point in the corresponding image region of this region in the obtained transmission image.

[0066] Low-energy detectors have a stronger detection ability for thin and easily penetrable objects to be inspected. That is, when detecting thin and easily penetrable objects to be inspected, the transmission images obtained by low-energy detectors are clearer, and the accuracy of the gray values of the pixel points in the transmission images is higher; while high-energy detectors have a stronger detection ability for thick and difficult-to-penetrate objects to be inspected. That is, when detecting thick and difficult-to-penetrate objects to be inspected, the transmission images obtained by high-energy detectors are clearer, and the accuracy of the gray values of the pixel points in the transmission images is higher.

[0067] When the object to be inspected includes multiple regions with different thicknesses, the electronic device uses fixed weights to perform image fusion on the two transmission images. The clarity of each image region in the obtained fusion image is different. For example, the object to be inspected includes 4 regions, namely region A, region B, region C, and region D. Among them, the thicknesses of region A and region B are 5 mm, and the thicknesses of region C and region D are 30 mm. Set the weight of the transmission image detected by the low-energy detector to 0.6, and the weight of the transmission image detected by the high-energy detector to 0.4. In the fusion image obtained using the above fusion weights, the image quality of the image regions corresponding to region A and region B is higher, while the image quality of the image regions corresponding to region C and region D is lower, and the overall image quality of the fusion image is lower, making it difficult to see the object details in region C and region D of the object to be inspected.

[0068] In the above step S101, the electronic device obtains multiple transmission images obtained by multiple detectors detecting the object to be inspected.

[0069] In order to improve the image quality of the fusion image and the accuracy of the detection result, the object to be inspected is detected using an X-ray source and multiple detectors. Each detector can separately receive the X-rays penetrating the object to be inspected to obtain a transmission image. In this way, the electronic device can obtain the transmission images obtained by multiple detectors and calculate the fusion image of the multiple transmission images. Among them, the number and arrangement method of the above multiple detectors can be set as needed. For example, the number of detectors is 2, 5, etc.; the arrangement method of the detectors is arranged vertically aligned, arranged side by side left and right, etc.; this is not limited here.

[0070] As an implementation method, the multiple detectors can be different types of detectors arranged vertically aligned. For example, in the dual-energy detection device shown in Fig. 2(a), the low-energy detector and the high-energy detector arranged vertically. When the object to be inspected is conveyed into the detection area along the over-package direction, the low-energy detector and the high-energy detector simultaneously detect the transmission image of the object to be inspected.

[0071] In one embodiment, the step of obtaining multiple transmission images obtained by multiple detectors detecting a subject may include: obtaining a high-energy transmission image obtained by a high-energy detector in a dual-energy detection device detecting the subject, and a low-energy transmission image obtained by a low-energy detector in the dual-energy detection device detecting the subject.

[0072] When using a dual-energy detection device to detect a subject, an electronic device may obtain a high-energy transmission image obtained by a high-energy detector in the dual-energy detection device detecting the subject, and obtain a low-energy transmission image obtained by a low-energy detector in the dual-energy detection device detecting the subject, and then perform image fusion on the high-energy transmission image and the low-energy transmission image.

[0073] As one embodiment, as shown in Fig. 2(b), the multiple detectors are low-energy detectors and high-energy detectors in two side-by-side dual-energy detection devices. When the subject is conveyed into the detection area along the over-pack direction, the low-energy detectors and high-energy detectors in the two dual-energy detection devices respectively detect the transmission images of the subject.

[0074] As one embodiment, the multiple detectors may be five detectors of the same type arranged side by side.

[0075] As one embodiment, an electronic device may obtain multiple initial transmission images obtained by multiple detectors detecting a subject, and perform image alignment, image cropping, and other processing on the multiple initial transmission images to obtain multiple transmission images of the subject.

[0076] In one case, when the multiple detectors are detectors of the same type arranged side by side, the subject is conveyed along the over-pack direction to the detection area where the multiple detectors are deployed. Each detector in the detection area may detect a partial area of the subject, so that the initial transmission images detected by each detector are images of partial areas of the subject. For example, the initial transmission image e may include the area E-H of the subject, the initial transmission image f may include the area E-G of the subject, the initial transmission image g may include the area G-H of the subject, and the initial transmission image h may include the area H of the subject.

[0077] Based on this, the electronic device can obtain multiple initial images detected by each detector for the object to be inspected, and can perform image processing on the multiple transmission images to obtain multiple transmission images for detecting each region of the object to be inspected. For example, by cropping the initial transmission image e and the initial transmission image f, two transmission images of the region E-F of the object to be inspected are obtained; by cropping the initial transmission image e, the initial transmission image f, and the initial transmission image g, three transmission images of the region G of the object to be inspected are obtained; by cropping the initial transmission image e and the initial transmission image h, two transmission images of the region H of the object to be inspected are obtained.

[0078] Furthermore, in the above step S102, for each transmission image, the electronic device can determine the thickness corresponding to each pixel point in the transmission image according to the gray value of each pixel point in the transmission image and the pre-established first correspondence relationship between the gray value and the thickness corresponding to the transmission image.

[0079] Since the detection performances of different detectors can be different, the detection capabilities of each detector for objects of different thicknesses can be different. Furthermore, when different detectors detect the same object to be inspected, the gray values of the pixel points at the same image position in the obtained transmission images can be different, and the jitter conditions of the gray values of the pixel points at the same image position in the obtained transmission images can also be different. That is to say, for the same penetration thickness region of the object to be inspected, in the respective transmission images detected by different detectors, the gray values of the pixel points in the image region corresponding to this region can be different, and, at the same thickness step, in the transmission images detected by different detectors, the change amount of the gray values of the pixel points is also different.

[0080] Based on this, when performing image fusion on the transmission images detected by multiple detectors, in order to use the transmission images detected by each detector to fuse into a clearer fused image, for each detector, a first correspondence relationship for characterizing the correspondence relationship between the gray value and the thickness can be pre-established according to the detection performance of the detector.

[0081] Among them, for each detector, the first correspondence relationship corresponding to the detector can characterize the credibility of the detector data detected by the detector at this thickness. The first correspondence relationship can include the correspondence relationship between the gray value, the standard deviation of the gray value and the thickness, or can also include the correspondence relationship between the gray value, the credibility of the gray value and the thickness, and no specific limitation is made thereto. For the sake of clear writing, the establishment method of the first correspondence relationship of the detector will be described in detail below.

[0082] For each detector, the first correspondence relationship corresponding to the detector is used to represent as a gray value-thickness function, or can also be represented as a gray thickness scale table, etc., and no specific limitation is made here.

[0083] In this way, after the electronic device obtains multiple transmission images, for each transmission image, first, it determines the first correspondence relationship between the gray value and the thickness corresponding to the detector that detects the transmission image as the first correspondence relationship corresponding to the transmission image. Then, according to the gray value of each pixel point in the transmission image and the first correspondence relationship corresponding to the transmission image, it determines the thickness corresponding to each pixel point in the transmission image.

[0084] Taking the dual-energy security inspection machine as an example, since the detection capabilities of the low-energy detector and the high-energy detector for different transmission thicknesses are different, the image quality of the transmission images detected by the low-energy detector and the high-energy detector is different. For each transmission image, the signal-to-noise ratio of the transmission image can characterize the image quality of the transmission image. Specifically, the smaller the signal-to-noise ratio, the higher the image quality of the transmission image.

[0085] The curves of the signal-to-noise ratios of the transmission images detected by the low-energy detector and the high-energy detector changing with the penetration thickness are as Figure 3 shown. At low penetration thicknesses, the signal-to-noise ratio of the transmission image detected by the low-energy detector is smaller. Therefore, the deviation between the penetration thickness detected by the low-energy detector and the true thickness of the object to be inspected is smaller, and the detection result of the transmission image is more accurate. As the transmission thickness increases, the signal-to-noise ratio of the transmission image detected by the high-energy detector is smaller, the deviation between the detected penetration thickness and the true thickness of the object to be inspected is smaller, and the detection result of the transmission image is more accurate.

[0086] For each detector, the first correspondence relationship between the gray value and the thickness corresponding to the detector can be established in advance according to the detection performance of the detector to determine the change in the detection performance of the detector for objects of different thicknesses.

[0087] Taking the dual-energy security inspection machine as an example, according to the detection performance of the low-energy detector, the first correspondence relationship between the gray value and the thickness of the low-energy detector is established; according to the detection performance of the high-energy detector, the first correspondence relationship between the gray value and the thickness of the high-energy detector is established. In this way, when the electronic device obtains the transmission image detected by the low-energy detector for the object to be inspected, it can determine the thickness corresponding to each pixel point in the transmission image according to the gray value of each pixel point in the transmission image and the first correspondence relationship between the gray value and the thickness of the low-energy detector established in advance. Correspondingly, when the electronic device obtains the transmission image detected by the high-energy detector for the object to be inspected, it can determine the thickness corresponding to each pixel point in the transmission image according to the gray value of each pixel point in the transmission image and the first correspondence relationship between the gray value and the thickness of the high-energy detector established in advance.

[0088] When performing image fusion using each transmission image, the target image size of the fusion image to be generated is the same as the image sizes of the multiple transmission images. Moreover, for each fusion pixel in the fusion image, it is obtained by fusing the base pixels at the same image position in each of the transmission images that generate the fusion image.

[0089] Based on this, in step S103 above, for each fusion pixel in the fusion image to be generated, the electronic device can determine the target thickness corresponding to this fusion pixel based on the thickness corresponding to the base pixel.

[0090] For each fusion pixel in the fusion image, determine the base pixel in each transmission image that has the same image position as this fusion pixel. Among them, the base pixel with the same image position as the fusion pixel is: in the image coordinate system, the pixel with the same image coordinates as this fusion pixel.

[0091] After determining the thickness corresponding to each pixel in each transmission image, for each fusion pixel in the fusion image to be generated, the electronic device can determine the base pixels in the multiple transmission images that have the same image position as this fusion pixel, and determine the target thickness corresponding to this fusion pixel based on the thicknesses corresponding to each of the base pixels.

[0092] As an implementation manner, the electronic device can calculate the fusion weights of each base pixel according to the gray values of each base pixel, and calculate the weighted sum value of the thicknesses corresponding to each base pixel to obtain the target thickness corresponding to this fusion pixel.

[0093] As an implementation manner, the electronic device can calculate the average value of the thicknesses of multiple base pixels and use this average value as the target thickness corresponding to this fusion pixel.

[0094] As an implementation manner, the electronic device can take the median of the thicknesses of multiple base pixels and use this median as the target thickness corresponding to this fusion pixel.

[0095] Furthermore, in step S104 above, for each fusion pixel, the electronic device can determine the target gray value corresponding to this fusion pixel according to the target thickness corresponding to this fusion pixel and the second correspondence between thickness and gray value preset, to obtain the fusion image.

[0096] To visually display the thickness variation of the object to be inspected, a second correspondence between thickness and gray value can be established in advance according to the gray display requirement of the fused image. For example, to distinguish regions of different thicknesses of the object to be inspected, a larger gray value can be set corresponding to a larger thickness, and as the thickness decreases, the corresponding gray value can gradually decrease. In this way, in the fused image determined using this second correspondence, the gray value differences of the image regions representing different thickness regions of the object to be inspected are relatively large, and the thickness differences of different regions of the object to be inspected can be visually displayed.

[0097] Based on this, the electronic device can obtain the second correspondence between thickness and gray value established in advance according to the gray display requirement of the fused image. For example, it can receive the second correspondence between thickness and gray value carried in the image fusion instruction sent by the user. When determining the target thickness corresponding to each fused pixel point in the fused image to be generated, the electronic device can determine the target gray value corresponding to this fused pixel point according to the target thickness corresponding to this fused pixel point and the preset second correspondence between thickness and gray value. After determining the target gray value of each fused pixel point in the fused image to be generated, the fused image can be obtained.

[0098] Since the detection performance of each detector and the credibility of the gray values of the pixel points in the transmission images detected by each detector under thickness variation are fully considered, this image fusion method can avoid the influence of the image quality of each transmission image caused by reasons such as X-ray hardening effect, non-uniform X-ray intensity, and inconsistent detector device responses on the image quality of the fused image when transmitting low-penetration objects. Moreover, during the image fusion process, the gray value differences of the transmission images obtained by each detector detecting low-penetration objects can be corrected. As shown in Figure 4(a), the fused image obtained using the image fusion method provided in this application can eliminate the stripes in the image and improve the image display effect compared with the corrected image obtained by correcting the gray values of the transmission images using the common linear correction method as shown in Figure 4(b).

[0099] As an implementation manner, after obtaining the fused image of the object to be inspected, the electronic device can output the fused image of the object to be inspected. For example, the electronic device can display the fused image of the object to be inspected on its own display screen. For another example, the electronic device can send the fused image of the object to be inspected to other electronic devices. In this way, the staff can view the fused image of the object to be inspected output by the electronic device and view the detailed information inside the object to be inspected through this fused image to obtain the detection result of the object to be inspected.

[0100] As an implementation manner of an embodiment of this application, the second correspondence includes the correspondence between the thickness and the gray value corresponding to different target regions of the preset object to be inspected;

[0101] As Figure 5 shown, the step S104 of determining the target gray value corresponding to each of the fusion pixel points according to the target thickness corresponding to the fusion pixel point and the second correspondence between the thickness and the gray value preset may include:

[0102] S501: For each of the fusion pixel points, determine the target area corresponding to the fusion pixel point.

[0103] S502: According to the target thickness corresponding to the fusion pixel point and the second correspondence corresponding to the preset target area, determine the target gray value corresponding to the fusion pixel point.

[0104] In order to distinguish each area of the object to be inspected when displaying the fusion image of the object to be inspected, the object to be inspected may be divided into multiple target areas. Further, for each target area of the object to be inspected, according to the gray display requirements of different target areas, the correspondence between the thickness and the gray value corresponding to the target area may be established in advance. In this way, the above-mentioned second correspondence between the thickness and the gray value may include the correspondence between the thickness and the gray value corresponding to different target areas of the object to be inspected preset.

[0105] Among them, the above-mentioned second correspondence may be represented by a thickness-gray value mapping curve or a thickness-gray value scale table, which is not limited herein.

[0106] For each fusion pixel point in the fusion image to be generated, after obtaining the target thickness of the fusion pixel point, the electronic device may first determine the target area corresponding to the fusion pixel point, and then determine the second correspondence between the thickness and the gray value corresponding to the target area, and determine the target gray value corresponding to the fusion pixel point according to the fusion pixel point and the second correspondence between the thickness and the gray value corresponding to the preset target area.

[0107] Exemplarily, in order to test the penetration ability, spatial resolution, line penetration resolution, and line resolution of the detector for low-penetration regions, the detector can be used to detect lead cakes, wire groups, and curves respectively. If the transmission image is generated using the same preset second correspondence between thickness and gray value, the obtained transmission image is shown in Fig. 6(a). Since the lead cake (TEST 4) has a large thickness, the gray value of its transmission image is high, making it difficult to clearly and intuitively distinguish the details of the lead cake. Since the wire group (TEST 3) and the curve (TEST 1, TEST 2) have small thicknesses, the gray values of their transmission images are low, making it difficult to clearly and intuitively distinguish the line details. In order to clearly and intuitively display the details of the lead cake, wire group, and curve, a second correspondence between thickness and gray value can be constructed for the lead cake, wire group, and curve respectively, and the transmission image is generated using each second correspondence. The obtained transmission image is shown in Fig. 6(b). The transmission image (TEST 4) is clearer and the image is more delicate, and the lead cake can be intuitively displayed. The transmission image (TEST 3) can clearly and intuitively display the wire group. The lines in the transmission images (TEST 1, TEST 2) are also clearer, and the line details of the curve can be intuitively displayed.

[0108] It can be seen that in the embodiment of the present application, according to the gray display requirements of different target regions, the second correspondence between thickness and gray value can be flexibly set for different regions of the object to be inspected. In this way, for the fusion image obtained according to each second correspondence, the detail information of each region with different thicknesses of the object to be inspected can be intuitively displayed without additional image processing.

[0109] As an implementation manner of the embodiment of the present application, as Figure 7 shown, in the above S103, for each fusion pixel point in the fusion image to be generated, the step of determining the target thickness corresponding to the fusion pixel point based on the thickness corresponding to the basic pixel point may include:

[0110] S701, for each fusion pixel point in the fusion image to be generated, calculate the fusion weight of each basic pixel point according to the gray value of the basic pixel point.

[0111] Since the detection performances of different detectors are different, in the transmission images obtained by each detector detecting the same object to be inspected, the gray values of the pixel points corresponding to the same position of the object to be inspected are different. Furthermore, the accuracy of the thickness of the object to be inspected represented by the gray values of each pixel value is also different. When performing image fusion, for each fusion pixel point in the fusion image to be generated, the gray values of the basic pixel points in the multiple transmission images used for image fusion are different, and the credibility of the thickness corresponding to each basic pixel point is also different.

[0112] In order to improve the image quality of the fused image and accurately calculate the target thickness corresponding to the fused pixel points, it is necessary to make full use of the gray-scale information with a relatively high degree of credibility in each transmission image. Therefore, the fusion weight corresponding to each transmission image can be determined according to the credibility of the gray-scale values of each pixel point in each transmission image. By using the obtained fusion weights to calculate the target thickness corresponding to the fused pixel points, the influence of the transmission image with a relatively low credibility on the image quality of the fused image can be reduced.

[0113] Based on this, for each fused pixel point in the to-be-generated fused image, the electronic device first determines the basic pixel points in each transmission image that have the same image position as this fused pixel point, and then calculates the fusion weights of each basic pixel point based on the gray-scale values of each basic pixel point.

[0114] As an implementation manner, when the first corresponding relationship of each detector is the corresponding relationship between the gray-scale value, the gray-scale value probability distribution parameter, and the thickness, for each fused pixel point in the to-be-generated fused image, according to the first corresponding relationship corresponding to each transmission image, calculate the probabilities corresponding to the gray-scale values of each basic pixel point, and according to the preset corresponding relationship between the probability and the fusion weight, calculate the fusion weights of each basic pixel point. Since the higher the probability corresponding to the gray-scale value, the higher the credibility of this gray-scale value, thus, the probability and the fusion weight can be in a positive correlation relationship.

[0115] For example, for each fused pixel point R in the to-be-generated fused image, according to the first corresponding relationship corresponding to each transmission image, calculate the probabilities corresponding to the gray-scale values of the basic pixel point R1, the basic pixel point R2, and the basic pixel point R3, and obtain that the probability corresponding to the gray-scale value of the basic pixel point R1 is 0.9, the probability corresponding to the gray-scale value of the basic pixel point R2 is 0.8, and the probability corresponding to the gray-scale value of the basic pixel point R3 is 0.3. According to the preset corresponding relationship between the probability and the weight, calculate that the fusion weight corresponding to the basic pixel point R1 is 0.45, the fusion weight corresponding to the basic pixel point R2 is 0.4, and the fusion weight corresponding to the basic pixel point R3 is 0.15.

[0116] S702, for each of the fused pixel points, calculate the weighted sum value of the thicknesses corresponding to the basic pixel points based on the fusion weights corresponding to the basic pixel points, to obtain the target thickness corresponding to this fused pixel point.

[0117] For each fused pixel point, the electronic device can calculate the weighted sum value of the thicknesses corresponding to each basic pixel point based on the fusion weights of each basic pixel point and the thicknesses corresponding to each basic pixel point, to obtain the target thickness corresponding to this fused pixel point.

[0118] For each fused pixel point, the target thickness of this fused pixel point can be expressed as:

[0119] x = Σk i x i

[0120] Wherein, x is the target thickness, and x i is the thickness corresponding to the i-th basic pixel point, and k i is the thickness corresponding to the i-th basic pixel point, 0 < i < n, and n is the number of basic pixel points.

[0121] Exemplarily, for the fused pixel point M, the fused pixel point M is obtained by fusing the basic pixel point P and the basic pixel point Q. The thickness corresponding to the basic pixel point P is 10 mm, the fusion weight of the basic pixel point P is 0.4, and the thickness corresponding to the basic pixel point Q is 16 mm, and the fusion weight of the basic pixel point Q is 0.6. The electronic device calculates the weighted sum value of the thicknesses corresponding to each basic pixel point, 10 * 0.4 + 16 * 0.6 = 13.6, that is, the target thickness corresponding to the fused pixel point M is 13.6 mm.

[0122] It can be seen that in the embodiment of the present application, image fusion is performed according to the fusion weights corresponding to each basic pixel point, and the relatively accurate detection information in each detector can be maximally utilized. In this way, the obtained target thickness is relatively accurate, and further, the gray value of the fused pixel point calculated using the target thickness has high accuracy.

[0123] As an implementation manner of the embodiment of the present application, as Figure 8 shown, the establishment method of the first correspondence relationship may include:

[0124] S801: For each detector, obtain the sample transmission images obtained by the detector detecting samples of different thicknesses.

[0125] S802: For each sample transmission image, calculate the gray value information of the pixel points in the sample transmission image.

[0126] S803: For each detector, establish a first correspondence relationship corresponding to the detector for characterizing the correspondence relationship between the gray value and the thickness according to the gray value information of each sample transmission image corresponding to the detector and the sample thickness.

[0127] In order to perform image fusion on the multiple transmission images detected by multiple detectors to obtain a fused image with high image quality, for each detector, a first correspondence relationship for characterizing the correspondence relationship between the gray value and the thickness can be established in advance according to the detection performance of the detector.

[0128] Specifically, for each detector, the detector can be used to detect multiple samples with different thicknesses respectively, and obtain the sample transmission images of each sample. In this way, for each detector, the electronic device can acquire multiple sample transmission images of the detector and the sample thickness corresponding to each sample transmission image, and for each sample transmission image, calculate the gray value information of the pixel points in the sample transmission image to obtain the corresponding relationship between each sample thickness and the gray value information corresponding to the sample thickness.

[0129] For each detector, the detection performance of the detector determines the credibility of the gray values detected at different thicknesses. For example, when the stability of the detector is poor, the credibility of the gray values of the pixel points in the transmission image detected by the detector is relatively low.

[0130] For each detector, the gray value information can characterize the credibility and accuracy of the gray values. Then, the first corresponding relationship can reflect the detection performance of the detector and the accuracy of the gray values detected by the detector at different thicknesses.

[0131] Among them, the gray value information can include the gray value mean and the gray value standard deviation, can also include the gray value mean and the probability corresponding to the gray value mean, can also include the gray value mean, and the deviation of each gray value relative to the gray value mean, which is not specifically limited here.

[0132] When the gray value information of each detector is different, the first corresponding relationships established for each detector are also different.

[0133] When the gray value information includes the gray value mean and the gray value standard deviation, the first corresponding relationship includes the corresponding relationship between the gray value mean, the gray value standard deviation, and the thickness. For a detector, the first corresponding relationship can be described in three dimensions of (x, μ, σ), that is, for a certain thickness x of the object to be inspected, the gray value corresponding to the detector is the sampling value under the Gaussian probability distribution, that is, the gray value mean corresponding to the detector is μ, and its gray value standard deviation is σ.

[0134] For each detector, the electronic device can establish a first corresponding relationship corresponding to the detector for characterizing the corresponding relationship between the gray value and the thickness according to the gray value information and the sample thickness of each sample transmission image corresponding to the detector.

[0135] As an implementation manner, for each detector, the electronic device can obtain the sample transmission images obtained by the detector detecting samples with different thicknesses. Then, for each sample transmission image, the electronic device can calculate the mean gray value and the standard deviation of the gray values of the pixel points in the sample transmission image. Furthermore, for each detector, according to the mean gray value, the standard deviation of the gray values, and the sample thickness of each sample transmission image corresponding to the detector, a first correspondence relationship between the mean gray value, the standard deviation of the gray values, and the thickness corresponding to the detector is established.

[0136] Taking a dual-energy security inspection machine as an example, in order to calibrate the first correspondence relationship between the mean gray value, the standard deviation of the gray values, and the thickness corresponding to each of the low-energy detector and the high-energy detector, the low-energy detector can be used to detect multiple steel plates with different thicknesses respectively to obtain multiple transmission images.

[0137] Furthermore, for the low-energy detector, the electronic device can obtain multiple transmission images corresponding to the low-energy detector, and for the transmission image, calculate the mean gray value and the standard deviation of the gray values of the pixel points in the transmission image. Then, according to the mean gray value, the standard deviation of the gray values, and the steel plate thickness of each transmission image corresponding to the low-energy detector, a first correspondence relationship between the mean gray value, the standard deviation of the gray values, and the thickness corresponding to the low-energy detector is established, denoted as the performance curve H(x, μ1, σ1). For the high-energy detector, the electronic device can obtain multiple transmission images corresponding to the high-energy detector, and for the transmission image, calculate the mean gray value and the standard deviation of the gray values of the pixel points in the transmission image. Then, according to the mean gray value, the standard deviation of the gray values, and the steel plate thickness of each transmission image corresponding to the high-energy detector, a first correspondence relationship between the mean gray value, the standard deviation of the gray values, and the thickness corresponding to the high-energy detector is established, denoted as the performance curve L(x, μ2, v2). After that, use Figure 9 The gray value-thickness curves shown represent the first correspondence relationships corresponding to the low-energy detector and the high-energy detector respectively, as Figure 9 shown. The curves in the figure can show the changing trend of the gray value of the detector with the change of the thickness. And each gray value includes an error bar, and the error bar can characterize the standard deviation of the gray value. Among them, the error bar is an index indicating the degree of data dispersion, used to show the potential error of the data or characterize the uncertainty of the data.

[0138] It can be seen that in the embodiments of the present application, for each detector, the first correspondence relationship established in advance according to the multiple transmission images obtained by detecting samples of multiple different thicknesses by this detector, which is used to characterize the correspondence relationship between the gray value and the thickness corresponding to this detector, can characterize the change of the detection performance of this detector with the change of thickness, can also characterize the change of the stability of this detector with the change of thickness, and can also characterize the change of the credibility of the detector data detected by this detector with the change of thickness. Moreover, the above first correspondence relationship can reflect the image quality of the transmission images obtained by this detector under different gray value conditions, so that the electronic device can fuse each transmission image according to this first correspondence relationship, so that the fused image can achieve the optimal signal-to-noise ratio imaging, and thus, a fused image with the optimal image quality can be obtained.

[0139] As an implementation manner of the embodiments of the present application, the first correspondence relationship includes the correspondence relationship between the gray value, the standard deviation of the gray value and the thickness.

[0140] The above step S701, for each fused pixel point in the to-be-generated fused image, the step of calculating the fusion weight of each basic pixel point according to the gray value of the basic pixel point may include:

[0141] For each fused pixel point in the to-be-generated fused image, based on the expression of the preset coefficient corresponding to the basic pixel point corresponding to this fused pixel point determined in advance, the gray value of this basic pixel point, and the standard deviation of the gray value, calculate the preset coefficient corresponding to each basic pixel point as the fusion weight of this basic pixel point;

[0142] Wherein, the expression of the preset coefficient is the expression of the function of the variance of the gray value corresponding to this fused pixel point with respect to the preset coefficients corresponding to each basic pixel point established in advance based on the gray value and the standard deviation of the gray value of the basic pixel point when the variance of the gray value obtains the minimum value; or, the expression of the preset coefficient is the expression of the function of the probability of the gray value corresponding to this fused pixel point with respect to the preset coefficients corresponding to each basic pixel point established in advance based on the gray value and the standard deviation of the gray value of the basic pixel point when the probability of the gray value obtains the maximum value.

[0143] For each detector, the electronic device can include the correspondence relationship between the gray value, the standard deviation of the gray value and the thickness according to the first correspondence relationship established in advance for characterizing the correspondence relationship between the gray value and the thickness, that is, the stability of the gray value detected by this detector at different thicknesses can be characterized by using a normal distribution curve, and further, the image quality of the transmission images detected by this detector at different thicknesses can be characterized.

[0144] When performing image fusion, in order to facilitate calculating the fusion weights of multiple base pixels for generating each fused pixel, an expression for the preset coefficient corresponding to each base pixel can be determined in advance. In this way, for each base pixel, the electronic device can use this expression, the gray value of this base pixel, and the standard deviation of the gray value to calculate the preset coefficient corresponding to this base pixel as the fusion weight of this base pixel.

[0145] As an implementation manner of the embodiments of the present application, the expression is:

[0146]

[0147] where k i is the fusion weight of the i-th base pixel; σ i is the standard deviation of the gray value corresponding to the i-th base pixel, 0 < i < n, and n is the number of base pixels.

[0148] The above expression of the preset coefficient can be a function of the variance of the gray value corresponding to this fused pixel with respect to the preset coefficients corresponding to each base pixel established in advance based on the gray value and the standard deviation of the gray value of the base pixel, and is the expression when the variance of the gray value takes the minimum value.

[0149] Specifically, for each fused pixel, it can be assumed in advance that the fusion weights of the respective base pixels for generating this fused pixel are preset coefficients. Then, according to the preset coefficients and the standard deviations of the gray values of the respective base pixels, a function of the variance of the gray value corresponding to this fused pixel with respect to the preset coefficients corresponding to each base pixel is established.

[0150] Taking a dual-energy security inspection machine as an example, taking the transmission images detected by a high-energy detector and a low-energy detector respectively as an example, when performing image fusion on the two transmission images, for the fused pixel to be generated, it can be set that the preset coefficient of the base pixel (μ1, σ1) in the low-energy detector is k, then the preset coefficient of the base pixel (∑2, σ2) in the high-energy detector is 1 - k. Since linearly amplifying the gray value does not change its quality, the gray value of the fused pixel can be set to μ1, then (μ2, σ2) becomes When μ1 is 1 above, the gray value of the fused pixel is k*(1, σ1) + (1 - k)(1, σ2).

[0151] Calculating the gray value of this fused pixel, the expression can be obtained:

[0152]

[0153] Since the variance D can characterize the degree of deviation of the grayscale value from the mathematical expectation, thus, in order to provide the image quality of the fused image, it is desirable that the variance of the fused pixel takes the minimum value.

[0154] When the variance of the fused pixel takes the minimum value, the solution is obtained as:

[0155]

[0156] And, when the variance of the fused pixel takes the minimum value

[0157] Correspondingly, the derivation method can be used to derive the expression of the fusion weight of each basic pixel for the fused pixel in the to-be-generated detection image when fusing the transmission images of the object under inspection detected by multiple detectors. Furthermore, the expression of the fusion weight of each basic pixel is:

[0158] The above expression of the preset coefficient can also be a function of the grayscale value probability corresponding to the fused pixel with respect to the preset coefficients corresponding to each basic pixel, which is established based on the grayscale value and the standard deviation of the grayscale value of the basic pixel, and is the expression when the grayscale value probability takes the maximum value.

[0159] Specifically, when the mean value of the grayscale values satisfied by multiple basic pixels is μ x for each basic pixel, the grayscale value μ i of this basic pixel and the standard deviation σ i corresponding to this basic pixel can be used to calculate the probability of obtaining the grayscale value μ i .

[0160]

[0161]

[0162]

[0163] Among them, μ x is the mean value of the grayscale values corresponding to multiple basic pixels, and μ i is the grayscale value of the i-th basic pixel among multiple basic pixels.

[0164] Furthermore, according to the idea of finding parameters by maximum likelihood similarity, for each fused pixel, the electronic device can calculate the probability that each basic pixel of the fused pixel simultaneously obtains its corresponding grayscale value.

[0165]

[0166]

[0167] Calculate the expression for the grayscale value corresponding to the fused pixel when this probability reaches its maximum value:

[0168]

[0169] The preset coefficients of the grayscale values of each basic pixel are used to obtain the fusion weights corresponding to each basic pixel:

[0170]

[0171] As an implementation, when multiple detectors are of the same type arranged side by side, since the detection performances of detectors of the same type are relatively close, for each fused pixel, the fusion weights of the basic pixels in the transmission images detected by each detector can all be where n is the number of detectors.

[0172] It can be seen that in the embodiments of the present application, through the expression of the preset coefficients corresponding to the basic pixels corresponding to the fused pixel determined in advance, the expression, the grayscale values of multiple basic pixels, and the standard deviations of the grayscale values corresponding to multiple pixels can be used to calculate the fusion weight corresponding to each basic pixel. In this way, using the fusion weight of each basic pixel and the thickness of each basic pixel to calculate the target thickness of the fused pixel can improve the calculation efficiency, and the obtained target thickness has a high accuracy. Furthermore, the image quality of the fused image obtained using this target thickness is high.

[0173] As an implementation of the embodiments of the present application, the first correspondence relationship includes the correspondence relationship between the grayscale value, the standard deviation of the grayscale value, and the thickness.

[0174] The above step of calculating the weighted sum value of the thicknesses corresponding to each basic pixel based on the fusion weights corresponding to each basic pixel for each fused pixel to obtain the target thickness corresponding to this fused pixel includes:

[0175] Calculate the target thickness corresponding to this fused pixel according to the following formula:

[0176]

[0177] where x is the target thickness, and x i is the thickness corresponding to the i-th basic pixel.

[0178] For each detector, the electronic device can include the correspondence relationship among the gray value, the standard deviation of the gray value, and the thickness according to the pre-established first correspondence relationship for characterizing the relationship between the gray value and the thickness. Furthermore, the electronic device can calculate the target thickness corresponding to the fused pixel points by using the fusion weights of the respective basic pixel points obtained according to the above expressions of the preset coefficients for the respective basic pixel points and the thicknesses corresponding to the respective basic pixel points.

[0179] Specifically, the target thickness corresponding to each fused pixel point can be expressed as:

[0180]

[0181] In this way, in subsequent calculations, the calculation formula of the target thickness corresponding to the obtained fused pixel points can be used to calculate the target thickness corresponding to each fused pixel point, improving the calculation efficiency and thus improving the image fusion efficiency.

[0182] To facilitate the understanding of the image fusion method provided in the embodiments of the present application, taking a dual-energy security inspection machine as an example, the following is an explanation in combination with Figure 10 the image fusion process shown below:

[0183] S1001: Calibrate steel plates with different thicknesses.

[0184] S1002: Calculate the gray values and standard deviations at different thicknesses and make a gray value template table.

[0185] S1003: Set the correspondence relationship between the thickness and the gray value.

[0186] S1004: Collect the over-pack gray maps I1(i, j) and I2(i, j) of the two detectors.

[0187] S1005: Calculate and convert them into thickness information matrices x1(i, j) and x2(i, j) according to the gray value template table.

[0188] S1006: Obtain the synthesized thickness according to the formula

[0189] S1007: Obtain the fused gray map Y(i, j) = f(x(i, j)) according to the thickness-gray value correspondence function.

[0190] When using a dual-energy security inspection machine to detect an object to be inspected, a first correspondence relationship including the gray value, the standard deviation of the gray value, and the thickness corresponding to the low-energy detector and the high-energy detector can be pre-constructed. In order to construct the above first correspondence relationship, the staff can pre-calibrate multiple steel plates with different thicknesses and use the low-energy detector and the high-energy detector to detect each steel plate respectively to obtain a plurality of transmission images.

[0191] The electronic device can obtain the transmission images obtained by each detector detecting steel plates with different thicknesses. For each transmission image, calculate the grayscale values and standard deviations of the pixel points in the transmission image. The electronic device can establish a corresponding relationship for each of the low-energy detector and the high-energy detector, which is used to characterize the corresponding relationship between the grayscale value and the thickness, based on the grayscale value, standard deviation, and steel plate thickness corresponding to each transmission image of the detector, and use this corresponding relationship to make a grayscale template table.

[0192] To meet the display requirements, the corresponding relationship between the thickness and the grayscale value can also be set in advance. In this way, when using the dual-energy security inspection machine to detect the object to be inspected, the over-pack grayscale image I1(i1, j1) and the over-pack grayscale image I2(i2, j2) of the object to be inspected can be obtained.

[0193] Furthermore, the electronic device can calculate the thickness information matrices x1(i1, j1) and x2(i2, j2) of the above over-pack grayscale images according to the grayscale template table. Then, according to the two thickness information matrices x1(i1, j1), x2(i2, j2), the standard deviation of each pixel point, and the composite thickness formula Calculate the composite thickness of each composite pixel point in the fused image. Then, according to the above corresponding relationship between the thickness and the grayscale value, calculate the grayscale value of each composite pixel point in the fused image, so as to obtain the fused grayscale image Y(i, j) = f(x(i, j)).

[0194] Using this image fusion method, the transmission images in FIGS. 11(a) and 11(b) are fused to obtain a clearer fused image as shown in FIG. 11(c).

[0195] Corresponding to the above image fusion method, an embodiment of the present application also provides an image fusion device. The following introduces an image fusion device provided by an embodiment of the present application.

[0196] As Figure 12 shown, an image fusion device includes:

[0197] An image acquisition module 1201, configured to acquire a plurality of transmission images obtained by a plurality of detectors detecting an object to be inspected;

[0198] A first thickness determination module 1202, configured to, for each transmission image, determine the thickness corresponding to each pixel point in the transmission image according to the grayscale value of each pixel point in the transmission image and the first corresponding relationship established in advance for the transmission image, which is used to characterize the corresponding relationship between the grayscale value and the thickness, where the first corresponding relationship is determined in advance according to the detection performance of the detector that detects the transmission image;

[0199] A second thickness determination module 1203, configured to determine a target thickness corresponding to each fused pixel in the to-be-generated fused image based on the thickness corresponding to a base pixel, where the base pixel is a pixel in the multiple transmission images that has the same image position as this fused pixel;

[0200] A grayscale value determination module 1204, configured to determine a target grayscale value corresponding to each fused pixel according to the target thickness corresponding to this fused pixel and a preset second correspondence between thickness and grayscale value, so as to obtain the fused image.

[0201] It can be seen that in the solution provided in the embodiment of the present application, an electronic device can obtain multiple transmission images obtained by multiple detectors detecting an object to be detected. For each transmission image, according to the grayscale value of each pixel in this transmission image and a pre-established first correspondence between grayscale value and thickness corresponding to this transmission image, determine the thickness corresponding to each pixel in this transmission image, where the first correspondence is determined in advance according to the detection performance of the detector that detected this transmission image; for each fused pixel in the to-be-generated fused image, a target thickness corresponding to this fused pixel can be determined based on the thickness corresponding to the base pixel in the multiple transmission images that has the same image position as this fused pixel, and then for each fused pixel, according to the target thickness corresponding to this fused pixel and a preset second correspondence between thickness and grayscale value, determine the target grayscale value corresponding to this fused pixel, so as to obtain the fused image.

[0202] Since the detection performances of different detectors for different thicknesses are different, the accuracy of the thickness of the object to be detected represented by the grayscale value of each pixel in the transmission image of each detector is also different. During the image fusion process, based on the first correspondence and the grayscale value of each pixel, determine the thickness corresponding to the position of this pixel, and then use the preset second correspondence to map the thickness back to the grayscale space. The pixels at the same position in each transmission image can be flexibly fused according to the determined thickness and the detection performance of each detector. Compared with the fusion method of assigning fixed weights to multiple transmission images in advance, the image quality of the obtained fused image is better, and the detailed information inside different thickness regions of the object to be detected can be clearly displayed. Moreover, by flexibly customizing the second correspondence, different thicknesses can be mapped to the expected grayscale values. In this way, without performing additional image processing, the distribution of regions with different thicknesses of the object to be detected can be relatively intuitively displayed in the fused image.

[0203] As an implementation manner of the embodiment of the present application, the second thickness determination module 1203 includes:

[0204] A weight calculation unit, configured to calculate the fusion weight of each base pixel for a fusion pixel in the to-be-generated fusion image according to the gray value of the base pixel;

[0205] A target thickness calculation unit, configured to calculate the weighted sum value of the thicknesses corresponding to the base pixels for each of the fusion pixels based on the fusion weights corresponding to the base pixels, so as to obtain the target thickness corresponding to the fusion pixel.

[0206] As an implementation manner of an embodiment of the present application, the first correspondence relationship includes the correspondence relationship between the gray value, the standard deviation of the gray value, and the thickness; the weight calculation unit may include:

[0207] A fusion weight determination subunit, configured to calculate the preset coefficient corresponding to each base pixel for a fusion pixel in the to-be-generated fusion image as the fusion weight of the base pixel based on the expression of the preset coefficient corresponding to the base pixel corresponding to the fusion pixel determined in advance, the gray value of the base pixel, and the standard deviation of the gray value;

[0208] Wherein, the expression of the preset coefficient is a function of the variance of the gray value corresponding to the fusion pixel with respect to the preset coefficients corresponding to each base pixel established in advance based on the gray value and the standard deviation of the gray value of the base pixel, and is the expression when the variance of the gray value obtains the minimum value; or, the expression of the preset coefficient is a function of the gray value probability corresponding to the fusion pixel with respect to the preset coefficients corresponding to each base pixel established in advance based on the gray value and the standard deviation of the gray value of the base pixel, and is the expression when the gray value probability obtains the maximum value.

[0209] As an implementation manner of an embodiment of the present application, the expression is:

[0210]

[0211] Where k i is the fusion weight of the i-th base pixel; σ i is the standard deviation of the gray value corresponding to the i-th base pixel, 0 < i < n, and n is the number of base pixels;

[0212] As an implementation manner of an embodiment of the present application, the target thickness calculation unit includes:

[0213] A target thickness calculation subunit, configured to calculate the target thickness corresponding to the fusion pixel according to the following formula:

[0214]

[0215] Where x is the target thickness, xi is the thickness corresponding to the i-th basic pixel point.

[0216] As an implementation manner of the embodiment of the present application, the second corresponding relationship includes a corresponding relationship between the thickness and the gray value corresponding to different target regions of the object to be detected preset;

[0217] The gray value determination module 1204 includes:

[0218] The region determination unit is configured to determine, for each fused pixel point, the target region corresponding to the fused pixel point;

[0219] The gray value determination unit is configured to determine the target gray value corresponding to the fused pixel point according to the target thickness corresponding to the fused pixel point and the preset second corresponding relationship corresponding to the target region.

[0220] As an implementation manner of the embodiment of the present application, the device further includes a first corresponding relationship establishment module, and the first corresponding relationship establishment module includes:

[0221] The sample image acquisition unit is configured to acquire, for each detector, a sample transmission image obtained by the detector detecting samples with different thicknesses;

[0222] The information calculation unit is configured to calculate, for each sample transmission image, the gray value information of the pixel points in the sample transmission image;

[0223] The relationship establishment unit is configured to establish, for each detector, a first corresponding relationship for characterizing the corresponding relationship between the gray value and the thickness according to the gray value information of each sample transmission image corresponding to the detector and the sample thickness.

[0224] As an implementation manner of the embodiment of the present application, the image acquisition module 1201 includes:

[0225] The image acquisition sub-module is configured to acquire a high-energy transmission image obtained by the high-energy detector in the dual-energy detection device detecting the object to be detected, and a low-energy transmission image obtained by the low-energy detector in the dual-energy detection device detecting the object to be detected.

[0226] Corresponding to the above image fusion method, an embodiment of the present application further provides an image fusion system, and the following introduces an image fusion system provided by an embodiment of the present application.

[0227] As Figure 13 shown, an image fusion system, the system includes a security inspection machine 1301 and an image processing device 1302, and the security inspection machine 1301 may include an X-ray source 13011 and a detector 13012.

[0228] The X-ray source 13011 is configured to emit X-rays;

[0229] The detector 13012 is configured to detect the object to be inspected and obtain a transmission image;

[0230] The image processing device 1302 is configured to obtain a plurality of transmission images obtained by a plurality of detectors detecting the object to be inspected; for each transmission image, according to the gray value of each pixel point in the transmission image and a pre-established first correspondence relationship corresponding to the transmission image for characterizing the relationship between the gray value and the thickness, determine the thickness corresponding to each pixel point in the transmission image, wherein the first correspondence relationship is pre-determined according to the detection performance of the detector that detects the transmission image; for each fusion pixel point in the fusion image to be generated, based on the thickness corresponding to the base pixel point, determine the target thickness corresponding to the fusion pixel point, wherein the base pixel point is the pixel point with the same image position as the fusion pixel point in the plurality of transmission images; for each fusion pixel point, according to the target thickness corresponding to the fusion pixel point and a preset second correspondence relationship between the thickness and the gray value, determine the target gray value corresponding to the fusion pixel point, and obtain the fusion image.

[0231] It can be seen that since the detection performances of different detectors for different thicknesses are different, the accuracy of the thickness of the object to be inspected represented by the gray value of each pixel point in the transmission images of each detector is also different. During the image fusion process, based on the first correspondence relationship and the gray value of each pixel point, determine the thickness corresponding to the position of the pixel point, and then use the pre-set second correspondence relationship to map the thickness back to the gray space. The pixel points with the same position in each transmission image can be flexibly fused according to the determined thickness and the detection performance of each detector. Compared with the fusion method of pre-assigning fixed weights to a plurality of transmission images, the image quality of the obtained fusion image is better, and the detailed information inside different thickness regions of the object to be inspected can be clearly displayed. Moreover, by flexibly customizing the second correspondence relationship, different thicknesses can be mapped to the desired gray values to be displayed. In this way, without performing additional image processing, the distribution of different thickness regions of the object to be inspected can be more intuitively displayed in the fusion image.

[0232] An embodiment of the present application further provides an electronic device, as Figure 14 shown, including:

[0233] A memory 1401 for storing a computer program;

[0234] A processor 1402, configured to implement the method steps described in any of the above embodiments when executing the program stored in the memory 1401.

[0235] And the above-mentioned electronic device may further include a communication bus and / or a communication interface. The processor 1402, the communication interface, and the memory 1401 complete communication with each other through the communication bus.

[0236] The communication bus mentioned in the above-mentioned electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of representation, only a thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0237] The communication interface is used for communication between the above-mentioned electronic device and other devices.

[0238] The memory may include a Random Access Memory (RAM), and may also include a Non-Volatile Memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located far from the aforementioned processor.

[0239] The above-mentioned processor may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0240] In another embodiment provided by the present application, there is also provided a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the steps of the method described in any of the above embodiments are implemented.

[0241] In another embodiment provided by the present application, there is also provided a computer program product containing instructions, which when running on a computer, causes the computer to execute the method steps described in any of the above embodiments.

[0242] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a solid-state disk (SSD), etc.

[0243] It should be noted that in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including", or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article, or device including a series of elements includes not only those elements but also other elements not explicitly listed, or also includes elements inherent to such process, method, article, or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article, or device including the element.

[0244] Each embodiment in this specification is described in a related manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the embodiments of the device, system, electronic device, computer-readable storage medium, and computer program product, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiments.

[0245] The above are only the preferred embodiments of the present application and are not intended to limit the protection scope of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application are all included in the protection scope of the present application.

Claims

1. An image fusion method, characterized in that, The method includes: Obtaining a plurality of transmission images obtained by detecting a to-be-detected object with a plurality of detectors; For each transmission image, according to the gray value of each pixel point in the transmission image and a pre-established first correspondence relationship between the gray value and the thickness corresponding to the transmission image, determining the thickness corresponding to each pixel point in the transmission image, where the first correspondence relationship is determined in advance according to the detection performance of the detector that detects the transmission image; For each fusion pixel point in the to-be-generated fusion image, based on the thickness corresponding to the basic pixel point, determining the target thickness corresponding to the fusion pixel point, where the basic pixel point is the pixel point with the same image position as the fusion pixel point in the plurality of transmission images; For each fusion pixel point, according to the target thickness corresponding to the fusion pixel point and a preset second correspondence relationship between the thickness and the gray value, determining the target gray value corresponding to the fusion pixel point, to obtain the fusion image.

2. The method according to claim 1, wherein The step of, for each fusion pixel point in the to-be-generated fusion image, based on the thickness corresponding to the basic pixel point, determining the target thickness corresponding to the fusion pixel point includes: For each fusion pixel point in the to-be-generated fusion image, calculating the fusion weight of each basic pixel point according to the gray value of the basic pixel point; For each fusion pixel point, based on the fusion weights corresponding to the basic pixel points, calculating the weighted sum value of the thicknesses corresponding to the basic pixel points, to obtain the target thickness corresponding to the fusion pixel point.

3. The method according to claim 2, wherein The first correspondence relationship includes the correspondence relationship between the gray value, the standard deviation of the gray value and the thickness; The step of, for each fusion pixel point in the to-be-generated fusion image, calculating the fusion weight of each basic pixel point according to the gray value of the basic pixel point includes: For each fusion pixel point in the to-be-generated fusion image, based on a pre-determined expression of the preset coefficient corresponding to the basic pixel point corresponding to the fusion pixel point, the gray value of the basic pixel point and the standard deviation of the gray value, calculating the preset coefficient corresponding to each basic pixel point as the fusion weight of the basic pixel point; where the expression of the preset coefficient is the expression when the variance of the gray value corresponding to the fusion pixel point with respect to the preset coefficients corresponding to each basic pixel point, established in advance based on the gray value and the standard deviation of the gray value of the basic pixel point, takes the minimum value; or, the expression of the preset coefficient is the expression when the probability of the gray value corresponding to the fusion pixel point with respect to the preset coefficients corresponding to each basic pixel point, established in advance based on the gray value and the standard deviation of the gray value of the basic pixel point, takes the maximum value.

4. The method according to claim 3, characterized in that, The expression is: Among them, k i is the fusion weight of the i-th basic pixel point; σ i is the standard deviation of the gray value corresponding to the i-th basic pixel point, 0 < i < n, where n is the number of basic pixel points; The step of, for each fusion pixel point, based on the fusion weights corresponding to the basic pixel points, calculating the weighted sum value of the thicknesses corresponding to the basic pixel points, to obtain the target thickness corresponding to the fusion pixel point includes: Calculating the target thickness corresponding to the fusion pixel point according to the following formula: where x is the target thickness, and x i is the thickness corresponding to the i-th basic pixel point.

5. The method according to any one of claims 1-4, characterized in that The second correspondence relationship includes the correspondence relationship between the thickness and the gray value corresponding to different target regions of the to-be-detected object preset; For each of the fused pixel points, determining a target gray value corresponding to the fused pixel point according to a target thickness corresponding to the fused pixel point and a second correspondence between thickness and gray value preset includes: For each of the fused pixel points, determining a target area corresponding to the fused pixel point; According to the target thickness corresponding to the fused pixel point and the second correspondence corresponding to the preset target area, determining the target gray value corresponding to the fused pixel point.

6. The method according to any one of claims 1-4, characterized in that The establishing manner of the first correspondence includes: For each detector, obtaining a sample transmission image obtained by the detector detecting samples with different thicknesses; For each sample transmission image, calculating gray value information of pixel points in the sample transmission image; For each detector, according to the gray value information of each sample transmission image corresponding to the detector and the sample thickness, establishing a first correspondence corresponding to the detector for characterizing the correspondence between gray value and thickness.

7. The method according to any one of claims 1-4, characterized in that The obtaining multiple transmission images obtained by multiple detectors detecting a test object includes: Obtaining a high-energy transmission image obtained by a high-energy detector in a dual-energy detection device detecting the test object, and a low-energy transmission image obtained by a low-energy detector in the dual-energy detection device detecting the test object.

8. An image fusion device, characterized in that, The device includes: An image acquisition module, configured to obtain multiple transmission images obtained by multiple detectors detecting a test object; A first thickness determination module, configured to, for each transmission image, determine thicknesses corresponding to respective pixel points in the transmission image according to gray values of each pixel point in the transmission image and a first correspondence corresponding to the transmission image for characterizing the correspondence between gray value and thickness, where the first correspondence is determined in advance according to detection performance of the detector that detects the transmission image; A second thickness determination module, configured to, for each fused pixel point in a to-be-generated fused image, determine a target thickness corresponding to the fused pixel point based on the thickness corresponding to a base pixel point, where the base pixel point is a pixel point having the same image position as the fused pixel point in the multiple transmission images; A gray value determination module, configured to, for each of the fused pixel points, determine a target gray value corresponding to the fused pixel point according to the target thickness corresponding to the fused pixel point and a second correspondence between thickness and gray value preset, to obtain the fused image.

9. An electronic device, characterized in that, including: A memory, configured to store a computer program; A processor, configured to implement the method according to any one of claims 1-7 when executing the program stored on the memory.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1-7 is implemented.

Images