An image enhancement method and device for X-ray security inspection machines

Abstract

This invention provides an image enhancement method and apparatus for X-ray security inspection machines. The method includes: acquiring a security inspection display video output by the X-ray security inspection machine; extracting a region of interest (ROI) map from the X-ray image frame sequence corresponding to the security inspection display video; striping the ROI map containing foreground items into multiple striped images, and performing super-resolution processing on the multiple striped images to obtain multiple super-resolution sub-images; and stitching the multiple super-resolution sub-images to obtain a target enhanced image. This method, by extracting the ROI from the security X-ray image and performing striping processing, can focus on key information areas and reduce the computational load of the super-resolution algorithm. By performing super-resolution processing on multiple striped images separately, it achieves precise enhancement of the ROI in the pseudo-color X-ray images output by older X-ray security inspection machines. It significantly improves image quality without replacing the core hardware of the X-ray security inspection machine and has network expansion capabilities.

Description

Technical Field

[0001] This invention relates to the field of security inspection image processing technology, and in particular to an image enhancement method and apparatus for X-ray security inspection machines. Background Technology

[0002] In the public transportation sector, X-ray security scanners, as key equipment for ensuring passenger safety and maintaining public order, are widely used in subways, train stations, airports, and other locations to conduct security checks on passengers' luggage and belongings, promptly detecting and intercepting various prohibited or dangerous items. However, early deployed X-ray security scanners generally suffered from poor image quality, such as blurry images, significant noise, and low contrast, severely affecting security personnel's accurate interpretation of image content and increasing the risk of missed or false detections. Furthermore, these early devices were mostly stand-alone units lacking network communication capabilities, making it impossible to upload collected image data to servers or the cloud for centralized management or further processing, thus limiting their intelligent upgrades and remote monitoring capabilities.

[0003] Existing image enhancement methods primarily target natural scene images captured by visible light or infrared cameras, improving image quality through multi-view image fusion or video sequence denoising. However, these methods are difficult to directly apply to X-ray security inspection images. This is because X-ray security inspection images are colored based on the attenuation response value of X-rays after penetrating an object, using manually set pseudo-color mapping rules. Their image quality is affected by various factors such as the stability of the X-ray source, detector sensitivity, and electronic noise. Their noise characteristics, blurring causes, and image structure differ fundamentally from natural scene images. Therefore, traditional visible light image enhancement algorithms have limited effectiveness in processing X-ray pseudo-color images and may even introduce new artifacts or distortions.

[0004] More importantly, there are currently no image enhancement devices or equipment specifically designed for older X-ray security inspection machines. Due to hardware limitations, older X-ray security inspection machines cannot have their image quality improved through simple software upgrades, while complete replacement faces problems such as high costs and long deployment cycles.

[0005] Therefore, how to solve the problem that existing technologies cannot effectively enhance the false-color X-ray images output by old X-ray security inspection machines, resulting in poor image quality, is an important issue that urgently needs to be addressed in the field of security inspection image processing. Summary of the Invention

[0006] This invention provides an image enhancement method and apparatus for X-ray security inspection machines, which overcomes the shortcomings of existing technologies that cannot effectively enhance the false-color X-ray images output by old X-ray security inspection machines, resulting in poor imaging quality. It can significantly improve image quality without replacing core hardware and has network expansion capabilities.

[0007] On one hand, the present invention provides an image enhancement method for X-ray security inspection machines, comprising: acquiring a security inspection display video output by the X-ray security inspection machine; extracting a region of interest map from the security X-ray image frame sequence corresponding to the security inspection display video; striping the region of interest map containing foreground objects into multiple striped images, and performing super-resolution processing on the multiple striped images to obtain multiple super-resolution sub-images; and stitching the multiple super-resolution sub-images to obtain a target enhanced image.

[0008] Furthermore, the region of interest (ROI) is extracted from the X-ray image frame sequence corresponding to the security inspection display video, including: for each X-ray image, determining the operation area image in the security X-ray image based on the color scheme of the X-ray security inspection machine's operating interface, and removing the operation area image from the security X-ray image to obtain an initial ROI; determining the boundary position of the ROI based on the proportion of different pixel categories in each pixel row and column of the initial ROI; and extracting the ROI from the initial ROI based on the boundary position of the ROI.

[0009] Furthermore, based on the proportion of different pixel categories in each pixel row and column of the initial region of interest (ROI) image, the boundary position of the ROI image is determined, including: projecting the initial ROI image along the Y direction to obtain multiple pixel columns in the Y direction; if the proportion of foreground pixels in a pixel column is lower than a set proportion threshold, the pixel column is determined as a background pixel column; if the proportion of foreground pixels in a pixel column is greater than or equal to a set proportion threshold, the pixel column is determined as a foreground pixel column; the nearest and farthest foreground pixel columns in the X direction are determined as the X-direction boundary position of the ROI image; wherein, the Y direction is perpendicular to the X direction, and the X direction is the path direction of the X-ray security inspection machine outputting the security X-ray image.

[0010] Furthermore, based on the proportion of different pixel categories in each pixel row and column of the initial region of interest (ROI) image, the boundary position of the ROI image is determined, including: projecting the initial ROI image along the X direction to obtain multiple pixel rows in the X direction; if the proportion of foreground pixels in a pixel row is lower than a set proportion threshold, the pixel row is determined as a background pixel row; if the proportion of foreground pixels in a pixel row is greater than or equal to a set proportion threshold, the pixel row is determined as a foreground pixel row; and the foreground pixel rows at the nearest and farthest ends in the Y direction are determined as the boundary positions of the ROI image in the Y direction.

[0011] Furthermore, the region of interest (ROI) map containing foreground objects is striped into multiple striped images, including: classifying the ROI map based on a pre-trained image classification network to obtain image classification results; if the image classification result indicates that the ROI map is a foreground object image, the ROI map is retained and striped into multiple striped images; if the image classification result indicates that the ROI map is a non-foreground object image, the ROI map is removed.

[0012] Further, the step of performing super-resolution processing on multiple striped images to obtain multiple super-resolution sub-images specifically includes: denoting the multiple striped images obtained by striping the region of interest map containing foreground objects as a striped image set; for each striped image in the striped image set, matching the striped image with the processed images in the initially empty super-resolution image set; if there is a processed image in the super-resolution image set that matches the striped image, using the processed image as the super-resolution processing result of the striped image; if there is no processed image in the super-resolution image set that matches the striped image, performing super-resolution processing on the striped image to obtain the super-resolution sub-image corresponding to the striped image, and adding the super-resolution sub-image to the super-resolution image set.

[0013] Further, the super-resolution sub-image is added to the super-resolution image set, followed by: removing the striped image furthest from the current time in the striped image set, and simultaneously removing the super-resolution sub-image furthest from the current time in the super-resolution image set; wherein the number of images in the striped image set is the same as and fixed as the number of images in the super-resolution image set.

[0014] Furthermore, super-resolution processing is performed on the striped image to obtain a super-resolution sub-image corresponding to the striped image. This includes: inputting the striped image into a pre-trained image enhancement model and outputting a super-resolution sub-image corresponding to the striped image; wherein, the image enhancement model is constructed based on a generative adversarial network, and is obtained by adversarial training on image samples composed of high-resolution image samples and corresponding low-resolution image samples, and fine-tuning it based on security X-ray image samples. The high-resolution image samples have a first resolution, and the low-resolution image samples have a second resolution, with the first resolution being much higher than the second resolution.

[0015] Furthermore, acquiring the security inspection display video output by the X-ray security inspection machine includes: acquiring the security inspection display video output by the X-ray security inspection machine through a video acquisition module connected to the display port of the X-ray security inspection machine; wherein, the security inspection display video includes a security X-ray image that is continuously scrolled along the X direction, and the X direction is the path direction of the security X-ray image output by the X-ray security inspection machine.

[0016] Secondly, the present invention also provides an image enhancement device for an X-ray security inspection machine, comprising: a security inspection display video acquisition module for acquiring the security inspection display video output by the X-ray security inspection machine; a region of interest (ROI) map extraction module for extracting the ROI map from the security X-ray image frame sequence corresponding to the security inspection display video; an image super-resolution processing module for striping the ROI map containing foreground objects into multiple striped images, and performing super-resolution processing on the multiple striped images to obtain multiple super-resolution sub-images; and a super-resolution sub-image stitching module for stitching the multiple super-resolution sub-images to obtain a target enhancement image.

[0017] This invention provides an image enhancement method for X-ray security inspection machines. It acquires the security inspection display video output by the X-ray security inspection machine, extracts the region of interest (ROI) map from the corresponding X-ray image frame sequence, then stripes the ROI map containing foreground items into multiple striped images. Super-resolution processing is then performed on these striped images to obtain multiple super-resolution sub-images. These sub-images are then stitched together to obtain the target enhanced image. This method, by extracting the ROI from the security X-ray image and performing striping processing, can focus on key information areas and reduce the computational load of the super-resolution algorithm. By performing super-resolution processing on multiple striped images separately, it achieves precise enhancement of the ROI in the pseudo-color X-ray images output by older X-ray security inspection machines. It significantly improves image quality without replacing the core hardware of the X-ray security inspection machine and has network expansion capabilities. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the X-ray pseudo-color image output by a conventional X-ray security inspection machine provided in an embodiment of the present invention.

[0020] Figure 2 This is a schematic flowchart of an image enhancement method for X-ray security inspection machines provided in an embodiment of the present invention.

[0021] Figure 3 This is a schematic diagram of the X-ray security inspection image output by the X-ray security inspection machine provided in the embodiment of the present invention.

[0022] Figure 4 This is the determination provided by the embodiments of the present invention. Figure 3 A schematic diagram of the operational area image and region of interest in a security X-ray image.

[0023] Figure 5 This is provided by the embodiments of the present invention. Figure 3 A schematic diagram of multiple striped images corresponding to the region of interest map in the image.

[0024] Figure 6 This is a schematic diagram comparing the striped image and the corresponding super-resolution sub-image provided in an embodiment of the present invention.

[0025] Figure 7 This is a schematic diagram of the overall architecture of the image enhancement method for X-ray security inspection machines provided in this embodiment of the invention.

[0026] Figure 8 This is a schematic diagram of the overall process of the image enhancement method for X-ray security inspection machines provided in the embodiments of the present invention.

[0027] Figure 9 This is a schematic diagram of the structure of the image enhancement device for X-ray security inspection machines provided in an embodiment of the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0029] It should be noted that X-ray security scanners are becoming increasingly common in public transportation. Scanning passengers' luggage and bags to detect prohibited or restricted dangerous items early has become a crucial means of ensuring passenger safety and the normal operation of public transportation. However, early X-ray security scanners had poor image quality, suffering from problems such as blurry images and high noise levels. Figure 1 The diagram illustrates a schematic of the output image of a traditional X-ray security inspection machine, specifically an X-ray pseudo-color image, provided in an embodiment of the present invention. Furthermore, traditional X-ray security inspection machines operate independently without network connectivity, making it impossible to transmit images to a server or cloud for post-processing.

[0030] To address these issues, existing technologies have proposed several image enhancement methods, primarily targeting natural scene images captured by visible light or infrared cameras. These methods improve image quality through multi-view image fusion or video sequence denoising. However, natural scene images captured by visible light or infrared cameras differ significantly from pseudo-color images output by X-ray security inspection machines in terms of noise characteristics, blurring causes, and image structure. This means that image enhancement algorithms suitable for natural scene images captured by visible light or infrared cameras are not applicable to pseudo-color images output by X-ray security inspection machines. Furthermore, there are currently no devices or equipment available for image enhancement of older X-ray security inspection machines.

[0031] In view of this, the present invention proposes an image enhancement method for X-ray security inspection machines, specifically, Figure 2 A schematic flowchart of the image enhancement method for X-ray security inspection machines provided in an embodiment of the present invention is shown.

[0032] like Figure 2 As shown, the method includes: S210, acquiring the security inspection display video output by the X-ray security inspection machine; S220, extracting the region of interest map from the security X-ray image frame sequence corresponding to the security inspection display video; S230, striping the region of interest map containing foreground objects into multiple striped images, and performing super-resolution processing on the multiple striped images to obtain multiple super-resolution sub-images; S240, stitching the multiple super-resolution sub-images to obtain a target enhancement image.

[0033] The following will provide a detailed description of steps S210-S240 and related steps.

[0034] S210: Acquire the security inspection display video output by the X-ray security inspection machine.

[0035] X-ray security inspection machines are devices that use the penetrating properties of X-rays to conduct security checks on luggage, packages, and other items. They are widely used in public places such as airports, subways, train stations, government agencies, and schools to detect whether they contain prohibited or dangerous items, such as knives, firearms, explosives, and drugs.

[0036] When an X-ray security inspection machine is working, its internal X-ray source emits a beam of fan-shaped or cone-shaped X-rays. These X-rays pass through the inspected item, and different materials absorb X-rays to varying degrees. A detector array located on the other side of the item receives the penetrated X-rays and converts them into electrical signals. Furthermore, the security inspection system generates a pseudo-color image of the item's internal structure based on the differences in the intensity of the electrical signals received by the detectors. Generally, organic materials (such as plastics, explosives, and drugs) appear orange, inorganic materials (such as metals) appear blue or green, and mixtures appear green.

[0037] To perform image enhancement processing on X-ray images output by a security inspection machine, it is first necessary to acquire the security inspection display video output by the machine. Specifically, this can be achieved by connecting an image enhancement device to one or more physical interfaces of the X-ray security inspection machine to receive video signals from the machine in real time. Converting the received video signals into digital video data yields the security inspection display video.

[0038] The image enhancement device can be considered as an apparatus for performing the image enhancement method for X-ray security inspection machines provided in the embodiments of the present invention.

[0039] The physical interface connecting the image enhancement device and the X-ray security inspection machine can be any one of the following: VGA, DVI, BNC, HDMI, and RJ45. No specific limitation is made here.

[0040] The security inspection video serves as the input source for subsequent image enhancement processing. It consists of a series of X-ray image frames that continuously scroll along the X-ray direction. Each frame of the security X-ray image reflects the internal structure of the inspected item at a certain moment. The X-ray direction is the direction in which the X-ray security inspection machine outputs the security X-ray image, which is also the direction of movement of the conveyor belt of the X-ray security inspection machine, and the direction in which the conveyor belt carries the item to be inspected through the security inspection channel.

[0041] Based on the security inspection display video output by the X-ray security inspection machine obtained in step S210, step S220 is further executed.

[0042] S220, Extract the region of interest map from the security X-ray image frame sequence corresponding to the security inspection display video.

[0043] As described above, the security screening video comprises a sequence of X-ray image frames that scroll continuously along the X-ray direction. In step S220, for each frame of the security X-ray image, a Region of Interest (ROI) map is extracted. The ROI map refers to a sub-image or region within the security X-ray image that requires special attention or processing and contains foreground items (such as passenger luggage or packages).

[0044] The region of interest (ROI) is extracted from each frame of the security X-ray image. Specifically, the security scanner operation area, which includes the buttons on the security scanner's interface, is removed from the X-ray image; these buttons do not require image enhancement. Furthermore, for the X-ray image after removing the security scanner operation area, rows and columns of pixels with a high proportion of background pixels also need to be removed. Thus, the ROI for each frame of the security X-ray image can be extracted. The specific extraction process of the ROI will be detailed in the following embodiments.

[0045] It is worth mentioning that this step extracts the region of interest map before enhancing the security X-ray image. The subsequent enhancement processing is only performed on the region of interest map, which can remove irrelevant information (such as background and noise), greatly reducing the number of pixels that need to be enhanced in the later process, thereby directly reducing the computational complexity and running time of the subsequent enhancement processing.

[0046] Based on step S220, which extracts the region of interest map from the security X-ray image frame sequence corresponding to the security inspection display video, step S230 is further executed.

[0047] S230, the region of interest map containing foreground objects is striped into multiple striped images, and super-resolution processing is performed on the multiple striped images to obtain multiple super-resolution sub-images.

[0048] Specifically, for the region of interest (ROI) extracted from each frame of the X-ray security inspection image, since noise or interference from the X-ray security inspection machine may cause color blocks, it is first necessary to classify the ROI to determine whether it is a real image containing passenger luggage or packages. This process can be implemented using existing image recognition algorithms or pre-trained classification networks; no specific limitation is made here.

[0049] Given that foreground items (such as passenger luggage or packages) are present in the region of interest (ROI) image, in order to further reduce the computational load of subsequent image enhancement processing, the ROI image is striped into multiple striped images. The width of the striped images can be set according to the speed of the security inspection machine conveyor belt, such as 1 pixel, 2 pixels, 8 pixels, 16 pixels, 32 pixels, 64 pixels, etc., without specific limitations here.

[0050] For multiple striped images corresponding to the same region of interest, super-resolution processing / enhancement processing is performed on each striped image to obtain a super-resolution sub-image corresponding to each striped image, thus obtaining multiple super-resolution sub-images corresponding to multiple striped images. This process can be implemented using a pre-trained image enhancement model, which will be explained in detail in the following embodiments.

[0051] In step S230, the region of interest map containing foreground objects is striped into multiple striped images, and super-resolution processing is performed on the multiple striped images to obtain multiple super-resolution sub-images. Then, step S240 is further executed.

[0052] S240, stitch the multiple super-resolution sub-images together to obtain the target enhanced image.

[0053] Specifically, by stitching together multiple super-resolution sub-images corresponding to the same security X-ray image, a complete super-resolution image, namely the target enhancement image, can be obtained.

[0054] In this embodiment, the method acquires the security inspection display video output by the X-ray security inspection machine, extracts the region of interest (ROI) map from the corresponding X-ray image frame sequence, and then stripes the ROI map containing foreground items into multiple striped images. Super-resolution processing is then performed on these striped images to obtain multiple super-resolution sub-images. These sub-images are then stitched together to obtain the enhanced target image. This method, by extracting the ROI from the security X-ray image and performing striping processing, can focus on key information areas and reduce the computational load of the super-resolution algorithm. By performing super-resolution processing on multiple striped images separately, it achieves precise enhancement of the ROI in the pseudo-color X-ray images output by older security X-ray security inspection machines. This significantly improves image quality without replacing the core hardware of the X-ray security inspection machine and also has network expansion capabilities.

[0055] Based on the above embodiments, the process of obtaining the security inspection display video output by the X-ray security inspection machine in step S210 will be described in detail below.

[0056] The video acquisition module, connected to the display port of the X-ray security inspection machine, acquires the security inspection display video output by the X-ray security inspection machine. The security inspection display video includes security X-ray images that are continuously scrolled along the X-axis, which is the direction in which the security X-ray images output by the X-ray security inspection machine move.

[0057] It is easy to understand that X-ray security inspection machines are equipped with multiple display ports, such as display port 1, display port 2, ..., display port N. These multiple display ports correspond one-to-one with multiple video acquisition modules of the image enhancement device. The value of N can be set according to specific circumstances and is not specifically limited here. For example, in a specific embodiment, N=2, that is, the X-ray security inspection machine is equipped with a main view display (output) port and a side view display (output) port.

[0058] Each display port is connected to its corresponding video capture module via any one of the following standard interfaces: VGA, DVI, BNC, HDMI, and RJ45. Based on this connection, the video capture module of the image enhancement device can receive video signals from the X-ray security inspection machine in real time, converting the received video signals into digital video data to obtain the security inspection display video.

[0059] The security inspection video serves as the input source for subsequent image enhancement processing. It consists of a series of X-ray image frames that continuously scroll along the X-ray direction. Each frame of the security X-ray image reflects the internal structure of the inspected item at a certain moment. The X-ray direction is the direction in which the X-ray security inspection machine outputs the security X-ray image, which is also the direction of movement of the conveyor belt of the X-ray security inspection machine, and the direction in which the conveyor belt carries the item to be inspected through the security inspection channel.

[0060] Based on the above embodiments, the process of extracting the region of interest map from the security X-ray image frame sequence corresponding to the security inspection display video in step S220 will be described in detail below.

[0061] Extracting the region of interest (ROI) from the X-ray image frame sequence corresponding to the security inspection display video includes: for each X-ray image, determining the operation area image in the security X-ray image based on the color scheme and background of the X-ray security inspection machine's operating interface, and removing the operation area image from the security X-ray image to obtain an initial ROI; determining the boundary position of the ROI based on the proportion of different pixel categories in each pixel row and column of the initial ROI; and extracting the ROI from the initial ROI based on the boundary position of the ROI.

[0062] As readily understood, as described in the embodiments above, the security inspection display video includes a series of security X-ray image frames that are continuously scrolled along the X-ray direction, i.e., frame by frame of security X-ray images. A particular frame of the security X-ray image is as follows: Figure 3 As shown, Figure 3 This diagram illustrates the X-ray security inspection image output by the X-ray security inspection machine provided in an embodiment of the present invention.

[0063] Because different X-ray security inspection machines are manufactured by different companies, it is difficult to obtain security inspection display videos containing only foreground items directly from the network port using the protocol. This results in the obtained security X-ray images containing useless security inspection machine operation areas. These operation areas include buttons for the security inspection machine's interface, and these buttons do not require image enhancement processing. Furthermore, image areas in the security X-ray image that do not contain foreground items also do not require image enhancement processing. Therefore, this embodiment removes the image portions containing the security inspection machine operation areas and the image portions without foreground items from the security X-ray image to extract the region of interest (ROI).

[0064] Specifically, for each X-ray security inspection image, the background color of the operating area (the interface of the X-ray security inspection machine) is uniform. By dividing the background color of the operating area image into pixels, pixels matching the background color are designated as candidate pixels for the operating area image. These candidate pixels are then labeled as connected components, forming rectangular connected components. This connected component represents the operating area image corresponding to the security inspection machine's operating interface. Figure 4 The area shown in the blue box is illustrated.

[0065] The operational area image is removed from the security X-ray image to obtain a security X-ray image containing only foreground items, i.e., the initial region of interest (ROI). The ROI has a white background, and the foreground items include luggage and packages passing through the security checkpoint of the X-ray machine.

[0066] Furthermore, the boundary positions of the region of interest (ROI) need to be determined based on the proportion of different pixel categories in each row and column of the initial ROI map. The boundary positions of the ROI map include the X-direction boundary position and the Y-direction boundary position.

[0067] For example, in one specific embodiment, the determination of the boundary position in the X direction is described in detail. Specifically, the initial region of interest map is projected along the Y direction using column pixels to obtain multiple pixel columns in the Y direction, and the pixel proportion of foreground pixels (non-white pixels) in each pixel column is counted. If the pixel proportion of foreground pixels in the pixel column is lower than a set proportion threshold, the pixel column is considered a background pixel column; conversely, if the pixel proportion of foreground pixels in the pixel column is greater than or equal to the set proportion threshold, the pixel column is considered a foreground pixel column.

[0068] In this context, the Y direction is perpendicular to the X direction, and the X direction is the direction in which the X-ray security inspection machine outputs the security X-ray image, which is the direction of movement of the conveyor belt of the X-ray security inspection machine, and also the direction in which the conveyor belt carries the items to be inspected through the security inspection channel.

[0069] The percentage threshold can be set according to needs, such as 1%, without specific limitations here.

[0070] By classifying pixel columns pixel by pixel in the X direction from left to right, the leftmost / farthest foreground pixel column can be found. Similarly, by classifying pixel columns pixel by pixel in the X direction from right to left, the rightmost / nearest foreground pixel column can be found. The leftmost / farthest and rightmost / nearest foreground pixel columns are the two boundary positions of the region of interest map in the X direction, and are simply referred to as the X-direction boundary positions.

[0071] For example, in another specific embodiment, the determination of the boundary position in the Y direction is described in detail. Specifically, the initial region of interest map is projected along the X direction using row pixels to obtain multiple pixel rows in the X direction, and the pixel proportion of foreground pixels (non-white pixels) in each pixel row is counted. If the pixel proportion of foreground pixels in a pixel row is lower than a set proportion threshold, the pixel row is considered a background pixel row; conversely, if the pixel proportion of foreground pixels in a pixel row is greater than or equal to the set proportion threshold, the pixel row is considered a foreground pixel row.

[0072] By classifying pixel rows pixel by pixel in the Y direction from top to bottom, the topmost / farthest foreground pixel row can be found. Similarly, by classifying pixel rows pixel by pixel in the Y direction from bottom to top, the bottommost / nearest foreground pixel row can be found. The topmost / farthest foreground pixel row and the bottommost / nearest foreground pixel row are the two boundary positions of the region of interest map in the Y direction, which are simply referred to as the Y-direction boundary positions.

[0073] Based on the above, the boundary positions of the region of interest (ROI) can be determined. The X-direction boundary position corresponds to the left and right boundaries of the ROI, and the Y-direction boundary position corresponds to the top and bottom boundaries of the ROI. Using the X-direction and Y-direction boundary positions of the ROI, the ROI can be extracted from the initial ROI to obtain the ROI map. (See [reference needed]). Figure 4 The area within the red box.

[0074] Based on the above embodiments, the process of striping the region of interest map containing foreground objects into multiple striped images in step S230 will be described in detail below.

[0075] The region of interest (ROI) map containing foreground objects is striped into multiple striped images, including: classifying the ROI map based on a pre-trained image classification network to obtain image classification results; if the image classification result indicates that the ROI map is a foreground object image, the ROI map is retained and striped into multiple striped images; if the image classification result indicates that the ROI map is a non-foreground object image, the ROI map is removed.

[0076] It's easy to understand that noise or interference from X-ray security scanners can cause color patches, meaning that the region of interest (ROI) extracted from each frame of a security X-ray image may not actually contain passenger luggage or packages. Therefore, before performing super-resolution processing on the ROI, it's necessary to determine in advance whether the ROI truly contains passenger luggage or packages.

[0077] Specifically, this embodiment uses a pre-trained image classification network to perform binary classification on the region of interest (ROI) image: by inputting the ROI image into the pre-trained image classification network, the image classification result can be obtained. The image classification result includes result 1 and result 2. Result 1 is a real image (foreground object image) in which the ROI image contains passenger luggage or packages, and result 2 is a noisy or interfering image (non-foreground object image) in which the ROI image contains noise or interference.

[0078] The image classification network is a binary classification network. Specifically, it can use the EfficientNet network or other pre-trained networks with binary classification capabilities. No specific restrictions are made here.

[0079] The training sample set used to train the image classification network includes foreground object image samples containing passenger luggage or packages and non-foreground object image samples (such as white background images, interfering images, and noisy images). When training the image classification network, either foreground object image samples or non-foreground object image samples are used as input, the predicted image category is used as output, and the difference between the predicted image category and the true label of the input image sample (foreground object image or non-foreground object image) is used as the training loss. After multiple rounds of iterative optimization, a well-trained image classification network can be obtained.

[0080] Furthermore, assuming that foreground items (such as passenger luggage or packages) exist in the region of interest (ROI) map, the ROI map is preserved for subsequent super-resolution processing. Moreover, to further reduce the computational cost of subsequent image enhancement processing, the ROI map is striped into multiple striped images, as detailed in [reference needed]. Figure 5 This diagram illustrates multiple striped images corresponding to a region of interest (ROI) map provided in an embodiment of the present invention. If it is determined that the ROI map does not contain foreground objects, the ROI map is removed, and no subsequent image enhancement processing is performed on it.

[0081] The width of the striped image can be set according to the speed of the security inspection machine's conveyor belt, such as 1 pixel, 2 pixels, 8 pixels, 16 pixels, 32 pixels, 64 pixels, etc., without specific limitations. The faster the conveyor belt moves, the wider the striped image can be set, and vice versa. This is because as the foreground item moves through the security inspection channel with the conveyor belt, a security X-ray image containing the foreground item is generated. The length of the region of interest (ROI) containing the foreground item in the X-axis direction is compressed or stretched depending on the speed of the conveyor belt.

[0082] Specifically, such as Figure 4 As shown, the size of the foreground object in the X direction is the same as the actual size of the foreground object. That is, the length-width-height ratio of the foreground object in the figure is the same as the actual length-width-height ratio of the foreground object. At this time, the speed of the transmission belt is the standard speed.

[0083] When the conveyor belt speed is greater than the standard speed, the length of the region of interest (ROI) map containing the foreground object in the X direction is compressed, which means... Figure 4The length of the area in the red box in the X direction will become shorter. In order to prevent the striped image from containing too many feature elements of the foreground objects and thus reduce the amount of subsequent calculations, and also to facilitate the successful matching of the striped image with the processed images in the initially empty super-resolution image set, the width of the striped image can be set as small as possible, such as 1 pixel, 2 pixels, 4 pixels or 8 pixels.

[0084] Similarly, when the conveyor belt speed is less than the standard speed, the length of the region of interest map containing the foreground object in the X direction is stretched, that is... Figure 4 The length of the area within the red box in the X direction will increase. In order to ensure that the striped image contains at least one complete feature element of the foreground object, which is conducive to its successful matching with the processed images in the initially empty super-resolution image set, the width of the striped image can be set as large as possible, such as 32 pixels or 64 pixels.

[0085] It should be noted that for the preserved region of interest (ROI) map, the multiple striped images obtained by striping it all have the corresponding ROI map identification labels (such as the number of the current ROI map) for use when stitching together super-resolution sub-images.

[0086] It should also be noted that the region of interest (ROI) is striped in this embodiment to reduce the computational load of subsequent image enhancement / super-resolution processing. This is because security X-ray images are displayed in a scrolling mode (e.g., sliding from left to right, column by column, pixel by pixel). There are a large number of overlapping foreground object image areas between two adjacent frames of security X-ray images. The striped images of the overlapping parts do not need to undergo repeated image enhancement / super-resolution processing.

[0087] Based on the above embodiments, the process of performing super-resolution processing on multiple striped images in step S230 will be described in detail below.

[0088] The steps of performing super-resolution processing on multiple striped images to obtain multiple super-resolution sub-images specifically include: denoteing the multiple striped images obtained by striping the region of interest map containing foreground objects as a striped image set; for each striped image in the striped image set, matching the striped image with the processed images in the initially empty super-resolution image set; if a processed image matching the striped image exists in the super-resolution image set, using the processed image as the super-resolution processing result of the striped image; if no processed image matching the striped image exists in the super-resolution image set, performing super-resolution processing on the striped image to obtain the corresponding super-resolution sub-image, and adding the super-resolution sub-image to the super-resolution image set.

[0089] First, multiple striped images containing regions of interest with foreground objects are denoted as a striped image set. The set of multiple striped images that have undergone super-resolution processing is denoted as the super-resolution image set. The super-resolution image set is initially empty. Here, M is the minimum value required to display a complete foreground object.

[0090] For each new striped image Before performing super-resolution processing, the striped image is compared with the super-resolution image set. The processed images (striped images that have undergone super-resolution processing, i.e., super-resolution sub-images) in the super-resolution image set are matched. The image contains elements related to the striped image. Matching processed images Then the processed image As this striped image The results of super-resolution processing; if the super-resolution image set There is no corresponding image in the striped image. If a matching processed image is found, then the striped image is... Perform super-resolution processing and add the resulting super-resolution sub-images to the super-resolution image set. middle.

[0091] It should be noted that the super-resolution image set There may be something related to this striped image. Matching processed images The reason is that in a series of security X-ray image frames that are continuously scrolled along the X-ray direction, there will be overlapping areas between adjacent frames. The smaller the width of the striped image is set, the smaller the super-resolution image set becomes. The image contains elements related to the striped image. Matching processed images The higher the probability, the better.

[0092] Adding super-resolution subgraphs to a super-resolution image set After that, the striped image set will be... The striped image furthest from the current time Remove the set and add the super-resolution image set The super-resolution sub-image furthest from the current time Remove from the set to preserve the striped image set. The number of striped images and the super-resolution image set The number of super-resolution subgraphs in the image is consistent and fixed, for example, M+1.

[0093] Furthermore, super-resolution processing is performed on the striped image to obtain a super-resolution sub-image corresponding to the striped image. This includes: inputting the striped image into a pre-trained image enhancement model and outputting a super-resolution sub-image corresponding to the striped image; wherein, the image enhancement model is constructed based on a generative adversarial network, and is obtained by adversarial training on image samples composed of high-resolution image samples and corresponding low-resolution image samples, and fine-tuning it based on security X-ray image samples. The high-resolution image samples have a first resolution, and the low-resolution image samples have a second resolution, with the first resolution being much higher than the second resolution.

[0094] In other words, this embodiment pre-trains a model specifically designed for image enhancement / super-resolution processing of striped images, i.e., an image enhancement model. The image enhancement model is built upon a Generative Adversarial Network (GAN) and consists of a generator and a discriminator. The generator employs a ResNet structure with an attention mechanism, and its main function is to generate high-resolution images from low-resolution images. The discriminator evaluates the high-resolution images generated by the generator and feeds the evaluation results back to the generator, allowing it to optimize the model.

[0095] Specifically, the training sample set used to train the image augmentation model includes high-resolution image samples and corresponding low-resolution image samples. The high-resolution image samples have a first resolution, and the low-resolution image samples have a second resolution. The first resolution is significantly higher than the second resolution, but the specific values ​​of the first and second resolutions can be set according to actual needs and are not specifically limited here.

[0096] Optionally, the high-resolution image sample can be a high-resolution, high-definition image IH from the field of photography. A low-resolution, low-definition image IL (i.e., a low-resolution image sample) is generated from the image IH using a degradation model, forming an IH-IL image sample pair.

[0097] When training the image augmentation model, the generator takes low-resolution image samples as input and high-resolution images as output. The discriminator takes the high-resolution images output by the generator as input, evaluates the high-resolution images by comparing them with the corresponding high-resolution images of the input low-resolution images, and feeds the evaluation results back to the generator to optimize its parameters, ultimately resulting in a trained generator. The trained generator is then used as the initial image augmentation model.

[0098] The initial image enhancement model needs to be fine-tuned using acquired security X-ray image samples. During fine-tuning, the striped image samples obtained from the striped security X-ray image samples are used as input, and the striped image samples after super-resolution processing are used as output. After fine-tuning, the final image enhancement model can be obtained.

[0099] Super-resolution processing of the striped image was performed using a trained image enhancement model. The resulting super-resolution sub-image can be found in [reference needed]. Figure 6 , Figure 6 This diagram illustrates a comparison between a striped image and its corresponding super-resolution sub-image provided in an embodiment of the present invention. Figure 6 In the image, the left image is the original image, i.e., the striped image without super-resolution processing, and the right image is the result image after super-resolution processing, i.e., the super-resolution sub-image.

[0100] In some other embodiments, Figure 7 A schematic diagram of the overall architecture of the image enhancement method for X-ray security inspection machines provided in an embodiment of the present invention is shown.

[0101] like Figure 7 As shown, the X-ray security inspection machine is equipped with multiple display ports, such as display port 1, display port 2, ..., display port N. These multiple display ports correspond one-to-one with multiple video acquisition modules (such as video acquisition module 1, video acquisition module 2, ..., video acquisition module N) of the image enhancement device. The display ports of the X-ray security inspection machine and the video acquisition modules of the image enhancement device can be connected using standard interfaces such as VGA / DVI / HDMI.

[0102] Based on the connection between each display port and the corresponding video acquisition module, the video acquisition module of the image enhancement device can receive video signals from the X-ray security inspection machine in real time, convert the received video signals into digital video data, and thus obtain the security inspection display video.

[0103] The image enhancement device executes the image enhancement method for X-ray security inspection machines described in the embodiments above on the security inspection display video, and can output a target enhanced image. Subsequently, the target enhanced image is sent to various terminals for display, including but not limited to three methods: 1) looping back to the display interface of the X-ray security inspection machine through the video output port of the image enhancement device; 2) directly outputting to other display terminal devices used to display high-definition enhanced images; 3) transmitting to a cloud-based image interpretation terminal via a network. These three methods can be related by "and" or "or," and are not specifically limited here.

[0104] In some other embodiments, Figure 8 The diagram illustrates the overall process of the image enhancement method for X-ray security inspection machines provided in this embodiment of the invention.

[0105] like Figure 8 As shown, firstly, the security inspection display video output by the X-ray security inspection machine is acquired and converted into a sequence of security X-ray image frames. Then, a region of interest (ROI) map is extracted from each frame of the security X-ray image, and it is determined whether there are foreground objects in the ROI map. The ROI map containing foreground objects is striped into multiple striped images, and super-resolution processing is performed on these multiple striped images to obtain multiple super-resolution sub-images. Finally, the target enhancement image obtained by stitching together multiple super-resolution sub-images is sent to various terminals for display.

[0106] Corresponding to the image enhancement methods for X-ray security inspection machines described in the above embodiments, the present invention also proposes an image enhancement device for X-ray security inspection machines. Specifically, Figure 9 A schematic diagram of the structure of the image enhancement device for X-ray security inspection machines provided in an embodiment of the present invention is shown.

[0107] like Figure 9 As shown, the device includes: a security inspection display video acquisition module 910, used to acquire the security inspection display video output by the X-ray security inspection machine; a region of interest map extraction module 920, used to extract the region of interest map from the security X-ray image frame sequence corresponding to the security inspection display video; an image super-resolution processing module 930, used to stripe the region of interest map containing foreground objects into multiple striped images, and perform super-resolution processing on the multiple striped images to obtain multiple super-resolution sub-images; and a super-resolution sub-image stitching module 940, used to stitch the multiple super-resolution sub-images to obtain a target enhancement image.

[0108] In this embodiment, the security inspection display video acquired by the security inspection display video acquisition module 910 is acquired from the X-ray security inspection machine; the region of interest (ROI) extraction module 920 extracts the ROI from the security X-ray image frame sequence corresponding to the security inspection display video; the image super-resolution processing module 930 stripes the ROI containing foreground objects into multiple striped images, and performs super-resolution processing on the multiple striped images to obtain multiple super-resolution sub-images; the super-resolution sub-image stitching module 940 stitches the multiple super-resolution sub-images to obtain the target enhanced image. This device, by extracting the ROI from the security X-ray image and performing striping processing, can focus on key information areas and reduce the computational load of the super-resolution algorithm; by performing super-resolution processing on multiple striped images respectively, it achieves accurate enhancement of the ROI in the X-ray pseudo-color image output by older X-ray security inspection machines, significantly improving image quality without replacing the core hardware of the X-ray security inspection machine, and has network expansion capabilities.

[0109] It should be noted that the image enhancement device for X-ray security inspection machines provided in the embodiments of the present invention can be referred to in correspondence with the image enhancement methods for X-ray security inspection machines described in the above embodiments, and will not be repeated here.

[0110] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0111] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0112] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image enhancement method for X-ray security inspection machines, characterized in that, include: Acquire the security inspection display video output by the X-ray security inspection machine; Extract the region of interest map from the security X-ray image frame sequence corresponding to the security inspection display video; The region of interest map containing foreground objects is striped into multiple striped images, and super-resolution processing is performed on the multiple striped images to obtain multiple super-resolution sub-images. By stitching together the multiple super-resolution sub-images, a target-enhanced image is obtained; The step of striping the region of interest (ROI) image containing foreground objects into multiple striped images includes: acquiring the movement speed of the X-ray security inspection machine's conveyor belt; dynamically adjusting the segmentation width of the striped images according to the movement speed of the conveyor belt; and segmenting the ROI image into continuous striped images based on the segmentation width. The step of performing super-resolution processing on multiple striped images to obtain multiple super-resolution sub-images specifically includes: The multiple striped images obtained by striping the region of interest map containing foreground objects are denoted as the striped image set; For each striped image in the striped image set, the striped image is matched with the processed images in the initially empty super-resolution image set; If a processed image that matches the striped image exists in the set of super-resolution images, the processed image is used as the super-resolution processing result of the striped image; If no processed image matching the striped image exists in the super-resolution image set, super-resolution processing is performed on the striped image to obtain a super-resolution sub-image corresponding to the striped image, and the super-resolution sub-image is added to the super-resolution image set.

2. The image enhancement method for X-ray security inspection machines according to claim 1, characterized in that, Extracting the region of interest map from the security X-ray image frame sequence corresponding to the security inspection display video includes: For each security X-ray image, the operation area image in the security X-ray image is determined according to the color scheme and background color of the X-ray security inspection machine operation interface, and the operation area image is removed from the security X-ray image to obtain the initial region of interest map; The boundary position of the region of interest is determined based on the proportion of different pixel categories in each row and column of the initial region of interest map. The region of interest (ROI) map is extracted from the initial ROI map based on the boundary positions of the ROI map.

3. The image enhancement method for X-ray security inspection machines according to claim 2, characterized in that, Determining the boundary position of the region of interest (ROI) based on the proportion of different pixel categories in each row and column of the initial ROI map includes: The initial region of interest map is projected along the Y direction by column pixels to obtain multiple pixel columns in the Y direction; If the proportion of foreground pixels in the pixel column is lower than a set proportion threshold, the pixel column is determined as a background pixel column. If the pixel proportion of foreground pixels in the pixel column is greater than or equal to a set proportion threshold, the pixel column is determined as a foreground pixel column; The nearest and farthest foreground pixel columns in the X direction are determined as the X-direction boundary positions of the region of interest map; Wherein, the Y direction is perpendicular to the X direction, and the X direction is the path direction of the X-ray security inspection machine outputting the security inspection X-ray image.

4. The image enhancement method for X-ray security inspection machines according to claim 3, characterized in that, Determining the boundary position of the region of interest (ROI) based on the proportion of different pixel categories in each row and column of the initial ROI map includes: The initial region of interest map is projected along the X direction by row pixels to obtain multiple pixel rows in the X direction; If the proportion of foreground pixels in the pixel row is lower than a set proportion threshold, the pixel row is determined as a background pixel row. If the pixel proportion of foreground pixels in the pixel row is greater than or equal to a set proportion threshold, the pixel row is determined as a foreground pixel row. The nearest and farthest foreground pixel rows in the Y direction are determined as the Y-direction boundary positions of the region of interest map.

5. The image enhancement method for X-ray security inspection machines according to claim 1, characterized in that, The step of striping the region of interest map containing foreground objects into multiple striped images includes: Based on a pre-trained image classification network, the region of interest map is classified to obtain the image classification result; If the image classification result indicates that the region of interest (ROI) is a foreground object image, the ROI is retained, and the ROI is striped into multiple striped images. If the image classification result indicates that the region of interest (ROI) is a non-foreground object image, then the ROI is removed.

6. The image enhancement method for X-ray security inspection machines according to claim 1, characterized in that, The step of adding the super-resolution sub-image to the super-resolution image set includes: Remove the striped image that is furthest from the current time in the striped image set, and at the same time remove the super-resolution sub-image that is furthest from the current time in the super-resolution image set; The number of images in the striped image set is the same as and fixed as the number of images in the super-resolution image set.

7. The image enhancement method for X-ray security inspection machines according to claim 1, characterized in that, The step of performing super-resolution processing on the striped image to obtain a super-resolution sub-image corresponding to the striped image includes: The striped image is input into a pre-trained image enhancement model, which outputs a super-resolution sub-image corresponding to the striped image. The image enhancement model is constructed based on a generative adversarial network. It is obtained by adversarial training on image samples composed of high-resolution image samples and corresponding low-resolution image samples, and fine-tuning them based on security X-ray image samples. The high-resolution image samples have a first resolution, and the low-resolution image samples have a second resolution. The first resolution is much higher than the second resolution.

8. The image enhancement method for X-ray security inspection machines according to claim 1, characterized in that, The acquisition of the security inspection display video output by the X-ray security inspection machine includes: The security inspection display video output by the X-ray security inspection machine is acquired through a video acquisition module connected to the display port of the X-ray security inspection machine. The security inspection display video includes security X-ray images that are continuously scrolled along the X-axis, where the X-axis is the path of the security X-ray images output by the X-ray security inspection machine.

9. An image enhancement device for X-ray security inspection machines, characterized in that, include: The security inspection display video acquisition module is used to acquire the security inspection display video output by the X-ray security inspection machine; The region of interest (ROI) extraction module is used to extract the ROI from the security X-ray image frame sequence corresponding to the security inspection display video. The image super-resolution processing module is used to stripe the region of interest map containing foreground objects into multiple striped images, and to perform super-resolution processing on the multiple striped images to obtain multiple super-resolution sub-images. A super-resolution sub-image stitching module is used to stitch together the multiple super-resolution sub-images to obtain a target-enhanced image; The step of striping the region of interest (ROI) image containing foreground objects into multiple striped images includes: acquiring the movement speed of the X-ray security inspection machine's conveyor belt; dynamically adjusting the segmentation width of the striped images according to the movement speed of the conveyor belt; and segmenting the ROI image into continuous striped images based on the segmentation width. The step of performing super-resolution processing on multiple striped images to obtain multiple super-resolution sub-images specifically includes: The multiple striped images obtained by striping the region of interest map containing foreground objects are denoted as the striped image set; For each striped image in the striped image set, the striped image is matched with the processed images in the initially empty super-resolution image set; If a processed image that matches the striped image exists in the set of super-resolution images, the processed image is used as the super-resolution processing result of the striped image; If no processed image matching the striped image exists in the super-resolution image set, super-resolution processing is performed on the striped image to obtain a super-resolution sub-image corresponding to the striped image, and the super-resolution sub-image is added to the super-resolution image set.

Images