A security X-ray image augmentation method, system, device and medium

Abstract

The application relates to a security X-ray image augmentation method, system, device and medium. The method estimates high-energy data and low-energy data of a contraband false color image inserted into a background false color image through a bottom-layer physical model of a security inspection machine, inserts the contraband false color image into a specified target area of the background false color image, then completes mapping of the high-energy data and the low-energy data to the false color image, generates a false color image with specific contraband inserted at a specified position, realizes TIP data enhancement based on the high-energy data and the low-energy data, can quickly generate a large amount of training data with labeled information, and achieves the purposes of directional data amplification and data enhancement.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of X-ray security inspection, in particular to a security inspection X-ray image augmentation method, system, device and medium. BACKGROUND

[0002] In the security inspection image target detection task, the long-tail distribution phenomenon of the data set has always been an unavoidable problem. Because most of the prohibited sample categories occupy a large proportion of the data set, the model is prone to overfitting to these high-frequency categories during the training process, and the recognition accuracy of the minority prohibited sample categories is significantly reduced. The imbalance of data distribution makes the detection model show obvious category bias in overall performance, especially in the actual security inspection scene, the recognition ability of rare prohibited samples is insufficient. In the prior art, the data augmentation of the minority prohibited sample categories faces multiple challenges: on the one hand, the features of different types of prohibited samples are highly coupled in the X-ray image, and it is difficult to effectively enhance specific types of prohibited samples through simple image processing methods; on the other hand, if artificial collection is used to obtain augmented data, not only professional security inspection equipment is needed, but also high labor cost and time cost will be generated in data collection, labeling and other aspects. In addition, traditional data augmentation methods are difficult to maintain the unique physical characteristics of X-ray images, such as high-energy attenuation characteristics and pseudo-color space conversion relationships, resulting in differences in physical properties between the generated augmented data and the real data, which affects the model training effect. SUMMARY

[0003] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0004] The main purpose of the embodiments of the present disclosure is to propose a security inspection X-ray image augmentation method, system, device and medium, which can quickly generate a large amount of training data with labeled information, and achieve the purpose of directional amplification of data and enhancement of data.

[0005] In a first aspect, the present application provides a security inspection X-ray image augmentation method, the method comprising:

[0006] obtaining a first X-ray pseudo-color image of a security inspection area and a second X-ray pseudo-color image of a prohibited sample, and determining a target region in the first X-ray pseudo-color image;

[0007] determine a first high-low energy image of the first X-ray pseudo-color image and a second high-low energy image of the second X-ray pseudo-color image, and in the case of aligning a center point of the second X-ray pseudo-color image with a center point of the target region, calculate a high energy image and a low energy image after inserting the second X-ray pseudo-color image into the target region according to the first high-low energy image and the second high-low energy image, and form a high-low energy ratio image according to a ratio of corresponding pixel points in the high energy image and the low energy image;

[0008] determine an RGB channel value of each pixel point after inserting the second X-ray pseudo-color image into the target region according to the high energy image, the low energy image and the high-low energy ratio image, and integrate the RGB channel values of all pixel points to obtain a third X-ray pseudo-color image.

[0009] The augmentation method for security X-ray images provided by the embodiment has at least the following beneficial effects:

[0010] The method estimates high-low energy data after inserting the contraband pseudo-color image into the background pseudo-color image through a bottom physical model of imaging of the security inspection machine, inserts the contraband pseudo-color image into the background pseudo-color image in a specified target region, then completes mapping of the high-low energy data to the pseudo-color image, generates a pseudo-color image with specific contraband inserted in a specified position, realizes TIP data enhancement based on the high-low energy data, can quickly generate a large amount of training data with labeled information, and achieves the purposes of directional data amplification and data enhancement.

[0011] In a second aspect, the application provides an augmentation system for security X-ray images, and the system comprises:

[0012] An image acquisition module is configured to acquire a first X-ray pseudo-color image of a security inspection region and a second X-ray pseudo-color image of contraband, and determine a target region in the first X-ray pseudo-color image.

[0013] A high-low energy image generation module is configured to determine a first high-low energy image of the first X-ray pseudo-color image and a second high-low energy image of the second X-ray pseudo-color image, and in the case of aligning a center point of the second X-ray pseudo-color image with a center point of the target region, calculate a high energy image and a low energy image after inserting the second X-ray pseudo-color image into the target region according to the first high-low energy image and the second high-low energy image, and form a high-low energy ratio image according to a ratio of corresponding pixel points in the high energy image and the low energy image.

[0014] A channel value calculation and integration module is configured to determine an RGB channel value of each pixel point after inserting the second X-ray pseudo-color image into the target region according to the high energy image, the low energy image and the high-low energy ratio image, and integrate the RGB channel values of all pixel points to obtain a third X-ray pseudo-color image.

[0015] In a third aspect, the present application provides an electronic device, comprising at least one controller and a memory connected to the controller; the memory stores instructions executable by the at least one controller, and the instructions are executed by the at least one controller to cause the at least one controller to perform the augmented method of security X-ray image as described above.

[0016] In a fourth aspect, the present application provides a computer-readable storage medium, which stores computer-executable instructions for causing a computer to perform the augmented method of security X-ray image as described above.

[0017] Additional aspects and advantages of the present application will be made apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or related technical description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0019] Figure 1 is a flow diagram of an embodiment of the augmented method of security X-ray image provided by the present application;

[0020] Figure 2 is a structural diagram of an embodiment of the augmented system of security X-ray image provided by the present application;

[0021] Figure 3 is a structural diagram of an embodiment of the electronic device provided by the present application. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solutions and advantages of the present application more clear, the following will further describe the present application in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0023] In the description of the present application, if there is a description of first, second, etc., it is only used to distinguish technical features for the purpose, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features or implicitly indicating the sequence of indicated technical features.

[0024] In the description of the present application, it should be understood that the orientation description, such as the orientation or position relationship indicated by up, down, etc. is based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element referred to must have a particular orientation, be constructed and function in a particular orientation, and therefore cannot be understood as a limitation of the present application.

[0025] As shown in Figure 1 An embodiment of the present application provides an augmented method of security X-ray image, the method comprising:

[0026] Step S110, acquiring a first X-ray pseudo-color image of a security area and a second X-ray pseudo-color image of contraband, and determining a target area in the first X-ray pseudo-color image;

[0027] In step S110, the security area here refers to an area where package security can be performed, for example, a security channel in an airport, subway, etc. scene where a security machine detects contraband in a package.

[0028] The first X-ray pseudo-color image (color RGB image) is an image collected by a security machine in a security area, and the first X-ray pseudo-color image can include a package containing no contraband. The second X-ray pseudo-color image is an X pseudo-color image of contraband.

[0029] The embodiment needs to insert the second X-ray pseudo-color image into the specified target area in the first X-ray pseudo-color image.

[0030] In step S110, the target area is a region selected from the first X-ray pseudo-color image, which is an insertion area for inserting the second X-ray pseudo-color image into the first X-ray pseudo-color image.

[0031] Step S120, determining a first high-low energy image of the first X-ray pseudo-color image and a second high-low energy image of the second X-ray pseudo-color image, and in the case of aligning the center point of the second X-ray pseudo-color image with the center point of the target area, calculating the high energy image and the low energy image after inserting the second X-ray pseudo-color image into the target area according to the first high-low energy image and the second high-low energy image, and forming a high-low energy ratio image according to the ratio of corresponding pixels in the high energy image and the low energy image;

[0032] In step S120, in a dual-energy X-ray security system, the dual-energy X-ray security system alternately emits or simultaneously detects X-rays of two different energy levels in a very short time:

[0033] Low-energy X-rays: lower energy, longer wavelength. More easily absorbed and attenuated by matter. High-energy X-rays: higher energy, shorter wavelength. Stronger penetration, not easy to be absorbed by matter.

[0034] The dual-energy X-rays are collected respectively, and the high-energy image and the low-energy image obtained are subjected to mathematical operation and fusion to obtain a pseudo-color image. Taking the first X-ray pseudo-color image and the first high-low energy image as an example, the first X-ray pseudo-color image can be subjected to inverse operation to obtain the first high-low energy image.

[0035] In step S120, the center point of the second X-ray pseudo-color image is determined, and then the center point of the target region is determined, and then the center point of the second X-ray pseudo-color image is aligned with the center point of the target region, and then according to the first high-low energy image corresponding to the first X-ray pseudo-color image and the second high-low energy image corresponding to the second X-ray pseudo-color image, the high-energy image and the low-energy image after the second X-ray pseudo-color image is inserted into the target region are calculated, and it should be noted that the image obtained after the second X-ray pseudo-color image is inserted into the target region is not the final X-ray pseudo-color image (color RGB image), and the high-energy image and the low-energy image here are high-low energy images obtained by operation based on the first high-low energy image and the second high-low energy image. Finally, the high-low energy value in the pixel is calculated based on the high-energy image and the low-energy image to obtain a high-low energy ratio image.

[0036] In step S130, according to the high-energy image, the low-energy image and the high-low energy ratio image, the RGB channel value of each pixel point after the second X-ray pseudo-color image is inserted into the target region is determined, and the RGB channel values of all pixel points are integrated to obtain a third X-ray pseudo-color image after the second X-ray pseudo-color image is inserted into the target region.

[0037] In order to solve the above problems, based on the essential characteristics of X-ray images which are derived from the physical law of differential absorption of high and low energy rays by matter, an enhanced idea of realizing physical property fusion in the target region by decomposing the attenuation characteristics of the original image (i.e. the first X-ray pseudo-color image) and the contraband image (i.e. the second X-ray pseudo-color image) is proposed. Specifically, first, the insertable region of the first X-ray pseudo-color image is determined, the second X-ray pseudo-color image is aligned with the center point of the target region, and then the high-low energy images of the first X-ray pseudo-color image and the second X-ray pseudo-color image are determined respectively, and then the new high-low energy image can be generated by superimposing the attenuation coefficients of the two, and the high-low energy ratio image is formed by calculating the ratio; for each pixel point after the second X-ray pseudo-color image is aligned with the center point of the target region, the high-low energy data and the ratio information are converted into a three-dimensional color vector, and finally the color data of all pixel points are integrated to generate an enhanced image with consistent physical properties.

[0038] Compared with the prior art, the embodiment estimates high and low energy data after the insertion of the prohibited item through the bottom layer physical model of the security inspection machine imaging, inserts the prohibited item into the specified target area, then completes the mapping of the high and low energy data to the pseudo-color image, generates the pseudo-color image of inserting the specific prohibited item in the specified position, and expands the model training set; the method realizes the TIP (Threat Image Projection) data enhancement based on the high and low energy data, can quickly generate a large amount of training data with labeled information, and achieves the purpose of directional data amplification and data enhancement.

[0039] In some embodiments of the present application, the determination of the target region in the first X-ray pseudo-color image in step S110 comprises:

[0040] Step S1111, according to the RGB channel value, the first X-ray pseudo-color image is divided into a plurality of candidate regions; any candidate region represents a different material attribute in the first X-ray pseudo-color image;

[0041] Wherein, the candidate region refers to the image sub-region divided by the RGB channel value difference, which can be realized by using clustering algorithm or threshold segmentation algorithm, for example, similar color distribution pixels are classified into the same candidate region based on K-means clustering. The material attribute refers to the color feature difference of different materials in X-ray imaging, for example, the RGB value distribution of metal, plastic or liquid is different.

[0042] Step S1112, select one candidate region as the target region from the plurality of candidate regions;

[0043] Here, the candidate region can be specified, for example, it is desired to insert the prohibited item into the metal shield (X-ray imaging is blue), then the region corresponding to the metal shield is selected.

[0044] Or,

[0045] Step S1121, determine a plurality of detection boxes in the first X-ray pseudo-color image based on a neural network model;

[0046] Here, the neural network model refers to a deep learning model for target detection, which can be realized by using algorithms such as Faster R-CNN or YOLO, and the detection box containing the potential object is automatically output by the trained model. The detection box refers to the rectangular region predicted by the model, and the coordinates and size thereof are obtained by regression calculation, for example, the anchor box mechanism is used to generate the candidate box, and the final result is screened by non-maximum suppression.

[0047] Step S1122, select the region in one detection box as the target region from the plurality of detection boxes.

[0048] Specifically, in the first embodiment, RGB channel values are used to distinguish different material regions. For example, by analyzing the histogram distribution of the RGB three channels, pixels with similar colors are classified into the same candidate region, and each candidate region represents a possible material type. Then, the target region is selected from the candidate regions according to the preset rules (such as area size, shape regularity) or manual interaction.

[0049] In the second embodiment, a pre-trained neural network model is used to extract features from the input image, generate multiple detection boxes covering potential objects, and select the target detection box as the final selected target region by sorting and filtering the initial boxes generated by sliding windows or region proposal networks according to classification scores.

[0050] Compared with the prior art, by dividing the candidate region according to the material attribute or automatically generating the detection box by the neural network model, the potential insertion position can be located more accurately, and the manual intervention can be reduced. By dividing the candidate region or generating the detection box, the region suitable for inserting the contraband can be quickly located in the complex X-ray image, such as avoiding selecting a position with cluttered background or incompatible material.

[0051] In some embodiments of the present application, after determining the first high-low energy image of the first X-ray false color image and the second high-low energy image of the second X-ray false color image in step S120, the method further comprises:

[0052] In step S210, a first high-low energy attenuation coefficient of the first high-low energy image is calculated; the first high-low energy attenuation coefficient includes a first high energy attenuation coefficient; the first high-low energy attenuation coefficient refers to the attenuation degree of X-rays penetrating an object in a high energy spectrum.

[0053] In step S220, a second high-low energy attenuation coefficient of the second high-low energy image is calculated; the second high-low energy attenuation coefficient includes a second high energy attenuation coefficient; the second high-low energy attenuation coefficient refers to the attenuation characteristics of the contraband material in the high energy spectrum.

[0054] In step S120, the high energy image after inserting the second X-ray false color image into the target region is calculated according to the first high-low energy image and the second high-low energy image, comprising:

[0055] In step S1211, a first logarithm of the first high energy attenuation coefficient with base e is calculated; a second logarithm of the second high energy attenuation coefficient with base e is calculated; and a high energy value of a pixel point is calculated with base e, with the negative sum of the first logarithm and the second logarithm as the exponent;

[0056] In step S1212, the high energy values of all pixel points are integrated to form a high energy image.

[0057] In the calculation of the high-energy image, first, the high-energy attenuation coefficients in the first high-low energy image corresponding to the first X-ray pseudo-color image and the second high-low energy image corresponding to the second X-ray pseudo-color image are extracted respectively. For each pixel point to be inserted, the high-energy attenuation coefficient in the first high-low energy image corresponding to the first X-ray pseudo-color image is logarithmically converted with the high-energy attenuation coefficient in the second high-low energy image corresponding to the second X-ray pseudo-color image, and then the sum of the two is taken as the negative number as the exponential parameter. Through the exponential function operation with the natural constant e as the base, the actual high-energy value of the pixel point after the contraband is superimposed is obtained. After all the pixel points are calculated according to this method, a complete high-energy image is formed.

[0058] Compared with the prior art, the method establishes an attenuation coefficient calculation system based on a physical model, ensures the authenticity of the X-ray image after the contraband is inserted in terms of energy distribution characteristics, and thus improves the accuracy of the pseudo-color image generated subsequently in terms of material characterization. This method avoids the distortion of physical properties caused by the traditional image superposition technology, and provides enhanced data conforming to the actual imaging principle for training the detection model.

[0059] In some embodiments of the present application, the first high-low energy attenuation coefficient further includes: a first low-energy attenuation coefficient; and the second high-low energy attenuation coefficient further includes: a second low-energy attenuation coefficient.

[0060] The step S120 of calculating the low-energy image after the second X-ray pseudo-color image is inserted into the target region according to the first high-low energy image and the second high-low energy image includes:

[0061] The step S1221 calculates, for any pixel point after the second X-ray pseudo-color image is inserted into the target region: a third logarithm of the first low-energy attenuation coefficient with e as the base; a fourth logarithm of the second low-energy attenuation coefficient with e as the base; and a low-energy value of the pixel point with e as the base and the negative number of the sum of the third logarithm and the fourth logarithm as the exponential.

[0062] The step S1222 integrates the low-energy values of all the pixel points to form a low-energy image.

[0063] Here, the steps are the same as described above, and will not be repeated here.

[0064] In some embodiments of the present application, the high-low energy ratio image formed according to the ratio of the corresponding pixels in the high-energy image and the low-energy image in the step S120 includes:

[0065] The step S1231 calculates, for any pixel point after the second X-ray pseudo-color image is inserted into the target region: a ratio between the high-energy value and the low-energy value of the pixel point.

[0066] Wherein, the ratio refers to the proportional relationship between the high energy value and the low energy value of the same pixel point, which can be directly calculated by division operation, and is used to reflect the material property difference of the object under different energy rays.

[0067] Step S1232, integrate the ratios of all pixel points to form a high-low energy ratio image.

[0068] Specifically, after generating the high-energy image and the low-energy image, the high-energy value and the low-energy value of each pixel point are calculated by ratio. For example, for a certain pixel point in the target region, if the high-energy value is 0.8 and the low-energy value is 0.2, the ratio calculation result is 4. The ratios of all pixel points are integrated into an independent image, which reflects the response difference of different materials to high-low energy rays, and provides key feature information for the calculation of RGB channel values in the subsequent process.

[0069] Compared with the prior art, the embodiment can generate a high-low energy ratio image that is more consistent with the real X-ray imaging rule, effectively enhance the material consistency between the synthesized second X-ray pseudo-color image and the first X-ray pseudo-color image, and further improve the long-tail distribution problem of the training data and improve the recognition accuracy of the detection model for a small number of prohibited items.

[0070] In some embodiments of the present application, the step S130 of determining the RGB channel value of each pixel point after the second X-ray pseudo-color image is inserted into the target region according to the high-energy image, the low-energy image and the high-low energy ratio image comprises:

[0071] Step S1311, determining any pixel point and a plurality of neighborhood pixel points in the neighborhood of the pixel point after the second X-ray pseudo-color image is inserted into the target region;

[0072] Wherein, the neighborhood pixel point refers to the adjacent pixel points within a certain range extending outward from the current pixel point, and a 3x3 or 5x5 sliding window can be used for neighborhood selection. By counting the physical property distribution characteristics of the pixels in the neighborhood, the accuracy of color restoration can be effectively improved.

[0073] Step S1312, flattening the ratio between the high-energy value and the low-energy value corresponding to the pixel point, the ratio between the high-energy value and the low-energy value, and the ratio between the high-energy value and the low-energy value corresponding to the plurality of neighborhood pixel points into a one-dimensional vector;

[0074] Wherein, flattening into a one-dimensional vector refers to arranging multi-dimensional data into a single-dimensional array in order, which can be realized by concatenating the high-energy value, the low-energy value and the ratio of each pixel in the neighborhood. This data organization form facilitates the subsequent neural network model to learn the local region features.

[0075] Step S1313, determining the RGB channel value of the pixel point according to the one-dimensional vector.

[0076] Specifically, in the calculation of the RGB channel value of the target region after the insertion of the contraband, first, a neighborhood window is established with the current pixel point as the center, and the high-energy value, low-energy value and ratio between the high and low energy of all pixel points in the window are extracted. After these values are arranged in spatial order and spliced into a continuous one-dimensional vector, they are input into the pre-trained color mapping model for nonlinear transformation, and finally the values of the R, G and B channels of the pixel point are output.

[0077] Compared with the prior art, by introducing a set of physical parameters of neighborhood pixels, a one-dimensional vector containing local region features is constructed, so that the color mapping process can comprehensively consider the continuity characteristics of the material distribution, thereby significantly improving the visual authenticity and physical consistency of the synthesized image.

[0078] In some embodiments of the present application, before determining the RGB channel value of each pixel point after the insertion of the second X-ray pseudo-color map into the target region according to the high-energy image, the low-energy image and the high-low energy ratio image in step S130, the method further comprises:

[0079] Step S310, determining the low-energy value of the pixel point in the first X-ray pseudo-color map and the low-energy value of the pixel point in the second X-ray pseudo-color map;

[0080] Step S320, correcting the low-energy value of the corresponding pixel point after the insertion of the second X-ray pseudo-color map into the target region according to the low-energy value of the pixel point in the first X-ray pseudo-color map and the low-energy value of the corresponding pixel point in the second X-ray pseudo-color map.

[0081] The correction operation refers to numerical adjustment of the low-energy value after insertion, which can be realized by weighted average method or fusion algorithm based on physical attenuation model, for example, the low-energy value of the original image and the low-energy value of the contraband are superimposed in proportion, so that the fused low-energy value not only retains the physical characteristics of the original image background, but also reflects the material properties of the contraband.

[0082] Specifically, when the second X-ray pseudo-color map is inserted into the target region, there may be a difference in energy intensity between the low-energy values of the first X-ray pseudo-color map and the second X-ray pseudo-color map. Here, by respectively extracting the low-energy values in the first high-low energy image corresponding to the first X-ray pseudo-color map and the second high-low energy image corresponding to the second X-ray pseudo-color map, local correction is performed on each pixel point after insertion. Further, for the pixel points in the overlapping region, the fusion weight can be dynamically adjusted based on the low-energy attenuation coefficients of the two, so that the corrected low-energy value conforms to the physical law of X-ray imaging.

[0083] Compared with existing technologies, this solution introduces a low-energy value correction mechanism to couple the calculation of the low-energy attenuation characteristics of the original image and the contraband, so that the inserted low-energy image is closer to the real imaging effect in terms of energy intensity and boundary transition. This can effectively eliminate the image fusion distortion problem caused by low-energy value mismatch and improve the physical rationality of the synthesized X-ray pseudo-color image.

[0084] To facilitate understanding by those skilled in the art, a method for augmenting security X-ray images is provided, comprising the following steps:

[0085] Step S911: For the first X-ray pseudo-color image (background image) of the case area, cluster it according to the RGB channel values ​​and divide it into four regions: blank region (number 0), orange region (number 1), blue region (number 2) and green region (number 3); each region represents a different material property.

[0086] Step S912: For the first X-ray pseudo-color image, use the trained AI recognition model to perform inference and generate detection box information stored in a JSON file, where each detection box contains a category label and position and size information box.

[0087] Step S913 provides the following two insertion modes.

[0088] Insert mode 1: Manually enter the background image number (0~3), select the area corresponding to the number as the target area, and randomly select points from the coordinate set of the target area. The center point of the second X-ray pseudo-color image (foreground image) inserted as a contraband;

[0089] For insertion mode 2, the category of the detection boxes in the input image is determined, and the coordinates of the points within the box region of that detection box are randomly generated. , as the center point for inserting the second X-ray pseudo-color image;

[0090] Step S914: For the second high- and low-energy image corresponding to the second X-ray pseudo-color image (foreground image), calculate the high- and low-energy attenuation coefficients, i.e., the high-energy attenuation coefficient. and low energy decay coefficient ;

[0091] Step S915: For the first high- and low-energy image corresponding to the first X-ray pseudo-color image (background image), calculate the high- and low-energy attenuation coefficients, i.e., the high-energy attenuation coefficient. and low energy decay coefficient ;

[0092] Step S916, the background image Align the coordinate points with the center of the foreground image, overlay high and low energy attenuation coefficients, and calculate the high and low energies (i.e., the high energy value) after the contraband is inserted. and low energy value :

[0093] ;

[0094] ;

[0095] wherein, is the air value, i.e. the intensity of the X-ray hitting the sensor after penetrating the air, which is usually the maximum value that the sensor can represent.

[0096] Step S917, according to the low energy value of the background image and the low energy value of the foreground image, correct the low energy value of the foreground image after being inserted into the background image by referring to a correction table to remove the effect of spectral hardening ;

[0097] Step S918, calculate the ratio of high energy and low energy ; ;

[0098] Step S919, normalize the high energy, low energy and the ratio to remove the effect of dimension;

[0099] Step S920, for the corrected high energy, low energy and the ratio, find the high energy and low energy of the eight-neighbor pixels of each pixel , and flatten them into a one-dimensional vector in order ; ;

[0100] Step S921, input into the neural network to obtain the RGB channel value of the corresponding pixel ;

[0101] Step S922, integrate all to save as a new pseudo-color image, i.e. the third X-ray pseudo-color image after the foreground image is inserted into the background image;

[0102] In some embodiments, :

[0103] ;

[0104] The neural network can be a KAN network, in which, for the first neuron of the next layer, its value is not obtained by weighted summation (as in a multi-layer perception MLP), but is calculated in the following way:

[0105] ;

[0106] wherein,​ is the learnable activation function on the edge connecting the th input neuron to the th output neuron. It can be understood that each input feature is first transformed by a dedicated, tailor-made small function, and then all the transformed results are added to obtain the value of the next layer of neurons.

[0107] The input here is a 27-dimensional variable, and the output of the neural network model KAN obtained by pre-training is an RGB false color map.

[0108] Step S923, repeat the above operation to generate a false color map with a specific prohibited item inserted at a specified position, and expand the model training set. The training set here is used for subsequent model training.

[0109] The method provided by the embodiment has the following beneficial effects:

[0110] The method designs an augmentation method for security X-ray images. The method uses two different insertion coordinate selection methods, estimates the high and low energy data after the insertion of prohibited items through the bottom physical model of the security machine imaging, inserts the prohibited items into the specified area, and then completes the mapping of the high and low energy data to the false color map through the deep learning method, realizes the TIP function, so that a large amount of training data with labeled information can be quickly generated, and the purpose of directional amplification of certain data is achieved.

[0111] As Figure 2 , one embodiment of the application provides an augmentation system for security X-ray images, which comprises:

[0112] The image acquisition module 1100 is configured to acquire a first X-ray false color map of a security area and a second X-ray false color map of a prohibited item, and determine a target area in the first X-ray false color map;

[0113] The high and low energy image generation module 1200 is configured to determine a first high and low energy image of the first X-ray false color map and a second high and low energy image of the second X-ray false color map, and in the case of aligning the center point of the second X-ray false color map with the center point of the target area, calculate the high energy image and the low energy image after inserting the second X-ray false color map into the target area according to the first high and low energy image and the second high and low energy image, and form a high and low energy ratio image according to the ratio of corresponding pixels in the high energy image and the low energy image;

[0114] The channel value calculation and integration module 1300 is configured to determine the RGB channel value of each pixel point after inserting the second X-ray false color map into the target area according to the high energy image, the low energy image and the high and low energy ratio image, and integrate the RGB channel values of all pixel points to obtain a third X-ray false color map after inserting the second X-ray false color map into the target area.

[0115] It should be noted that the augmented security X-ray image system provided by the embodiment and the augmented security X-ray image method described above are based on the same inventive concept, and therefore the content of the augmented security X-ray image method described above is also applicable to the content of the augmented security X-ray image system of the embodiment, which will not be described here.

[0116] As Figure 3 One embodiment of the present application further provides an electronic device, which comprises a memory and a processor. The memory stores a computer program, and the processor implements the augmented security X-ray image method described above when executing the computer program. The electronic device comprises:

[0117] at least one battery;

[0118] at least one memory;

[0119] at least one processor;

[0120] at least one program;

[0121] The program is stored in the memory, and the processor executes the at least one program to implement the augmented security X-ray image method described above.

[0122] The electronic device can be any intelligent terminal, such as a mobile phone, a tablet computer, a personal digital assistant (PDA), a vehicle-mounted computer, etc.

[0123] The electronic device of the embodiment of the present application will be described in detail below.

[0124] The processor 1600 can be implemented in the form of a general central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute related programs to implement the technical solutions provided by the embodiments of the present application.

[0125] The memory 1700 can be implemented in the form of a read only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1700 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present application are implemented by software or firmware, the related program codes are saved in the memory 1700 and are called and executed by the processor 1600 to implement the augmented security X-ray image method of the embodiments of the present application.

[0126] The input / output interface 1800 is configured to realize information input and output.

[0127] The communication interface 1900 is configured to realize communication interaction between the device and other devices, and the communication can be realized through a wired manner (for example, a USB, a network cable, and the like) or a wireless manner (for example, a mobile network, WIFI, Bluetooth, and the like).

[0128] The bus 2000 is configured to transmit information between various components (for example, the processor 1600, the memory 1700, the input / output interface 1800, and the communication interface 1900) of the device.

[0129] The processor 1600, the memory 1700, the input / output interface 1800, and the communication interface 1900 are connected to each other through the bus 2000 to realize communication connection between the device.

[0130] The disclosure also provides a storage medium, which is a computer readable storage medium, and stores computer executable instructions for causing a computer to execute the security inspection X-ray image augmentation method.

[0131] The memory is a non-transitory computer readable storage medium, and can be used to store a non-transitory software program and a non-transitory computer executable program. In addition, the memory can include a high-speed random access memory, and can also include a non-transitory memory, for example, at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged relative to the processor, and the remote memory can be connected to the processor through a network. Examples of the network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0132] The embodiments described in the disclosure are used to more clearly illustrate the technical solutions of the disclosure, and do not constitute a limitation on the technical solutions provided by the disclosure. It can be understood by those skilled in the art that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the disclosure are also applicable to similar technical problems.

[0133] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation on the disclosure, and can include more or fewer steps than those shown in the figures, or combine certain steps or different steps.

[0134] The apparatus embodiments described above are merely exemplary, and the units described as separate units can or can not be physically separate, i.e., can be located in one place, or can be distributed over multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purposes of the embodiments.

[0135] Those skilled in the art can understand that all or some of the steps in the method disclosed above, the functional modules / units in the system and the device can be implemented as software, firmware, hardware and appropriate combinations thereof.

[0136] The terms "first", "second", "third", "fourth" and the like in the description of the application and in the claims of the foregoing drawings, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of the terms so

[0137] It should be understood that in this application, "at least one" means one or more, and "multiple" means two or more. "And / or" is used to describe the relationship between the associated objects, which means that there can be three relationships, for example, "A and / or B" can mean that there are three cases: only A, only B, and A and B at the same time, where A and B can be singular or plural. The character " / " generally represents that the associated objects before and after are in an "or" relationship. "At least one of the following" or the like means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c, can mean a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0138] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented by other manners. For example, the apparatus embodiments described above are merely illustrative, for example, the division of units is merely a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, apparatuses or units, and can be electrical, mechanical or other forms.

[0139] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place or can be distributed to a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0140] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0141] If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part of the prior art that contributes to the technical solutions or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing an electronic device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program storage media.

[0142] The above is a specific description of the preferred embodiments of the present application, but the embodiments of the present application are not limited to the above described embodiments. Those skilled in the art can make various equivalent modifications or replacements without departing from the spirit of the embodiments of the present application, and these equivalent modifications or replacements are all included in the scope defined by the claims of the embodiments of the present application.

Claims

1. An augmented method of security x-ray images, characterized in that, The method comprises: obtaining a first X-ray pseudo-color image of a security check area and a second X-ray pseudo-color image of contraband, and determining a target region in the first X-ray pseudo-color image; determining a first high-low energy image of the first X-ray pseudo-color image and a second high-low energy image of the second X-ray pseudo-color image, calculating a first high-low energy attenuation coefficient of the first high-low energy image; the first high-low energy attenuation coefficient comprises: a first high-energy attenuation coefficient; calculating a second high-low energy attenuation coefficient of the second high-low energy image; the second high-low energy attenuation coefficient comprises: a second high-energy attenuation coefficient; and in the case of aligning the center point of the second X-ray pseudo-color image with the center point of the target region, calculating a high-energy image and a low-energy image after inserting the second X-ray pseudo-color image into the target region according to the first high-low energy image and the second high-low energy image, and a high-low energy ratio image formed according to the ratio of corresponding pixel points in the high-energy image and the low-energy image; calculating a high-energy image after inserting the second X-ray pseudo-color image into the target region according to the first high-low energy image and the second high-low energy image, comprising: calculating the first logarithm of the first high-energy attenuation coefficient with e as the base; calculating the second logarithm of the second high-energy attenuation coefficient with e as the base; and calculating the high-energy value of the pixel point with e as the base and the negative of the sum of the first logarithm and the second logarithm as the exponent; integrating the high-energy values of all pixel points to form the high-energy image; the first high-low energy attenuation coefficient further comprises: a first low-energy attenuation coefficient; the second high-low energy attenuation coefficient further comprises: a second low-energy attenuation coefficient; calculating a low-energy image after inserting the second X-ray pseudo-color image into the target region according to the first high-low energy image and the second high-low energy image, comprising: calculating the third logarithm of the first low-energy attenuation coefficient with e as the base; calculating the fourth logarithm of the second low-energy attenuation coefficient with e as the base; and calculating the low-energy value of the pixel point with e as the base and the negative of the sum of the third logarithm and the fourth logarithm as the exponent; integrating the low-energy values of all pixel points to form the low-energy image; determining the RGB channel value of each pixel point after inserting the second X-ray pseudo-color image into the target region according to the high-energy image, the low-energy image and the high-low energy ratio image, and integrating the RGB channel values of all pixel points to obtain a third X-ray pseudo-color image.

2. The augmented method of security X-ray images according to claim 1, wherein, The determination of the target region in the first X-ray pseudo-color image comprises: dividing the first X-ray pseudo-color image into a plurality of candidate regions according to the RGB channel value; any candidate region represents different material properties in the first X-ray pseudo-color image; selecting one of the candidate regions as the target region from the plurality of candidate regions; or, determining a plurality of detection boxes in the first X-ray pseudo-color image based on a neural network model; selecting a region in one of the detection boxes as the target region from the plurality of detection boxes.

3. The augmented method of security X-ray images according to claim 1, wherein, The high-low energy ratio image formed according to the ratio of corresponding pixel points in the high-energy image and the low-energy image comprises: calculating the ratio between the high-energy value and the low-energy value of the pixel point; The ratio of all pixel points is integrated to form the high-low energy ratio image.

4. The augmented method of security X-ray images according to claim 1, wherein, The method further comprises, before the step of determining the RGB channel value of each pixel point after the second X-ray pseudo-color image is inserted into the target region according to the high-energy image, the low-energy image and the high-low energy ratio image: determining the low-energy value of the pixel point in the first X-ray pseudo-color image and the low-energy value of the corresponding pixel point in the second X-ray pseudo-color image; correcting the low-energy value of the corresponding pixel point after the second X-ray pseudo-color image is inserted into the target region according to the low-energy value of the pixel point in the first X-ray pseudo-color image and the low-energy value of the corresponding pixel point in the second X-ray pseudo-color image. The system comprises:

5. The augmented method of security X-ray images according to claim 1, wherein, an image acquisition module configured to acquire a first X-ray pseudo-color image of a security inspection region and a second X-ray pseudo-color image of contraband, and determine a target region in the first X-ray pseudo-color image; a high-low energy image generation module configured to determine a first high-low energy image of the first X-ray pseudo-color image and a second high-low energy image of the second X-ray pseudo-color image, calculate a first high-low energy attenuation coefficient of the first high-low energy image, wherein the first high-low energy attenuation coefficient comprises a first high-energy attenuation coefficient, calculate a second high-low energy attenuation coefficient of the second high-low energy image, wherein the second high-low energy attenuation coefficient comprises a second high-energy attenuation coefficient, and in the case that a center point of the second X-ray pseudo-color image is aligned with a center point of the target region, calculate a high-energy image and a low-energy image after the second X-ray pseudo-color image is inserted into the target region according to the first high-low energy image and the second high-low energy image, and a high-low energy ratio image formed according to a ratio of corresponding pixel points in the high-energy image and the low-energy image; and calculate the high-energy image after the second X-ray pseudo-color image is inserted into the target region according to the first high-low energy image and the second high-low energy image, comprising: calculating a first logarithm of the first high-energy attenuation coefficient with e as the base, calculating a second logarithm of the second high-energy attenuation coefficient with e as the base, and calculating a high-energy value of a pixel point with e as the base and the negative of the sum of the first logarithm and the second logarithm as the exponent; 6. An augmented system of security x-ray images characterized in that, integrate the high-energy values of all pixel points to form the high-energy image; the first high-low energy attenuation coefficient further comprises a first low-energy attenuation coefficient, and the second high-low energy attenuation coefficient further comprises a second low-energy attenuation coefficient; calculate the low-energy image after the second X-ray pseudo-color image is inserted into the target region according to the first high-low energy image and the second high-low energy image, comprising: ​ ​ ​ ​ calculating a third logarithm with base e of the first low-energy attenuation coefficient; calculating a fourth logarithm with base e of the second low-energy attenuation coefficient; and calculating a low-energy value of a pixel point with base e and with an exponent of a negative of a sum of the third logarithm and the fourth logarithm; integrating the low-energy values of all pixel points to form the low-energy image; a channel value calculating and integrating module, configured to determine an RGB channel value of each pixel point after the second X-ray pseudo-color image is inserted into the target region according to the high-energy image, the low-energy image and the high-low energy ratio image, and integrate the RGB channel values of all pixel points to obtain a third X-ray pseudo-color image.

7. An electronic device, comprising: The security X-ray image augmentation method comprises at least one controller and a memory connected in communication with the controller; the memory stores instructions executable by the at least one controller, and the instructions are executed by the at least one controller to enable the at least one controller to perform the security X-ray image augmentation method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer readable storage medium stores computer executable instructions for causing a computer to perform the security X-ray image augmentation method according to any one of claims 1 to 5.

Images