X-ray security inspection method and device for detecting rotating dangerous goods based on appearance deformation correction

Abstract

This invention provides a method and apparatus for rotating hazardous materials detection in X-ray security inspections based on appearance deformation correction, belonging to the field of target object detection technology. The method includes acquiring an X-ray security image to be detected; training a rotation detection model based on appearance deformation correction; and, based on the X-ray security image to be detected, using the trained rotation detection model based on appearance deformation correction, obtaining the rotation detection results of all targets in the X-ray security image after appearance deformation correction. This invention effectively solves the problem of performance degradation in target detection tasks caused by the deformation characteristics of three-dimensional rotation.

Description

Technical Field

[0001] This invention belongs to the field of target object detection technology, and in particular relates to a method and device for rotating detection of dangerous goods in X-ray security inspection based on appearance deformation correction. Background Technology

[0002] As society continues to develop, public transportation faces increasing pressure, making security checks to ensure people's safety increasingly important. Currently, security checks are typically conducted manually, examining images generated by X-ray machines for prohibited items. However, manual checks suffer from high labor costs, frequent missed or false detections, and low efficiency. Therefore, it is necessary to utilize machines to replace manual security checks.

[0003] Because X-ray security images generated by X-ray security inspection machines have a serious problem of stereoscopic rotation, the visual appearance of the target that the existing model can extract is severely distorted, which exacerbates the problem of intra-class diversity and makes it impossible to correctly distinguish the target category. This leads to incorrect predictions and missed predictions, resulting in a decline in detection performance, which poses a challenge to the task of rotating target detection based on X-ray security images. Summary of the Invention

[0004] To address the aforementioned shortcomings in existing technologies, this invention provides a method and apparatus for rotating hazardous materials in X-ray security inspection based on appearance deformation correction. This invention extracts the three-dimensional semantic information of each target and corrects the original deformed appearance of the target, effectively solving the problem of performance degradation in target detection tasks caused by the deformation features of three-dimensional rotation. This allows the detector to more easily distinguish target categories and improves model detection performance.

[0005] Firstly, in order to achieve the above objectives, the technical solution adopted by this invention is: a method for detecting rotating hazardous materials in X-ray security inspection based on appearance deformation correction, comprising the following steps:

[0006] S1. Obtain the X-ray security inspection image to be detected;

[0007] S2. Train a rotation detection model based on appearance deformation correction, and based on the X-ray security inspection image to be inspected, use the trained rotation detection model based on appearance deformation correction to obtain the rotation detection results of all targets in the X-ray security inspection image after appearance deformation correction, thus completing the appearance deformation correction of dangerous goods for X-ray security inspection.

[0008] The beneficial effects of this invention are: it extracts the stereo semantic information of each target and corrects the original deformed appearance of the target, effectively solving the problem of performance degradation of target detection task caused by the deformation features of stereo rotation, making it easier for the detector to distinguish the target category and improving the model detection performance.

[0009] Further, step S2 includes the following steps:

[0010] S201. Obtain images containing contraband generated by an X-ray machine, construct an image dataset, and based on the constructed image dataset, use an X-ray converter to simulate the three-dimensional rotation information of the contraband and project it onto a real X-ray image to generate an extended X-ray dataset, which is then divided into a training set and a test set.

[0011] S202. Based on the X-ray images in the training set, each batch of X-ray images is input into the hazardous materials detection network to obtain image feature maps and candidate box sets.

[0012] S203. Input the image feature map and candidate box set into the stereo perception guidance module to obtain the stereo rotation semantic information of each type of hazardous product in this batch, and calculate the stereo perception loss.

[0013] S204. Input the image feature map, candidate box set and stereo rotation semantic information of each hazardous type into the progressive deformation calibration module to obtain the features of all targets in the X-ray image after visual appearance deformation correction.

[0014] S205. Input the corrected features into the rotation detection head to obtain the rotation detection results of all targets in the image after correction of their visual appearance deformation.

[0015] S206. Determine whether the number of iterations has been reached. If yes, obtain the rotation detection model based on appearance deformation correction and proceed to step S207. Otherwise, return to step S201.

[0016] S207. The accuracy of the rotation detection model based on appearance deformation correction is tested using the test set to obtain the trained rotation detection model based on appearance deformation correction.

[0017] S208. Based on the X-ray security inspection image to be inspected, the trained rotation detection model based on appearance deformation correction is used to obtain the rotation detection results of all targets in the X-ray security inspection image after appearance deformation correction, thus completing the appearance deformation correction of dangerous goods for X-ray security inspection.

[0018] The beneficial effects of the above-mentioned further scheme are: the process extracts the stereo semantic information of each target, and the original deformed appearance of the target is corrected, which effectively solves the problem of the performance degradation of the target detection task caused by the deformation features of stereo rotation, making it easier for the detector to distinguish the target category and improve the model detection performance.

[0019] Furthermore, step S201 includes the following steps:

[0020] S2011. The X-ray machine acquires the X-ray security inspection images to be inspected, and the items in each image are labeled according to the X-ray security inspection images to be inspected.

[0021] S2012. Based on the annotation results, select images containing prohibited items and construct an image dataset;

[0022] S2013. Based on the constructed image dataset, simulate the three-dimensional rotation information of the contraband, convert the rotating three-dimensional network into two-dimensional X-ray image blocks through an X-ray converter, and insert them into real X-ray images to generate an extended X-ray dataset.

[0023] S2014. Divide the generated extended X-ray dataset into a training set and a test set.

[0024] The beneficial effects of the above-mentioned further scheme are: this step uses the X-ray converter to generate more image data of contraband, expands the X-ray dataset, makes the dataset contain more semantic information, and increases the data scale for the detection model to learn.

[0025] Furthermore, step S203 includes the following steps:

[0026] S2031. Map the candidate box set onto the image feature map to obtain the feature set p of all candidate boxes;

[0027] S2032. Based on the feature set p of all candidate boxes, calculate the stereo rotation semantic information l of each target through the stereo perception guidance module. (i) ;

[0028] S2033, Based on the stereoscopic rotation semantic information of each target (i) Calculate the stereo perception loss.

[0029] The beneficial effect of the above-mentioned further scheme is that this step enables the model to fully learn the rich stereo semantic information in the simulated image information, and enables the network to have the ability to perceive stereo rotation.

[0030] Furthermore, the expression for the stereo perception loss is as follows:

[0031]

[0032] l (i) =max(f zy (p i ),f zx (p i ))

[0033] Among them, L rot f represents stereo perception loss, CE() represents the crossover loss function, and f zy and fzx This represents a stereo semantic perceptron. and p represents the stereo rotation level label of the simulated data to which the feature of the i-th candidate box belongs, respectively. i Let represent the feature of the i-th candidate box in the feature set p of all candidate boxes, and max represents taking the maximum value.

[0034] The beneficial effect of the above-mentioned further scheme is that this step guides the stereo perception guidance module to learn accurate stereo rotation semantics through supervised learning.

[0035] Furthermore, step S204 includes the following steps:

[0036] S2041. Map the candidate box set onto the image feature map to obtain candidate features, and divide it into three different subsets p according to the stereo rotation semantic information of each dangerous product type. 1 p 2 and p 3 The data is then input into the progressive deformation calibration module, and the following formula is used to apply it to each subset p. 1 p 2 and p 3 The candidate targets were restored to varying degrees:

[0037]

[0038] in, p represents the restored candidate target. i' Let r represent the i'-th proposal feature. 2→1 and r 3→2 Both indicated that the network had been restored;

[0039] S2042. To calibrate the recovered candidate targets, a subset p is used. 1 Candidate features, construct and update each category c k The corresponding prototype Ωc k :

[0040]

[0041] in, Indicates the category c at the t-th iteration. k The prototype, n k This indicates that it is classified into category c. k The number of candidate box features, y i The label represents the true category to which the feature of the i-th candidate box belongs. Representing subset p 1 The i-th candidate box feature in the data. Indicates the class c at the (t-1)th iteration. kThe prototype, where 'a' represents the prototype at the t-th iteration. Cosine similarity between aggregated candidate features;

[0042] S2043, using the constructed prototype Ωc k Deformation calibration loss is calculated from two perspectives: feature consistency and stereo rotation consistency.

[0043] S2044. Based on the deformation calibration loss, obtain the features of all targets in the X-ray image after visual appearance deformation correction.

[0044] The beneficial effect of the above-mentioned further solution is that this step enables the model to correct the deformation features caused by the rotation of the solid.

[0045] Furthermore, the deformation calibration loss includes the deformation calibration loss for the feature consistency angle:

[0046]

[0047] Among them, L fc represents the deformation calibration loss of the feature consistency angle, cosine() represents the cosine similarity, and i represents the feature of the i-th candidate box.

[0048] Deformation calibration loss for 3D rotational consistency angle:

[0049]

[0050] Among them, L sc This represents the deformation calibration loss for the three-dimensional rotational consistency angle.

[0051] The beneficial effect of the above-mentioned further scheme is that this step constrains the calibrated features to remain consistent with the undeformed features through supervised learning.

[0052] Secondly, the present invention also provides an X-ray security inspection dangerous goods rotation detection device based on appearance deformation correction, including a processor and a memory, wherein the processor reads a computer program or instruction in the memory to execute the steps of the X-ray security inspection dangerous goods rotation detection method based on appearance deformation correction.

[0053] Thirdly, the present invention provides a computer-readable storage medium storing a computer program that, when executed, implements the steps of an X-ray security inspection method for rotating hazardous materials based on appearance deformation correction. Attached Figure Description

[0054] Figure 1 This is a flowchart of the method of the present invention.

[0055] Figure 2The flowchart of the training process of the rotation detection model based on appearance deformation correction in the method for correcting the appearance deformation of dangerous goods for X-ray security inspection provided in the embodiments of the present invention is shown.

[0056] Figure 3 A schematic diagram of the stereoscopic perception guidance module in the method for correcting the appearance deformation of dangerous goods for X-ray security inspection provided in this embodiment of the invention.

[0057] Figure 4 A schematic diagram of the progressive deformation calibration module in the method for correcting the appearance deformation of dangerous goods for X-ray security inspection provided in this embodiment of the invention.

[0058] Figure 5 This is a structural diagram of the X-ray security inspection dangerous goods rotation detection device based on appearance deformation correction provided in an embodiment of the present invention.

[0059] Among them, 31-memory, 32-processor. Detailed Implementation

[0060] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0061] Example 1

[0062] To overcome the performance loss caused by the deformation features due to three-dimensional rotation in X-ray security inspection scenarios for the detection of dangerous goods, such as Figure 1 As shown, this invention takes target visual appearance deformation correction as its core starting point. This invention provides an X-ray security inspection method for rotating dangerous goods based on appearance deformation correction, and its implementation method is as follows:

[0063] S1. Obtain the X-ray security inspection image to be detected.

[0064] In this embodiment, in locations requiring security checks, security screening equipment is used to scan luggage to acquire X-ray security images of the luggage (hereinafter referred to as luggage images). In this invention, luggage refers to a general term for suitcases, parcels, bags, boxes, backpacks, handbags, etc.

[0065] S2. Train a rotation detection model based on appearance deformation correction, and based on the X-ray security inspection image to be inspected, use the trained rotation detection model based on appearance deformation correction to obtain the rotation detection results of all targets in the X-ray security inspection image after appearance deformation correction, thus completing the appearance deformation correction of dangerous goods for X-ray security inspection. The implementation method is as follows:

[0066] S201. Obtain images containing contraband generated by an X-ray machine, construct an image dataset, and based on the constructed image dataset, simulate the three-dimensional rotation information of the contraband using an X-ray converter, project it onto a real X-ray image to generate an extended X-ray dataset, which is then divided into a training set and a test set. The implementation method is as follows:

[0067] S2011. The X-ray machine acquires the X-ray security inspection images to be inspected, and the items in each image are labeled according to the X-ray security inspection images to be inspected.

[0068] S2012. Based on the annotation results, select images containing prohibited items and construct an image dataset;

[0069] S2013. Based on the constructed image dataset, simulate the three-dimensional rotation information of the contraband, convert the rotating three-dimensional network into two-dimensional X-ray image blocks through an X-ray converter, and insert them into real X-ray images to generate an extended X-ray dataset.

[0070] S2014. Divide the generated extended X-ray dataset into a training set and a test set.

[0071] In this embodiment, when building the image dataset, several X-ray security inspection images generated by X-ray security inspection machines are collected from real-world scenes and stored in JPG format with varying resolutions. Each image is labeled with details including the image name, the type of object, and the object's location (the pixel area it occupies in the image). This labeling information for each image forms a labeling file. When an image contains multiple objects, each object has a corresponding labeling information in the labeling file. The security inspection items include various categories, such as "batteries," "electronic devices," "glass bottles," "guns," "knives," "lighters," "metal bottles," "plastic bottles," "pressure containers," and "umbrellas." Images containing prohibited items are selected from the labeled images to build the image dataset. Then, a fine-tunable X-ray converter is constructed to generate stereoscopic rotation semantic information for X-ray images. By manually defining the rotation angle of a given category of 3D mesh, a rotated 3D mesh is obtained and projected onto the 2D X-ray image to generate the stereoscopic semantics of the rotated prohibited items. The process of generating the rotated 3D mesh is implemented according to the following formula:

[0072]

[0073] in,

[0074]

[0075] Where M represents the original 3D mesh of the contraband, M′ represents the rotated 3D mesh obtained by applying a rotation angle to M, and R x R y and R z Let θ represent the rotation transformation matrices around the x-axis, y-axis, and z-axis in three-dimensional space, respectively. zy θ represents the manually defined angle of rotation in the zy plane. zx θ represents the manually defined angle of rotation in the zx plane. xy This represents the manually defined angle of rotation in the xy-plane, where V represents the set of vertices of the original 3D mesh, and R... x (θ zy ) represents applying θ around the x-axis in three-dimensional space. zy The rotation transformation matrix of the angle, R y (θ zx ) represents applying θ around the y-axis in three-dimensional space. zx The rotation transformation matrix of the angle, R z (θ xy ) represents applying θ around the z-axis in three-dimensional space. xy The rotation transformation matrix of the angle.

[0076] Finally, the images in the dataset are divided into training and test sets according to a preset ratio (e.g., 4:1).

[0077] S202. Based on the X-ray images in the training set, each batch of X-ray images is input into the hazardous materials detection network to obtain image feature maps and a set of candidate boxes.

[0078] In this embodiment, the training set data is input into a batch of X-ray images into a hazardous materials detection network. Convolutional layers are used to generate multi-layer initial feature maps for each image. Subsequently, the candidate box generation network predicts the regions in each image that may contain targets of interest, i.e., candidate box regions, based on the obtained multi-layer initial feature maps. The generated candidate box regions are then filtered through operations such as non-maximum suppression to obtain a set of candidate boxes for each image.

[0079] like Figure 2As shown, firstly, based on the stereo semantic information generation part of the stereo perception guidance module, simulated stereo rotation information of contraband is generated and projected onto the X-ray image of the original data to generate simulated data. The simulated data is input into the backbone network to generate image feature maps and proposed candidate boxes. These are then input into the stereo rotation classification of the stereo perception guidance module to obtain stereo rotation semantic information for each hazardous item category in the batch, and stereo perception loss is calculated to achieve stereo semantic guidance. The image feature maps, candidate box set, and stereo rotation semantic information for each hazardous item category are input into the progressive feature calibration module. A progressive recovery network is used to progressively calibrate the distorted features and construct a prototype. Deformation calibration loss is calculated from both feature consistency and stereo rotation consistency perspectives to calibrate the standardized features. Finally, the corrected features are input into the rotation detection head to obtain the rotation detection results for all targets in the image after visual appearance deformation correction.

[0080] S203. Input the image feature map and candidate box set into the stereo perception guidance module to obtain the stereo rotation semantic information of each hazardous product category in this batch, and calculate the stereo perception loss.

[0081] In this embodiment, the stereoscopic rotation semantic information and stereoscopic perception loss of each hazardous product type are input into the stereoscopic perception guidance module by the image feature map and candidate box set. For example... Figure 3 As shown, based on the stereo semantic information generation part of the stereo perception guidance module, simulated stereo rotation information of contraband is generated. The image feature map and proposed candidate boxes are input into the stereo rotation classification of the stereo perception guidance module to obtain the stereo rotation semantic information of each type of hazardous item in the batch. Stereo semantic guidance is then achieved by calculating the stereo perception loss, as follows:

[0082] S2031. Map the candidate box set onto the image feature map to obtain the feature set of all candidate boxes p = {p1, ..., p...} n}, where p n This represents the feature of the nth candidate box, where n represents the total number of candidate box features.

[0083] In this embodiment, the candidate box generated by the candidate box generation network consists of five parameters: center point (x, y), width w, height h, and rotation angle θ. This step requires converting the four coordinates of the candidate box into a rectangle and mapping the rectangle onto the feature map of the image. The features enclosed by the rectangle are then extracted, which are the candidate box features, and finally, the corresponding candidate box feature set is obtained.

[0084] S2032. Based on the feature set p of all candidate boxes, calculate the stereo rotation semantic information l of each target. (i) , where i represents the i-th target.

[0085] In this embodiment, in order to obtain the stereo rotation semantic information of each target, the present invention designs two stereo rotation angle sensors f. zy and f zx To predict stereo rotation semantic information for a given instance, each proposed feature p in the feature set p. i Input to classifier f zy and f zx In this process, the semantic information of stereo rotation is discriminated, and the semantic information of stereo rotation for each target is calculated based on the feature set p of all candidate boxes. (i) This is achieved using the following formula:

[0086] l (i) =max(f zy (p i ),f zx (p i ))

[0087] Where, p i f represents the feature of the i-th candidate box in the candidate box feature set p. zy and f zx This represents a stereo semantic perceptron, and max(·) represents the maximum value between the two.

[0088] S2033, Based on the stereoscopic rotation semantic information of each target (i) Calculate the stereo perception loss.

[0089] In this embodiment, each proposal feature p in the candidate box feature set p is... i Input to classifier f zy and f zx After determining the semantic information of stereo rotation, the cross-entropy loss is used as a constraint, and the stereo perception loss is calculated according to the following formula:

[0090]

[0091] Among them, L rot This represents the stereo perception loss, and CE() represents the crossover loss function. and and represent the stereo rotation level labels of the simulated data to which the i-th candidate box feature belongs.

[0092] S204. Input the image feature map, candidate box set, and stereoscopic rotation semantic information of each hazardous type into the progressive deformation calibration module to obtain the features of all targets in the X-ray image after visual appearance deformation correction.

[0093] In this embodiment, the progressive deformation calibration module, after the network has acquired stereo perception capabilities, performs feature calibration on objects with shape deformations, enabling the detector to more easily distinguish target categories and improve model detection performance. The image feature map, candidate box set, and stereo rotation semantic information for each hazardous item category are input into the progressive deformation calibration module to obtain rotation detection results for all targets in the image after correction of their appearance deformation. Figure 4 As shown, the image feature map, candidate box set, and stereo rotation semantic information of each hazardous product type are input into the progressive feature calibration module. The progressive recovery network is used to progressively calibrate the distorted features, and a prototype is constructed. The deformation calibration loss is calculated from the perspectives of feature consistency and stereo rotation consistency, thereby obtaining the calibrated and standardized features through the following steps:

[0094] S2041. Map the candidate box set onto the image feature map to obtain candidate features, and divide it into three different subsets p according to the stereo rotation semantic information of each dangerous product type. 1 p 2 and p 3 And input it into the progressive deformation calibration module, for each subset p 1 p 2 and p 3 The candidate targets were restored to varying degrees.

[0095] In this embodiment, after supervised training, the stereo semantic perception module is able to fully perceive and distinguish the candidate set p = {p1, ..., p}. n Each proposal feature p in} i' three-dimensional semantics Based on stereo semantics Divide the candidate set p into three distinct subsets p 1 p 2 and p 3 Among them, p 1 The candidate targets in the subset have weak stereo semantics, while the subset p 2 and p 3 The candidate targets exhibit strong stereo semantics, leading to severe visual distortion and requiring varying degrees of restoration. Therefore, this invention introduces two restoration networks r 3→2 and r 2→1 Each network consists of an arrangement of linear layers and activation layers. 3→2 Intended to calibrate p 3 The visual features of the medium deformation make its stereo semantics and visual appearance similar to that of the subset p. 2 Similar; while r 2→1 Using the same method, for subset p 2 The deformable visual features are calibrated to make their stereo semantics and visual appearance consistent with subset p.1 Similar. The recovery process can be described as follows:

[0096]

[0097] in, p represents the restored candidate target. i' Let r represent the i'-th proposal feature. 2→1 and r 3→2 Both indicated that the network had been restored.

[0098] S2042. To calibrate the recovered candidate targets, a subset p is used. 1 Candidate features, construct and update each category c k The corresponding prototype Ωc k .

[0099] In this embodiment, the restored visual features of each category should be related to the subset p. 1 The proposed features are similar. This invention constructs a global prototype library Ω to guide model learning, with a label set C = {c1, c2, ..., c...}. n Each category c in} k There is a corresponding prototype Ωc in the global prototype library Ω. k This prototype is achieved by aggregating a subset p 1 Candidate features from p are used to construct and update the subset. 1 The candidate features exhibit subtle stereo semantics and possess complete discriminative visual features; these prototypes represent the non-deformable visual appearance features of each category. At the t-th iteration, if the global prototype library Ω does not yet contain category c... k The prototype Ωc k If the aggregation is successful, the aggregated features will be directly stored in the global prototype library Ω. Otherwise, the aggregated features will be compared with the prototype Ωc based on similarity. k The prototype is updated through fusion. At the end of the t-th iteration, the process of building and updating the global prototype library Ω can be described as follows:

[0100]

[0101] in, Indicates the category c at the t-th iteration. k The prototype, n k This indicates that it is classified into category c. k The number of candidate box features, y i The label represents the true category to which the feature of the i-th candidate box belongs. Representing subset p 1 The i-th candidate box feature in the data. Indicates the class c at the (t-1)th iteration. kThe prototype, where 'a' represents the prototype at the t-th iteration. Cosine similarity between the aggregated candidate features and the aggregated candidate features.

[0102] S2043. Calculate the deformation calibration loss from the perspectives of feature consistency and three-dimensional rotation consistency.

[0103] In this embodiment, from the perspective of feature consistency, it is expected that in the t-th iteration, category c k The features of the recovery proposal and its prototype features The cosine distance between them should be as small as possible:

[0104]

[0105] Among them, L fc The deformation calibration loss represents the feature consistency angle, cosine() represents the cosine similarity, and p i' This indicates the candidate target after restoration.

[0106] From the perspective of stereo rotation consistency, this invention utilizes a stereo semantic classifier f zy and f zx To constrain the degree to which the recovered visual features are classified as having slight stereo rotation semantics:

[0107]

[0108] Among them, L sc This represents the deformation calibration loss for the three-dimensional rotational consistency angle.

[0109] S2044. Based on the deformation calibration loss, obtain the features of all targets in the X-ray image after visual appearance deformation correction.

[0110] S205. Input the corrected features into the rotation detection head to obtain the rotation detection results of all targets in the image after correction of their visual appearance deformation.

[0111] S206. Determine whether the number of iterations has been reached. If yes, obtain the rotation detection model based on appearance deformation correction according to the stereo perception loss, and proceed to step S207. Otherwise, return to step S201.

[0112] In this embodiment, steps S202 to S205 are repeated until a preset number of iterations are performed to obtain a rotation detection model based on appearance deformation correction. Each time a batch of training set images is input into the network, steps S202 to S205 are executed once, updating the network parameters. After iterating and updating the network parameters a preset number of times, a model for correcting the target's visual appearance deformation is obtained.

[0113] S207. The accuracy of the rotation detection model based on appearance deformation correction is tested using a test set to obtain a trained rotation detection model based on appearance deformation correction.

[0114] In this embodiment, images from the test set data are input into the trained rotation detection model based on appearance deformation correction to verify the accuracy of the rotation detection model based on appearance deformation correction. Using test set data to test the accuracy of the rotation detection model based on appearance deformation correction is a mature existing technology (such as using a scoring method), and will not be elaborated further here.

[0115] S208. Based on the X-ray security inspection image to be inspected, the trained rotation detection model based on appearance deformation correction is used to obtain the rotation detection results of all targets in the X-ray security inspection image after appearance deformation correction, thus completing the appearance deformation correction of dangerous goods for X-ray security inspection.

[0116] Example 2

[0117] like Figure 5 As shown, the present invention provides an X-ray security inspection dangerous goods rotation detection device based on appearance deformation correction, including a processor and a memory. The processor reads computer programs or instructions in the memory to execute the steps of the X-ray security inspection dangerous goods rotation detection method based on appearance deformation correction as described in any one of Embodiment 1.

[0118] like Figure 5 As shown, this embodiment also provides a device for correcting the appearance deformation of dangerous goods inspected by X-ray security, including a processor 32 and a memory 31. It may further include communication components, sensor components, power supply components, multimedia components, and input / output interfaces as needed. The memory, communication components, sensor components, power supply components, multimedia components, and input / output interfaces are all connected to the processor 32. As previously mentioned, the memory 31 can be a static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, etc.; the processor 32 can be a central processing unit (CPU), graphics processing unit (GPU), field-programmable gate array (FPGA), application-specific integrated circuit (ASIC), digital signal processing (DSP) chip, etc. Other communication components, sensor components, power supply components, multimedia components, etc., can all be implemented using common components found in existing smartphones, and will not be specifically described here.

[0119] Example 3

[0120] The present invention provides a computer-readable storage medium storing a computer program that, when executed, implements the steps of the X-ray security inspection method for rotating dangerous goods based on appearance deformation correction as described in any one of Embodiment 1.

[0121] This embodiment also provides a computer program product containing instructions that, when run on a computer, cause the computer to perform the above-described actions. Figure 1 The specific implementation of the X-ray security inspection method for rotating dangerous goods based on appearance deformation correction will not be elaborated here.

[0122] The method and apparatus for correcting the appearance deformation of hazardous materials in X-ray security inspection provided by this invention acquires an X-ray security inspection image to be inspected and inputs it into a pre-trained rotation detection model based on appearance deformation correction. This yields the rotation detection results for all targets in the image after appearance deformation correction. This process extracts the stereo semantic information of each target and corrects the originally deformed appearance of the target, effectively solving the problem of performance degradation in target detection tasks caused by the deformation features of stereo rotation. This allows the detector to more easily distinguish target categories and improves the model's detection performance.

[0123] The foregoing has provided a detailed description of the X-ray security inspection method and apparatus for detecting rotating hazardous materials based on appearance deformation correction provided by this invention. Any obvious modifications made by those skilled in the art without departing from the essence of this invention will fall within the scope of protection of this invention's patent rights.

Claims

1. A method for detecting rotating hazardous materials in X-ray security inspection based on appearance deformation correction, characterized in that, Includes the following steps: S1. Obtain the X-ray security inspection image to be detected; S2. Train a rotation detection model based on appearance deformation correction, and based on the X-ray security inspection image to be inspected, use the trained rotation detection model based on appearance deformation correction to obtain the rotation detection results of all targets in the X-ray security inspection image after appearance deformation correction, thus completing the appearance deformation correction of dangerous goods for X-ray security inspection; step S2 includes the following steps: S201. Obtain images containing contraband generated by an X-ray machine, construct an image dataset, and based on the constructed image dataset, use an X-ray converter to simulate the three-dimensional rotation information of the contraband and project it onto a real X-ray image to generate an extended X-ray dataset, which is then divided into a training set and a test set. S202. Based on the X-ray images in the training set, each batch of X-ray images is input into the hazardous materials detection network to obtain image feature maps and candidate box sets. S203. Input the image feature map and candidate box set into the stereo perception guidance module to obtain the stereo rotation semantic information of each type of hazardous product in this batch, and calculate the stereo perception loss. S204. Input the image feature map, candidate box set, and stereoscopic rotation semantic information of each hazardous type into the progressive deformation calibration module to obtain the features of all targets in the X-ray image after visual appearance deformation correction; step S204 includes the following steps: S2041. Map the candidate box set onto the image feature map to obtain candidate features, and divide them into three different subsets according to the stereo rotation semantic information of each hazardous product type. , and The data is then input into the progressive deformation calibration module, and the following formula is used for each subset. , and The candidate targets were restored to varying degrees: in, Indicates the candidate targets after restoration. Indicates the first One proposed feature, and Both indicated that the network had been restored; S2042. To calibrate the recovered candidate targets, a subset is used. Candidate features, construct and update each category Corresponding prototype : in, Indicates the first t Category at the next iteration The prototype, Indicates being categorized as a category The number of candidate box features, Indicates the first i The label of the true category to which each candidate bounding box belongs. Representing a subset The first in i Each candidate bounding box feature Indicates the first t Class at -1 iteration The prototype, Indicates the first t Prototype in the next iteration Cosine similarity between the aggregated candidate features; S2043, Utilizing the constructed prototype Deformation calibration loss is calculated from two perspectives: feature consistency and stereo rotation consistency. S2044. Based on the deformation calibration loss, obtain the features of all targets in the X-ray image after visual appearance deformation correction. S205. Input the corrected features into the rotation detection head to obtain the rotation detection results of all targets in the image after correction of their visual appearance deformation. S206. Determine whether the number of iterations has been reached. If yes, obtain the rotation detection model based on appearance deformation correction and proceed to step S207. Otherwise, return to step S201. S207. The accuracy of the rotation detection model based on appearance deformation correction is tested using the test set to obtain the trained rotation detection model based on appearance deformation correction. S208. Based on the X-ray security inspection image to be inspected, the trained rotation detection model based on appearance deformation correction is used to obtain the rotation detection results of all targets in the X-ray security inspection image after appearance deformation correction, thus completing the appearance deformation correction of dangerous goods for X-ray security inspection.

2. The X-ray security inspection method for rotating hazardous materials based on appearance deformation correction according to claim 1, characterized in that, Step S201 includes the following steps: S2011. The X-ray machine acquires the X-ray security inspection images to be inspected, and the items in each image are labeled according to the X-ray security inspection images to be inspected. S2012. Based on the annotation results, select images containing prohibited items and construct an image dataset; S2013. Based on the constructed image dataset, simulate the three-dimensional rotation information of the contraband, convert the rotating three-dimensional network into two-dimensional X-ray image blocks through an X-ray converter, and insert them into real X-ray images to generate an extended X-ray dataset. S2014. Divide the generated extended X-ray dataset into a training set and a test set.

3. The X-ray security inspection method for rotating dangerous goods based on appearance deformation correction according to claim 1, characterized in that, Step S203 includes the following steps: S2031. Map the candidate box set onto the image feature map to obtain the feature set of all candidate boxes. ; S2032, Based on the feature set of all candidate boxes The stereoscopic perception guidance module calculates the stereoscopic rotation semantic information of each target. ; S2033, Based on the stereoscopic rotation semantic information of each target Calculate the stereo perception loss.

4. The X-ray security inspection method for rotating hazardous materials based on appearance deformation correction according to claim 3, characterized in that, The expression for the stereo perception loss is as follows: in, This indicates a loss of stereoscopic perception. Represents the crossover loss function. and This represents a stereo semantic perceptron. and They represent the first The stereo rotation level label of the simulated data to which each candidate box feature belongs. Represents the feature set of all candidate boxes The Middle i Each candidate bounding box feature This indicates taking the maximum value.

5. The X-ray security inspection method for rotating hazardous materials based on appearance deformation correction according to claim 1, characterized in that, The deformation calibration loss includes the deformation calibration loss for the feature consistency angle: in, The deformation calibration loss represents the feature consistency angle. Represents cosine similarity. Indicates the first i Each candidate bounding box feature; Deformation calibration loss for 3D rotational consistency angle: in, This represents the deformation calibration loss for the three-dimensional rotational consistency angle.

6. A rotating detection device for dangerous goods in X-ray security inspection based on appearance deformation correction, characterized in that, It includes a processor and a memory, wherein the processor reads computer programs or instructions from the memory to perform the steps of the X-ray security inspection method for rotating hazardous materials based on appearance deformation correction as described in any one of claims 1-5.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed, it implements the steps of the X-ray security inspection method for rotating dangerous goods based on appearance deformation correction as described in any one of claims 1-5.

Images