A method for detecting objects in images of X-ray security inspection machines

Abstract

The present invention discloses a method for object detection in X-ray security inspection machine images, comprising: obtaining an X-ray security inspection machine image data set, generating high-quality X-ray image samples through a generative adversarial network, expanding the data set, and optimizing the training process of the adversarial network generator through reinforcement learning; constructing a multi-task learning model, jointly training object detection, object segmentation, and object category recognition, and in the absence of labeled data, generating feature representations as initial weights through self-supervised learning, which are applied to the training of the object detection model; the present invention generates an adversarial network: traditional data enhancement methods may have limitations in the diversity and authenticity of generated samples. The samples generated by GAN can better simulate complex situations in actual environments, thereby improving the robustness of the model in practical applications. Reinforcement learning optimization: using reinforcement learning to optimize the training process of GAN, the generation strategy can be dynamically adjusted to make the generated image samples more diverse and realistic.

Description

Technical Field

[0001] The present invention relates to the technical field of image object detection, and in particular to an image object detection method for an X-ray security inspection machine. Background Art

[0002] exist In optical security inspection systems, the main task of object detection methods is to identify objects in images and perform classification and analysis. Problems with current technologies include poor image quality, difficulty in detection due to overlapping objects, real-time issues, and false positives and false negatives.

[0003] 1. Image quality issues:

[0004] Noise and artifacts: X-ray images may contain noise and artifacts, which affect the accuracy of object detection.

[0005] Uneven contrast: Differences in contrast between objects can make some objects difficult to identify.

[0006] 2. Object Detection Accuracy:

[0007] Overlapping objects: When multiple objects overlap, it may lead to false detection or missed detection.

[0008] Small object detection: Detection of small objects may be limited by image resolution and processing algorithms.

[0009] 3. Computational complexity:

[0010] Real-time requirements: Complex processing and analysis of images may result in excessively long calculation times, affecting real-time performance.

[0011] 4. Algorithm generalization ability:

[0012] Insufficient training data: The training of deep learning models requires a large amount of labeled data. Insufficient data will affect the generalization ability of the model.

[0013] 5. Environmental changes: Different security equipment and environmental conditions may cause the model to perform inconsistently in different scenarios.

[0014] 6. False positive and false negative problems:

[0015] False positive: Mistakenly identifying a non-dangerous item as a dangerous item may result in a false alarm.

[0016] False negative: Dangerous items are misclassified as non-dangerous items, which may lead to missed reports. Summary of the Invention

[0017] In order to solve one of the above technical problems, a method for detecting objects in X-ray security inspection machine images is provided, which has higher accuracy, better data processing capability, higher real-time performance and adaptability, and can effectively solve The main technical issues in object detection in optical security inspection machines.

[0018] In order to achieve the above objects, the technical solution adopted by the present invention is:

[0019] A first aspect: A method for detecting objects in an X-ray security inspection machine image, comprising the following steps:

[0020] Obtain an X-ray security inspection machine image dataset, generate high-quality X-ray image samples using a generative adversarial network, expand the dataset, and optimize the training process of the adversarial network generator through reinforcement learning;

[0021] Build a multi-task learning model to jointly train object detection, object segmentation, and object category recognition. In the absence of labeled data, self-supervised learning is used to generate feature representations as initial weights for object detection model training.

[0022] Object detection is performed based on the transformer network, long-range dependencies in the image are processed through the attention mechanism, and the inference path is dynamically selected according to the needs of the actual detection task through dynamic reasoning technology;

[0023] According to the Isolation Forest anomaly detection model, abnormal objects or unusual combinations of objects are identified in X-ray images, and an adaptive decision-making system is constructed based on the identified abnormal objects or unusual combinations of objects.

[0024] Optimal: Obtain an X-ray security inspection machine image dataset, generate high-quality X-ray image samples using a generative adversarial network, expand the dataset, and optimize the training process of the adversarial network generator through reinforcement learning, specifically including:

[0025] Collecting and organizing an X-ray image dataset, the X-ray image dataset including several different types of objects, and preprocessing the X-ray image dataset, the preprocessing including normalization and resizing;

[0026] Constructing a GAN (Gan Adversarial Network) model, the GAN model including a generator and a discriminator, wherein the generator receives noise input and generates simulated X-ray image samples, and the discriminator distinguishes between real images and generated images;

[0027] Training the model, where in each iteration, the generator generates several forged X-ray images, the discriminator distinguishes the forged X-ray images from real images, and the adversarial loss function is used to optimize the generator and the discriminator so that the generated images gradually approach the real images;

[0028] Use the trained generator to generate high-quality X-ray image samples for expanding the dataset, where the expression is:

[0029] ;

[0030] in, is the discriminator, is a generator, is a real image, It's noise. is the data distribution, is the noise distribution.

[0031] Preferably, the training process of optimizing the adversarial network generator through reinforcement learning is expressed as:

[0032] ;

[0033] in, is in the time step Rewards, It’s an instant reward. is the discount factor.

[0034] Preferred: Build a multi-task learning model to jointly train object detection, object segmentation, and object category recognition. In the absence of labeled data, generate feature representations as initial weights through self-supervised learning and apply them to the training of the object detection model. Specifically,

[0035] The multi-task learning model includes sub-task networks for object detection, object segmentation, and object category recognition;

[0036] According to the multi-task learning model sharing underlying features, the multi-task learning model is trained by a joint loss function;

[0037] In response to the weighted summation of the losses of each task, the entire multi-task learning model is optimized, the performance of the multi-task learning model is evaluated on the validation set, and the weight parameters are adjusted to obtain the best performance of the multi-task learning model. The expression is:

[0038] ;

[0039] in, is the detection loss, is the segmentation loss, is the classification loss, and is the weight coefficient.

[0040] Preferably, the self-supervised learning generates feature representations through rotation prediction, image restoration, or contrastive learning, constructs a self-supervised model, uses unlabeled X-ray image data, trains the self-supervised model to extract effective feature representations, and uses the feature representations obtained through self-supervised learning as initial weights for training the object detection model. The expression is:

[0041] ;

[0042] in, is the reconstruction loss, is the contrast loss.

[0043] Preferred: Object detection is performed based on the transformer network, long-range dependencies in the image are processed through the attention mechanism, and the inference path is dynamically selected according to the needs of the actual detection task through dynamic reasoning technology, including:

[0044] Perform object detection based on the DETR transformer network model, using the attention mechanism to find the location and bounding box of objects in the global image range;

[0045] Through the self-attention mechanism, the DETR transformer network model focuses on important areas in the image and handles long-range dependencies;

[0046] Use some labeled data for training, and evaluate and adjust on the validation set. The self-attention mechanism expression is:

[0047] ;

[0048] ;

[0049] in, is the query matrix, is the bond matrix, is the value matrix, is the dimension of the key.

[0050] Preferably: the dynamic reasoning technology includes:

[0051] Dynamically select the optimal reasoning path based on the dynamic requirements of the actual detection task of the current input image, including the complexity and required accuracy of the image;

[0052] Different computing resources are allocated to each reasoning path, and reinforcement learning or heuristic algorithms are used to dynamically select the reasoning path. The expression is:

[0053] ;

[0054] in, is the path efficiency, is the path accuracy.

[0055] Preferably, an abnormal object or unusual combination of objects is identified in an X-ray image based on an Isolation Forest anomaly detection model, and an adaptive decision-making system is constructed based on the identification of abnormal objects or unusual combination of objects, specifically including:

[0056] Build an anomaly detection model to identify unusual objects or unusual combinations of objects, including weapons, explosives, flammable liquid containers, and living animals. Unusual combinations of objects include electronic devices and suspicious items, clothing and prohibited tools, and metal objects and liquids.

[0057] Train anomaly detection models using historical and simulated anomaly data and deploy them in live systems;

[0058] In the X-ray image stream, the anomaly detection model analyzes and detects abnormal objects in real time, triggering an alarm mechanism or starting manual review. The anomaly detection model expression is:

[0059] ;

[0060] in, is the depth of the outlier, is the number of samples, and Depth is the depth of the tree.

[0061] Optimum: Adaptively adjust the detection strategy based on the type and number of objects detected in real time and the abnormal situation data in the historical time period; Adaptively adjust the detection threshold based on the real-time detection results, and optimize the rules and parameters of the decision-making system through the feedback loop. The formula is:

[0062] Base Threshold+Adjustment Factor;

[0063] Among them, Base Threshold is the basic threshold and Adjustment Factor is the adjustment factor.

[0064] The second aspect: A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the processor implements the X-ray security inspection machine image object detection method.

[0065] Compared with the prior art, the present invention has the following beneficial effects:

[0066] This invention uses a generative adversarial network (GAN): Traditional data augmentation methods may have limitations in the diversity and authenticity of generated samples. The samples generated by GAN can better simulate the complex situations in the real environment, thereby improving the robustness of the model in practical applications.

[0067] Reinforcement Learning Optimization: Utilizing reinforcement learning to optimize the GAN training process, we can dynamically adjust the generation strategy, making the generated image samples more diverse and realistic. This approach can improve image generation quality through continuous feedback, addressing the sample diversity issue in traditional GAN ​​training.

[0068] Multi-task learning: By jointly training multiple tasks (such as object detection, segmentation, and classification), the model can improve its ability to recognize different types of objects while learning shared features. This approach not only improves the model's detection accuracy but also enhances its performance in complex scenarios.

[0069] Self-supervised learning: Generating feature representations through self-supervised learning in the absence of labeled data can further improve model performance, especially when data is scarce. This approach leverages the potential information in unlabeled data to address the data shortage problem in traditional supervised learning.

[0070] Transformer Networks: Traditional convolutional neural networks (CNNs) may perform poorly when dealing with long-range dependencies. Transformer Networks, through an attention mechanism, can effectively capture long-range dependencies in images, thereby improving the detection capabilities of complex objects and scenes.

[0071] Dynamic Inference: By dynamically selecting inference paths based on the actual detection task, the computational burden is reduced and system efficiency is improved. This approach can adjust the inference strategy according to specific task requirements and optimize the use of computing resources.

[0072] Real-time anomaly detection: Using anomaly detection models (such as IsolationForest) can identify unusual objects and anomalies in real time, reducing false positives and false negatives and improving system reliability.

[0073] Adaptive decision-making: Adaptive decision-making based on real-time data and historical records can adjust detection strategies according to environmental changes, improving the adaptability and flexibility of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 The figure is a flow chart of the method for detecting objects in images of X-ray security inspection machines according to the present invention. DETAILED DESCRIPTION

[0075] The following description is intended to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are merely examples, and those skilled in the art may conceive of other obvious variations.

[0076] Example 1:

[0077] Reference Figure 1 As shown, a method for detecting objects in an X-ray security inspection machine image includes the following steps:

[0078] Obtain an X-ray security inspection machine image dataset, generate high-quality X-ray image samples using a generative adversarial network, expand the dataset, and optimize the training process of the adversarial network generator through reinforcement learning;

[0079] Build a multi-task learning model to jointly train object detection, object segmentation, and object category recognition. In the absence of labeled data, self-supervised learning is used to generate feature representations as initial weights for object detection model training.

[0080] Object detection is performed based on the transformer network, long-range dependencies in the image are processed through the attention mechanism, and the inference path is dynamically selected according to the needs of the actual detection task through dynamic reasoning technology;

[0081] According to the Isolation Forest anomaly detection model, abnormal objects or unusual combinations of objects are identified in X-ray images, and an adaptive decision-making system is constructed based on the identified abnormal objects or unusual combinations of objects.

[0082] Specifically: Obtain an X-ray security inspection machine image dataset, generate high-quality X-ray image samples through a generative adversarial network, expand the dataset, and optimize the training process of the adversarial network generator through reinforcement learning, including:

[0083] Collecting and organizing an X-ray image dataset, the X-ray image dataset including several different types of objects, and preprocessing the X-ray image dataset, the preprocessing including normalization and resizing;

[0084] Many different types of objects (such as metals, liquids, electronic devices, etc.);

[0085] Constructing a GAN (Gan Adversarial Network) model, the GAN model including a generator and a discriminator, wherein the generator receives noise input and generates simulated X-ray image samples, and the discriminator distinguishes between real images and generated images;

[0086] Training the model, where in each iteration, the generator generates several forged X-ray images, the discriminator distinguishes the forged X-ray images from real images, and the adversarial loss function is used to optimize the generator and the discriminator so that the generated images gradually approach the real images;

[0087] Use the trained generator to generate high-quality X-ray image samples for expanding the dataset, where the expression is:

[0088] ;

[0089] in, is the discriminator, is a generator, is a real image, It's noise. is the data distribution, is the noise distribution.

[0090] Combining GAN with reinforcement learning: Generative Adversarial Networks (GANs) generates high-quality X-ray image samples, and combines them with reinforcement learning to optimize the generation strategy, improving the diversity and authenticity of the dataset and thus enhancing the robustness of the model.

[0091] Specifically: The training process of optimizing the adversarial network generator through reinforcement learning is expressed as:

[0092] ;

[0093] in, is in the time step Rewards, It’s an instant reward. is the discount factor.

[0094] Specifically: Build a multi-task learning model to jointly train object detection, object segmentation, and object category recognition. In the absence of labeled data, generate feature representations as initial weights through self-supervised learning and apply them to the training of the object detection model. Specifically,

[0095] The multi-task learning model includes sub-task networks for object detection, object segmentation, and object category recognition;

[0096] According to the multi-task learning model sharing underlying features, the multi-task learning model is trained by a joint loss function;

[0097] In response to the weighted summation of the losses of each task, the entire multi-task learning model is optimized, the performance of the multi-task learning model is evaluated on the validation set, and the weight parameters are adjusted to obtain the best performance of the multi-task learning model. The expression is:

[0098] ;

[0099] in, is the detection loss, is the segmentation loss, is the classification loss, and is the weight coefficient. Application of multi-task learning and self-supervised learning: Jointly training detection, segmentation, and recognition tasks, combined with self-supervised learning techniques, improves detection accuracy and model generalization capabilities.

[0100] Specifically, the self-supervised learning generates feature representations through rotation prediction, image restoration, or contrastive learning, constructs a self-supervised model, uses unlabeled X-ray image data, trains the self-supervised model to extract effective feature representations, and uses the feature representations obtained through self-supervised learning as initial weights for training the object detection model. The expression is:

[0101] ;

[0102] in, is the reconstruction loss, is the contrast loss.

[0103] Specifically: Object detection is performed based on the transformer network, long-distance dependencies in the image are processed through the attention mechanism, and the inference path is dynamically selected according to the needs of the actual detection task through dynamic reasoning technology, including:

[0104] Perform object detection based on the DETR transformer network model, using the attention mechanism to find the location and bounding box of objects in the global image range;

[0105] Through the self-attention mechanism, the DETR transformer network model focuses on important areas in the image and handles long-range dependencies;

[0106] Use some labeled data for training, and evaluate and adjust on the validation set. The self-attention mechanism expression is:

[0107] ;

[0108] ;

[0109] in, is the query matrix, is the bond matrix, is the value matrix, The transformer network is used to handle long-range dependencies in images, which improves the accuracy and efficiency of object detection, especially in complex scenes.

[0110] Specifically: The dynamic reasoning technology includes:

[0111] Dynamically select the optimal reasoning path based on the dynamic requirements of the actual detection task of the current input image, including the complexity and required accuracy of the image;

[0112] Different computing resources are allocated to each reasoning path, and reinforcement learning or heuristic algorithms are used to dynamically select the reasoning path. The expression is:

[0113] ;

[0114] in, is the path efficiency, is the path accuracy.

[0115] Specifically: Based on the Isolation Forest anomaly detection model, we identify abnormal objects or unusual combinations of objects in X-ray images and build an adaptive decision-making system based on the identification of abnormal objects or unusual combinations of objects. Specifically, we include:

[0116] Build an anomaly detection model to identify unusual objects or unusual combinations of objects, including weapons, explosives, flammable liquid containers, and living animals. Unusual combinations of objects include electronic devices and suspicious items, clothing and prohibited tools, and metal objects and liquids.

[0117] Aspects of unusual objects include:

[0118] Prohibited items:

[0119] Weapons: Firearms and their components, including the metal frames of pistols and rifles, can be identified as unusual objects in X-ray images due to their unique shape and density, even when disguised. Knives, such as folding knives and daggers, also have metal blades of varying lengths and shapes that appear distinct from ordinary luggage, making them easily detected.

[0120] Explosives-related: For example, detonators have a relatively regular small tubular metal shape; if the gunpowder component commonly found in homemade explosives is carried, it may appear as a block or powder aggregate with certain density and shape characteristics. These are abnormal objects that are strictly prohibited during security checks.

[0121] Flammable liquid containers: For example, large-capacity containers of flammable liquids such as gasoline and alcohol, whose appearance and internal liquids are different from those of conventionally allowed items in X-ray images, will be identified as abnormal objects because these items pose a great safety hazard in an aircraft transportation environment.

[0122] Live animals (unauthorized): If a passenger attempts to conceal a pet or other small animal in luggage, X-ray images will reveal features such as its skeleton and body contours that differ from those of ordinary luggage, making it considered an abnormal object. For example, if a kitten is concealed in a backpack, its spine, limbs, and other skeletal features, as well as the contours of its soft tissues, can be discerned on a security X-ray. Under normal circumstances, live animals should not be present in such luggage.

[0123] Out-of-specification lithium batteries: Lithium batteries can appear specific in images based on their internal structure and external packaging. However, if the specifications of a lithium battery exceed the standards allowed by the airline for carry-on or checked baggage, such as an ultra-large capacity power bank, it will be considered an abnormal object because large-capacity lithium batteries pose safety risks such as overheating and fire during flight.

[0124] Unusual aspects of object combinations include:

[0125] Combination of electronic devices and suspicious items: For example, in a laptop bag, in addition to normal images of laptop components, if there are also unidentified objects connected by wires, and the objects have suspicious shapes and density characteristics similar to a combination of batteries and detonators, this combination is very unusual and is very likely to involve the disguise of an improvised explosive device, which requires special inspection during security checks.

[0126] Combination of clothing and prohibited tools: If metal tools of a certain length and shape, such as screwdrivers and wrenches, are mixed among a pile of clothing, and these tools do not meet the requirements for the types of tools that can be carried on board, the combination of clothing and tools that should not be there will be unusual from the X-ray image, and may be a combination of items prepared for improper behavior such as intending to damage aircraft equipment.

[0127] Unusual density distribution within luggage: Even if items are arranged in a disorderly manner, the density distribution within normal luggage is relatively normal. For example, when common items such as clothing and books are mixed together, the overall density is relatively uniform, and the image characteristics of each area have a certain pattern. However, if a suitcase has a very high density in one area, such as a large accumulation of metal, while another area is almost empty, and this contrasting density distribution cannot be explained by normal luggage items, this combination is unusual and may contain unusual objects such as prohibited large metal products, requiring further inspection and verification.

[0128] Train anomaly detection models using historical and simulated anomaly data and deploy them in live systems;

[0129] In the X-ray image stream, the anomaly detection model analyzes and detects abnormal objects in real time, triggering an alarm mechanism or starting manual review. The anomaly detection model expression is:

[0130] ;

[0131] in, is the depth of the outlier, is the number of samples, and Depth is the depth of the tree. Dynamic reasoning and real-time anomaly detection: Dynamic reasoning technology reduces computational burden. The real-time anomaly detection model identifies anomalies and reduces false positives and false negatives. The adaptive decision system provides flexible detection strategy adjustments, improving overall system performance.

[0132] Specifically: Adaptively adjust the detection strategy based on the type and number of objects detected in real time and the abnormal situation data in the historical time period; Adaptively adjust the detection threshold based on the real-time detection results, and optimize the rules and parameters of the decision-making system through the feedback loop. The formula is:

[0133] Base Threshold+Adjustment Factor;

[0134] Among them, Base Threshold is the basic threshold and Adjustment Factor is the adjustment factor.

[0135] 1. Combination of GAN and reinforcement learning:

[0136] Generative Adversarial Networks (GANs): Traditional data augmentation methods may have limitations in the diversity and authenticity of generated samples. Samples generated by GANs can better simulate the complexities of real-world environments, thereby improving the robustness of models in practical applications.

[0137] Reinforcement Learning Optimization: Utilizing reinforcement learning to optimize the GAN training process, we can dynamically adjust the generation strategy, making the generated image samples more diverse and realistic. This approach can improve image generation quality through continuous feedback, addressing the sample diversity issue in traditional GAN ​​training.

[0138] For example, in airport security, GANs can generate images of various types of dangerous goods, increasing the diversity of datasets. Combined with reinforcement learning, the model can adaptively adjust during the generation process, producing samples that match the actual situation, thereby improving the object detection system's ability to recognize complex objects.

[0139] 2. Application of multi-task learning and self-supervised learning:

[0140] Multi-task learning: By jointly training multiple tasks (such as object detection, segmentation, and classification), the model can improve its ability to recognize different types of objects while learning shared features. This approach not only improves the model's detection accuracy but also enhances its performance in complex scenarios.

[0141] Self-supervised learning: Generating feature representations through self-supervised learning in the absence of labeled data can further improve model performance, especially when data is scarce. This approach leverages the potential information in unlabeled data to address the data shortage problem in traditional supervised learning.

[0142] For example, for object detection in optical images, multi-task learning can simultaneously optimize the ability to identify an object's location, boundaries, and category. In scenarios with unlabeled data, self-supervised learning techniques can generate useful feature representations through contrastive learning, further improving the accuracy of the detection system.

[0143] 3. Introduction of Transformer Network:

[0144] Transformer Networks: Traditional convolutional neural networks (CNNs) may perform poorly when dealing with long-range dependencies. Transformer Networks, through an attention mechanism, can effectively capture long-range dependencies in images, thereby improving the detection capabilities of complex objects and scenes.

[0145] For example, in light images, transformer networks can effectively process global contextual information, such as detecting the relative position and structure of complex items inside a package. This approach offers advantages over traditional CNNs in handling complex scenes, improving the system's overall detection accuracy.

[0146] 4. Dynamic reasoning and real-time anomaly detection:

[0147] Dynamic Inference: By dynamically selecting inference paths based on the actual detection task, the computational burden is reduced and system efficiency is improved. This approach can adjust the inference strategy according to specific task requirements and optimize the use of computing resources.

[0148] Real-time anomaly detection: Using anomaly detection models (such as IsolationForest) can identify unusual objects and anomalies in real time, reducing false positives and false negatives and improving system reliability.

[0149] Adaptive decision-making: Adaptive decision-making based on real-time data and historical records can adjust detection strategies according to environmental changes, improving the adaptability and flexibility of the system.

[0150] For example, when handling high-volume security inspections, dynamic reasoning technology can adjust calculation paths based on real-time conditions to optimize processing efficiency. Real-time anomaly detection models can quickly identify and address anomalous objects, reducing false positives and missed alerts. Adaptive decision-making systems can automatically adjust detection strategies based on varying environmental conditions (such as peak traffic periods) to improve overall detection performance.

[0151] In summary:

[0152] Compared with the existing technology, the present invention not only has significant progress in the diversity, accuracy and efficiency of data processing, but can also effectively solve the main technical difficulties in object detection in traditional optical security inspection machines and improve the performance and reliability of the overall system.

[0153] Example 2

[0154] A sort of The object detection method for optical security inspection machine images, applied to deep learning and multimodal fusion, includes the following steps:

[0155] Step 1: Integrate data from different Datasets from optical security inspection machines and different environments to improve the model's generalization capabilities; datasets include data from different angles, exposure conditions, and object types; use synthetic image generation techniques (e.g., using GAN generators) to expand the dataset and simulate rare objects and abnormal situations (e.g., objects in different backgrounds and environments); and apply image enhancement techniques such as rotation, scaling, translation, and color transformation to improve the model's robustness to image deformation and noise.

[0156] Geometric transformations: such as rotation, flipping, scaling, etc., make the model robust to objects of different angles and scales.

[0157] Color jittering: Enhance the model's adaptability to different lighting conditions by adjusting brightness, contrast, saturation, etc.

[0158] Noise addition: Add Gaussian noise, salt and pepper noise, etc. to the image to enable the model to correctly identify objects in the presence of noise interference;

[0159] Geometric transformation matrix: ;

[0160] Noise addition: ;

[0161] in, For the enhanced image, The mean is , the standard deviation is Noise

[0162] Synthetic sample generation: Generate synthetic samples based on existing object categories, simulating various occlusion situations, object overlaps, and lighting conditions;

[0163] Multimodal fusion: through integration The combination of optical images and other sensor data enhances the comprehensiveness and accuracy of object detection. This innovation leverages multimodal information to improve the recognition ability of complex scenes and objects.

[0164] Step 2: Joint detection and classification: Use a multi-task learning network (such as YOLOv5, Efficient Det) to simultaneously detect and classify objects, improving both detection and classification accuracy. Introduce a fine-grained feature learning module to extract more detailed features of objects to enhance the ability to distinguish similar objects. Introduce a dynamic convolutional layer to dynamically adjust the convolution kernel weights based on the input data to adapt to the characteristics of different objects.

[0165] Integrate attention mechanisms (such as SE blocks or CBAM) to enhance focus on key information and automatically optimize important areas of feature maps;

[0166] Dynamic convolution and attention mechanism: Dynamic convolution and attention mechanism improve the ability to recognize object details, enable the model to adaptively adjust under different conditions, and enhance the robustness of detection;

[0167] Step 3: Fusion of X-ray images with other sensor data: Integrate other sensor data (such as millimeter wave radar, infrared images) with Light images, combined with multimodal information through deep fusion networks (such as Transformer fusion modules), provide more comprehensive object information; using multi-angle shooting Fusion of optical images to reconstruct three-dimensional information of objects and improve object recognition and classification;

[0168] Real-time optimization and feedback mechanism: Real-time optimization and automated training systems enable continuous improvement of models during actual operation, solving the problem of traditional models being unable to be updated in a timely manner.

[0169] Based on contextual information (such as background objects and environmental information of the detection area), use context-aware networks (such as Transformer models) to improve the accuracy of object detection;

[0170] Context perception and relationship modeling: Through context-aware analysis and target relationship modeling, the recognition ability of complex scenes and object combinations is improved, reducing false detections and missed detections;

[0171] In complex scenes, the relationship between objects is modeled through relational networks to improve the recognition ability of complex object combinations;

[0172] Step 4: Use lightweight deep learning models (such as Mobile Net and Efficient Net) to optimize computing performance, ensure fast processing in a real-time environment, implement a dynamic adjustment mechanism, and automatically adjust model parameters and processing strategies based on actual operating conditions (such as processing speed and detection accuracy). Integrate user feedback mechanisms to collect false detections and missed detections in real time, and dynamically adjust and optimize model parameters. Automated training system: Build an automated model training and update system to automatically update the model based on real-time data and user feedback to maintain continuous improvement in detection capabilities.

[0173] Specifically,

[0174] 1. Innovation in multimodal fusion:

[0175] Existing optical security inspection systems typically rely solely on light patterns for object detection and recognition. This approach can be limited in scenes with occluded, overlapping, or complex objects.

[0176] Multimodal data fusion: By integrating X-ray images with data from other sensors (such as millimeter-wave radar and infrared imagery), a more comprehensive understanding of the object is provided. This fusion can significantly improve detection accuracy. For example, millimeter-wave radar can penetrate most materials, helping to detect objects obscured by X-ray images; infrared imagery can help identify objects with abnormal temperatures.

[0177] Multi-view fusion: Combining light images taken from different angles to reconstruct the three-dimensional information of an object is rare in existing technologies. This method can more clearly identify complex object combinations and improve recognition accuracy.

[0178] 2. Application of dynamic convolution and attention mechanism:

[0179] Traditional deep learning models use fixed convolution kernels for feature extraction and may not be able to adapt to changes in different objects and environmental conditions.

[0180] Dynamic Convolution: This layer introduces a dynamic convolution layer that dynamically adjusts the convolution kernel weights based on the input data. This innovation improves the model's adaptability to different object characteristics, making it more flexible and accurate when processing different objects or complex scenes.

[0181] Attention Mechanism: Applying an attention mechanism (such as SE blocks or CBAM) increases focus on key information and automatically optimizes important areas of the feature map. This mechanism can significantly improve the model's ability to extract key features and reduce interference from background noise.

[0182] 3. Real-time optimization and automated training system:

[0183] Existing optical security inspection systems usually rely on static models during operation and cannot be optimized and updated in real time according to actual conditions.

[0184] Real-time Processing and Optimization: Using lightweight models and dynamic adjustment strategies, we achieve rapid processing and optimization in real-world environments. This enables the system to respond to environmental changes in real time, improving detection efficiency. For example, the system can automatically adjust model parameters based on actual operating conditions, ensuring efficient performance even under high loads.

[0185] Automated training system: Build an automated model training and update system that automatically updates the model based on real-time data and user feedback. This approach solves the problem of traditional models requiring manual updates, enabling the system to continuously improve and adapt to new detection needs.

[0186] 4. Context awareness and relationship modeling:

[0187] Traditional methods usually ignore the relationships between objects and scene context, and have limited ability to recognize object combinations in complex scenes.

[0188] Context-aware optimization: Context-aware networks (such as the Transformer model) analyze scene context to provide more accurate object recognition. For example, the model can adjust its detection strategy based on scene context to distinguish between background objects and primary objects.

[0189] Target Relationship Modeling: Relational Networks are used to model the relationships between targets, improving the ability to identify complex combinations of objects. For example, when multiple objects are found in luggage, the system can identify their relationships, thereby improving the accuracy of identifying potential threats.

[0190] For example:

[0191] Practical Application Scenario: During airport security checks, traditional optical scanners may struggle to identify complex combinations of objects in luggage, such as mixtures of metal objects with batteries and liquids. With the introduction of multimodal data fusion technology, millimeter-wave radar can penetrate liquid containers, and infrared imaging can detect abnormal battery heating, thereby improving the ability to identify potentially dangerous items.

[0192] Complex scene processing: Using dynamic convolution and attention mechanisms can help the model better identify occluded objects in a piece of luggage. For example, if a large package contains multiple small items, the attention mechanism can automatically focus on the key features of these small items, even if they are partially obscured by other items.

[0193] Real-time Adjustment: During peak hours, traditional systems can experience slow processing speeds and reduced efficiency. Using real-time optimization and automated training systems, models can dynamically adjust to handle high loads, improving overall processing speed and accuracy while avoiding performance bottlenecks during busy periods.

[0194] Example 3

[0195] An embodiment of the present application also provides a computer-readable storage medium, which can be set in a server to store at least one instruction or at least one program related to an X-ray security inspection machine image object detection method in a method embodiment. The at least one instruction or the at least one program is loaded and executed by the processor to implement the X-ray security inspection machine image object detection method provided in the above method embodiment.

[0196] Optionally, in this embodiment, the storage medium may be located in at least one of a plurality of network servers in the computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard drive, a magnetic disk, or an optical disk, among other media capable of storing program code.

[0197] Example 4

[0198] An embodiment of the present invention further provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method for detecting objects in X-ray security inspection images provided in the various optional embodiments described above.

[0199] It should be noted that the order of the embodiments of the present application described above is for descriptive purposes only and does not represent the superiority or inferiority of the embodiments. The above description is of specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0200] The various embodiments in this application are described in a progressive manner. Similar portions between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the device, equipment, and storage medium embodiments are generally similar to the method embodiments, so their descriptions are relatively simple. For relevant portions, refer to the descriptions of the method embodiments.

[0201] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0202] With the above-described preferred embodiments of the present invention as inspiration, and with reference to the above description, relevant personnel are fully capable of making various changes and modifications without departing from the technical scope of this invention. The technical scope of this invention is not limited to the contents of the specification and must be determined according to the scope of the claims.

[0203] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various modifications and substitutions within the technical scope disclosed in the present invention, and such modifications and substitutions are intended to be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A method for detecting objects in X-ray security inspection machine images, characterized in that: The following steps are involved: Obtain an X-ray security inspection machine image dataset, generate high-quality X-ray image samples using a generative adversarial network, expand the dataset, and optimize the training process of the adversarial network generator through reinforcement learning; Build a multi-task learning model to jointly train object detection, object segmentation, and object category recognition. In the absence of labeled data, self-supervised learning is used to generate feature representations as initial weights for object detection model training. Object detection is performed based on the transformer network. The attention mechanism is used to process long-distance dependencies in the image. Dynamic reasoning technology is used to dynamically select the reasoning path based on the actual detection task requirements. Specifically, it includes: Perform object detection based on the DETR transformer network model, using the attention mechanism to find the location and bounding box of objects in the global image range; Through the self-attention mechanism, the DETR transformer network model focuses on important areas in the image and handles long-range dependencies; The system is trained using a number of labeled data sets and evaluated and adjusted on a validation set. The dynamic inference techniques described include: Dynamically select the optimal reasoning path based on the dynamic requirements of the actual detection task of the current input image, including the complexity and required accuracy of the image; Allocate different computing resources to each reasoning path and use reinforcement learning or heuristic algorithms to dynamically select the reasoning path; Based on the Isolation Forest anomaly detection model, we identify abnormal objects or unusual combinations of objects in X-ray images and build an adaptive decision-making system based on the identification of abnormal objects or unusual combinations of objects. Specifically, we include: Build an anomaly detection model to identify unusual objects or unusual combinations of objects, including weapons, explosives, flammable liquid containers, and living animals. Unusual combinations of objects include electronic devices and suspicious items, clothing and prohibited tools, and metal objects and liquids. Train anomaly detection models using historical and simulated anomaly data and deploy them in live systems; In the X-ray image stream, the anomaly detection model analyzes and detects abnormal objects in real time, triggering an alarm mechanism or initiating manual review. The adaptive decision-making system includes: adaptively adjusting the detection strategy based on the type and number of objects detected in real time and the anomaly data within a historical time period; adaptively adjusting the detection threshold based on the real-time detection results, and optimizing the rules and parameters of the decision-making system through a feedback loop. Collecting and organizing an X-ray image dataset, the X-ray image dataset including several different types of objects, and preprocessing the X-ray image dataset, the preprocessing including normalization and resizing; Constructing a GAN model, the GAN model including a generator and a discriminator, wherein the generator receives noise input and generates simulated X-ray image samples, and the discriminator distinguishes between real images and generated images; Training the model, wherein in each iteration, the generator generates several forged X-ray images, the discriminator distinguishes the forged X-ray images from real images, and the generator and the discriminator are optimized using an adversarial loss function; Use the trained generator to generate high-quality X-ray image samples for expanding the dataset; The training process of optimizing the adversarial network generator through reinforcement learning is expressed as: ； in, is in the time step Rewards, It’s an instant reward. is the discount factor.

2. The method according to claim 1, wherein: Build a multi-task learning model to jointly train object detection, object segmentation, and object category recognition. In the absence of labeled data, generate feature representations as initial weights through self-supervised learning and apply them to the training of the object detection model. Specifically, The multi-task learning model includes sub-task networks for object detection, object segmentation, and object category recognition; According to the multi-task learning model sharing underlying features, the multi-task learning model is trained by a joint loss function; In response to the loss of each task, the weighted summation is performed through the weights, the entire multi-task learning model is optimized, the performance of the multi-task learning model is evaluated on the validation set, and the weight parameters are adjusted to obtain the best performance of the multi-task learning model.

3. The method according to claim 2, wherein: The self-supervised learning generates feature representations through rotation prediction, image restoration or contrastive learning, constructs a self-supervised model, uses unlabeled X-ray image data, trains the self-supervised model to extract effective feature representations, and uses the feature representations obtained by self-supervised learning as initial weights for the training of the object detection model.

4. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the X-ray security inspection machine image object detection method according to any one of claims 1 to 3.

Images