Railway vehicle contraband intelligent security management system based on image recognition

Abstract

The application provides an image recognition-based intelligent prohibited article security check management system for rail cars, and relates to the fields of rail transit and security check technology.The system comprises an image acquisition module for acquiring multi-angle, high-definition images of luggage at a rail car security check entrance;an image preprocessing module for sequentially performing denoising on the acquired images using an improved deep singular value decomposition denoising method and data enhancement processing based on a hybrid network;an intelligent recognition and classification module for performing feature extraction and classification on the preprocessed images based on a convolutional neural network with symmetry rotation equivariance, automatically recognizing and classifying prohibited articles;an alarm and processing module for triggering an alarm when a suspected prohibited article is identified and pushing the identification result to a security check personnel terminal for subsequent processing.The system has strong robustness, high recognition accuracy, and can adapt to complex scene changes, thereby improving the intelligent level and safety guarantee capability of rail car security check.

Description

Technical Field

[0001] This invention relates to the fields of rail transit and security inspection technology, and in particular to an intelligent security inspection management system for prohibited items in rail vehicles based on image recognition. Background Technology

[0002] With the rapid development of rail transit, station security checks have become a crucial link in ensuring public safety. Traditional manual security checks rely on the experience and judgment of security personnel, resulting in low efficiency, high false detection rates, and difficulty in handling large crowds. In recent years, with the development of artificial intelligence, image recognition, and deep learning technologies, building intelligent recognition systems for rail vehicle security checks has become a research hotspot.

[0003] However, several challenges remain in actual railcar security inspection environments:

[0004] Image quality is complex: Due to the variety of imaging devices, changes in lighting conditions, and the variety of luggage and packages, the acquired images often suffer from problems such as noise interference, blurring, and occlusion.

[0005] Limited sample size: Some prohibited items have scarce samples, making it difficult to support the training of high-precision supervised learning models;

[0006] The items are diverse in type and form: prohibited items may include knives, firearms, flammable liquids and many other types, with significant differences in shape, color and size, and their placement is not fixed;

[0007] The recognition system is sensitive to changes in the pose of objects: the recognition performance of traditional convolutional neural networks (CNNs) drops significantly when faced with image changes such as rotation, tilting, and partial occlusion.

[0008] Therefore, there is an urgent need for an image recognition and classification system with capabilities such as noise reduction, enhancement, multi-scale adaptation, and rotational recognition to improve the intelligence level and safety assurance capabilities of railcar security inspection. Summary of the Invention

[0009] To address the technical problems of poor image quality, insufficient samples, and unstable recognition in existing railcar security inspection image recognition systems, this invention aims to construct a robust, highly accurate, and adaptable intelligent image recognition system for prohibited items on railcars that can adapt to complex scene changes.

[0010] To achieve the above objectives, the following technical solution is adopted:

[0011] This invention provides an intelligent security inspection and management system for prohibited items on rail vehicles based on image recognition, the system comprising:

[0012] The image acquisition module is used to acquire high-definition images of luggage and items from multiple angles at the security checkpoint entrance of the railcar;

[0013] The image preprocessing module is used to sequentially apply an improved depth singular value decomposition denoising method to the acquired images for denoising and data augmentation processing based on a hybrid network.

[0014] The intelligent recognition and classification module is used to extract features and classify preprocessed images based on a convolutional neural network with symmetric rotation and other invariance, and to automatically identify and classify prohibited items.

[0015] The alarm and processing module is used to trigger an alarm when suspected prohibited items are identified, and push the identification results to the security personnel's terminal for further processing.

[0016] Furthermore, the image preprocessing module includes an image denoising submodule, used to perform the following steps:

[0017] The acquired image is divided into multiple overlapping sub-blocks;

[0018] Sparse coding and dictionary learning are performed on each sub-block to remove noise and preserve key structural features;

[0019] Each sub-block is subjected to adaptive threshold denoising through singular value decomposition, where the threshold parameter is dynamically learned through a neural network;

[0020] All denoised sub-blocks are reconstructed into a complete denoised image by overlapping weighted average.

[0021] Furthermore, the image denoising submodule employs a deep neural network for sparse coding and utilizes a multilayer perceptron to adaptively learn regularization parameters to adapt to different noise types and intensities.

[0022] Furthermore, the image preprocessing module also includes a data augmentation submodule, used to perform the following steps:

[0023] Several images are randomly selected from a set of training images that have undergone denoising.

[0024] Each selected image is downsampled and scaled to a uniform predetermined scale.

[0025] The downsampled images are stitched together in a grid layout to create a new hybrid training image;

[0026] Based on the multi-hot encoded labels of multiple original images, a fusion label corresponding to the new hybrid training image is generated.

[0027] Furthermore, the fused label is a weighted sum of the labels of each original image, and the weight coefficients are determined based on the area ratio or spatial position relationship of each original image in the fused training image.

[0028] Furthermore, the convolutional neural network based on symmetric rotation equivariance includes at least one symmetric rotation equivariance convolutional layer;

[0029] The convolution kernel of the symmetric rotation equivariant convolutional layer is constructed using a discrete circular band parameterization method. Specifically, the region of the two-dimensional convolution kernel is divided into K concentric, non-overlapping circular bands. Each circular band is associated with a trainable parameter, and the trainable parameter value is mapped to all positions within the corresponding circular band using a binary mask matrix. Finally, the parameterization results of all circular bands are superimposed to form a complete convolution kernel, which effectively extracts the contours, edges, and geometric features of contraband items.

[0030] Furthermore, the intelligent recognition and classification module also includes a rotation-equal feature aggregation mechanism;

[0031] The rotation-equal feature aggregation mechanism generates R versions of the discrete circular parameterized convolution kernel at predetermined discrete rotation angles, and merges these rotation versions to form an aggregated convolution kernel, which is used to perform convolution operations on the input feature map and outputs a convolution kernel that fuses multiple rotation angles to enhance the model's feature response capability to rotated, tilted, or partially occluded objects.

[0032] Furthermore, the convolutional neural network consists of a feature extraction backbone network composed of multiple cascaded symmetrically rotated equivariant convolutional blocks;

[0033] Each of the symmetric rotation equivariant convolutional blocks contains at least one symmetric rotation equivariant convolutional layer, a batch normalization layer, and a nonlinear activation function.

[0034] Furthermore, the convolutional neural network, following the feature extraction backbone network, sequentially includes:

[0035] The global adaptive average pooling layer is used to compress the three-dimensional feature map output by the feature extraction backbone network into a one-dimensional feature vector, thereby compressing the extracted multi-scale features into a fixed-length vector.

[0036] The classifier consists of one or more fully connected layers, and a Softmax function is applied to the output of the last fully connected layer to generate a probability distribution of the input image belonging to each prohibited item category.

[0037] Furthermore, the alarm and processing module is used to perform the following steps:

[0038] When the confidence level of the identification exceeds the preset threshold, the location, type and confidence level of the item are automatically marked, and an alarm message is pushed to the security personnel's terminal in real time to initiate the manual verification process.

[0039] Compared with the prior art, the present invention achieves the following beneficial effects:

[0040] 1. This invention introduces an improved deep singular value decomposition (SVD) denoising method in the image preprocessing stage, comprehensively utilizing image segmentation technology, sparse representation theory, and deep neural network architecture. In this method, by locally segmenting the image and combining it with a trainable neural network to replace the traditional orthogonal matching pursuit (OMP) for sparse coding, the ability to preserve image structural information is improved. Simultaneously, a multilayer perceptron (MLP) network is introduced to adaptively learn the regularization parameters, achieving dynamic adaptation to different noise types and intensities, fundamentally improving image clarity and subsequent recognition accuracy.

[0041] 2. To address the problem of insufficient image samples for prohibited items, a grid-based data augmentation method was invented. This method randomly selects samples from the denoised images, performs scale-uniform downsampling, and then stitches them together in a grid pattern to generate new training images. This not only effectively preserves the key structural features of the original images but also enhances the model's robustness and generalization ability to object shape changes, occlusion relationships, and multi-target combinations by simulating complex security inspection scenarios involving multiple items coexisting and multiple scales mixing.

[0042] 3. To overcome the limitations of traditional CNNs in rotation invariance, this invention designs a recognition model that integrates a symmetric rotationally equivariant convolution (SR-Conv) structure, constructing an intelligent recognition and classification mechanism based on symmetric rotationally equivariant convolution. This network constructs rotationally sensitive convolutional layers by parameterizing the convolution kernels with discrete circular bands, exhibiting significant advantages in extracting local shape features and edge directions of images, effectively improving the recognition accuracy of prohibited items placed in arbitrary orientations.

[0043] 4. This invention further introduces a local rotation feature aggregation mechanism, which enhances the model's feature response capability in scenarios such as angled knives, tilted liquid containers, and side-mounted firearms through multi-angle rotation weight fusion. This ensures that the model can still accurately extract core recognition features even when the viewpoint is deflected or partially occluded, greatly improving the stability and reliability of recognition.

[0044] In summary, this invention addresses the practical needs of railcar security inspections by proposing a complete intelligent image recognition and classification processing mechanism, from image quality optimization and sample enhancement to recognition structure design and feature robustness improvement. This significantly enhances the practicality, accuracy, and engineering deployment capabilities of the prohibited items detection system.

[0045] It should be understood that the description in the Summary of the Invention is not intended to limit the key or essential features of the embodiments of the present invention, nor is it intended to restrict the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0046] The above and other features, advantages, and aspects of the various embodiments of the present invention will become more apparent from the accompanying drawings and the following detailed description. The drawings are provided for a better understanding of the invention and are not intended to limit the invention. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0047] Figure 1 This is a schematic diagram of a module of an intelligent security inspection management system for prohibited items on a railcar based on image recognition, according to an embodiment of the present invention.

[0048] Figure 2 This is a flowchart illustrating an intelligent security inspection management system for prohibited items on a railcar based on image recognition, according to an embodiment of the present invention.

[0049] Figure 3 This is a schematic diagram of the structure of a convolutional neural network based on symmetric rotation equivariance according to an embodiment of the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0052] Figure 1 This is a schematic diagram of a module of an intelligent security inspection management system for prohibited items on a railcar based on image recognition, according to an embodiment of the present invention. Figure 2 This is a flowchart illustrating an image recognition-based intelligent security inspection management system for prohibited items on a railcar, according to an embodiment of the present invention. Figure 1 and Figure 2 As shown, a smart security inspection and management system 100 for prohibited items on rail vehicles based on image recognition is disclosed. The system 100 includes:

[0053] The image acquisition module 110 is used to acquire multi-angle, high-definition images of luggage and items at the security checkpoint entrance of the railcar;

[0054] An image acquisition module 110 is deployed at the security checkpoint of the railcar, employing a high-definition image acquisition system, including a high-resolution camera, an X-ray security scanner, multispectral imaging equipment, and a CT scanner for detecting special items when necessary. This ensures multi-angle, high-definition imaging of passengers' luggage, backpacks, bags, and other items. For larger, more complex, or obstructed items, the system can automatically switch to a higher resolution mode or combine multimodal imaging technology to achieve comprehensive perception of the item's shape, structure, and material, providing detailed image data for subsequent processing.

[0055] The image preprocessing module 120 is used to sequentially perform denoising on the acquired image using an improved depth singular value decomposition denoising method and data augmentation processing based on a hybrid network.

[0056] To improve the clarity and recognition quality of the original image, the system performs a series of preprocessing operations on the acquired image data. Optionally, in one embodiment, the image preprocessing module 120 includes an image denoising submodule 121 and a data augmentation submodule 122.

[0057] The image denoising submodule 121 is used to denoise the acquired image using an improved depth singular value decomposition denoising method.

[0058] Considering that images are often subject to noise interference (such as X-ray speckle and motion blur) in the security inspection environment of railcars, this system adopts an improved deep singular value decomposition denoising method to decompose and reconstruct images. This method is specifically designed to eliminate image noise caused by factors such as complex environment, equipment vibration and interference radiation, thereby improving the robustness and effectiveness of the contraband identification model.

[0059] This algorithm integrates image segmentation techniques, sparse representation theory, and deep network structures. Its core idea is to suppress unstructured noise in images to the maximum extent possible while preserving the integrity of the image's structural information. Its key innovation lies in replacing the traditional Orthogonal Matching Pursuit (OMP) with a trainable neural network module for sparse coding, while simultaneously introducing a Multilayer Perceptron (MLP) network to adaptively learn the regularization parameters.

[0060] Preferably, in an embodiment of the present invention, a lightweight convolutional neural network (CNN) module is used to replace the traditional orthogonal matching pursuit (OMP) algorithm to approximate the solution of the sparse coding matrix. The CNN module uses noisy image sub-blocks. Given the input, directly output its sparse representation. The network consists of a 3×3 convolutional layer, a ReLU activation layer, and a 1×1 convolutional layer connected sequentially, and is jointly trained with the main denoising network to minimize reconstruction error. Optimize for the target.

[0061] Furthermore, the image denoising submodule 121 is used to perform the following steps 2.1: segmenting the acquired image into multiple overlapping sub-blocks; performing sparse coding and dictionary learning on each sub-block to remove noise and retain key structural features; performing adaptive threshold denoising on each sub-block through singular value decomposition, wherein the threshold parameter is dynamically learned through a neural network; and reconstructing all denoised sub-blocks into a complete denoised image through overlapping weighted averaging. Here, a deep neural network is used for sparse coding, and a multilayer perceptron is used to adaptively learn regularization parameters to adapt to different noise types and intensities. Specifically, this includes the following sub-steps:

[0062] 2.1.1 Image Segmentation and Modeling

[0063] Input security check image Divided into multiple overlapping sub-blocks Each sub-block represents information about a local region in the image. The denoising task for each sub-block can be modeled as the following optimization problem:

[0064]

[0065] The denoised sub-block data retains key structural features, such as the outline of the tool and the outline of the contraband. : The original sub-block data, from the luggage inspection images of the railcar, represents the first sub-block in the data to be denoised. i Individual blocks. : Noise-reduced sub-block data, that is, a cleaner data block obtained through optimization calculation. D : Dictionary matrix, representing the data basis used for sparse coding, which can learn the structural features of the data in order to better reconstruct the data. : Sparse coding matrix, representing In the dictionary The sparse representation on the dictionary bases means that each sub-block of data is represented by a linear combination of dictionary bases, which preserves important information while removing noise. : Represents the reconstruction error, calculated using the Frobenius norm. rather than sparse representation The goal is to minimize this error to ensure that the data retains as much of its original information as possible after denoising. Regularization parameter, used to control sparsity, determines the sparsity during optimization. The sparsity requirement is such that the larger the value, the stronger the sparsity (i.e., the fewer non-zero elements), which can better remove noise. Norm constraints ensure sparse representation It has fewer non-zero elements to enhance the sparsity of the data, thus making noise removal more effective.

[0066] In some optional embodiments, the dictionary matrix D D is a trainable parameter matrix that is jointly optimized and learned with the CNN module using gradient descent. In some alternative embodiments, the dictionary matrix D can also be obtained by pre-training on the main dataset using the K-SVD algorithm.

[0067] The core idea of ​​this optimization formula is to utilize sparse coding methods and a dictionary. D To approximate the original data At the same time, apply Regularization is used to ensure sparsity, so that the denoised data It can retain important information while effectively removing environmental noise and equipment interference.

[0068] 2.1.2 Singular Value Decomposition and Thresholding Denoising

[0069] To further improve local denoising performance, the algorithm performs denoising on each sub-block. Perform Singular Value Decomposition (SVD):

[0070]

[0071] and : These are the left and right singular vector matrices obtained from singular value decomposition, representing the direction of the image structure. : Singular value matrix, representing the distribution of image energy.

[0072] Noise effects are removed by adaptive threshold truncation of singular values:

[0073]

[0074] It is the denoised sub-block. It is the singular value matrix after denoising. This indicates that singular values ​​are truncated by a threshold, τ, which is adaptively learned through a multilayer perceptron (MLP) network model.

[0075] 2.1.3 Image Reconstruction

[0076] All denoised sub-blocks Then, by using an overlapping weighted average, the complete image is reconstructed.

[0077]

[0078] in, This is the final denoised data;N It represents the total number of sub-blocks.

[0079] The data augmentation submodule 122 is used to perform data augmentation processing on the denoised original image based on a hybrid network. Specifically, it performs the following step 2.2: Data Augmentation Based on Mesh Hybridization.

[0080] To address the issues of limited number and uneven distribution of images of prohibited items in railcar security inspection scenarios, this system introduces a grid hybrid data augmentation method to improve the generalization ability and robustness of the image recognition model under different changes in the shape, size, and position of the items.

[0081] The specific approach involves randomly selecting several images from the denoised original image samples. Each selected image is then downsampled and scaled to a uniform predetermined scale. These downsampled images are then stitched together in a grid pattern to create a new mixed training image. This not only preserves the structural information of the original contraband but also simulates the coexistence of complex backgrounds and interfering objects in various real-world security inspection scenarios, enhancing the model's adaptability to multiple scales and combinations.

[0082] The mesh blending generation process is as follows:

[0083]

[0084]

[0085] Where s is the downsampling factor, which controls the degree of reduction in data resolution and helps the model learn multi-scale information. The downsampling factor s can be randomly selected in the interval [2, 4] to scale the image to different uniform scales. : indicates a downsampling operation, which can typically be average pooling, max pooling, or interpolation methods. The downsampled data is then reduced in resolution to form new training samples. n Number of samples. The mixed data is composed of multiple downsampled data sample grids, which enables the model to learn different pattern combinations. : No. i Data after downsampling of each sample. : Represents the grid stitching operation, which combines multiple samples into a new data sample according to certain rules. Among them, the grid stitching operation Defined as: n Zhang downsampled image ,according to The grid layout is spliced ​​together (if) n (for square numbers), or according to 2×ceil ( The approximate grid layout of the images is stitched together to generate new mixed training images. .

[0086] In terms of label processing, a fused label corresponding to the new mixed training image is generated based on the multi-hot encoded labels of multiple original images. The fused label is a weighted sum of the labels of the original images, with the weighting coefficients determined based on the area proportion or spatial relationship of each original image within the mixed training image.

[0087] Due to new images The image region contains multiple prohibited or common items, therefore its label is a weighted fusion of multiple original labels: the labels of all categories appearing in the mixed data are combined as the mixed data label, and the calculation formula is:

[0088]

[0089] : Represents the label of the original sample, using multi-hot encoding, indicating that the image may contain multiple types of prohibited items simultaneously. : is a sample The contribution weight, used to fuse label information from different data samples, is usually related to the proportion of image area or positional relationship. Generate image The fusion tags retain the original image category information while reflecting the actual situation of multiple items coexisting.

[0090] This method can effectively improve the model's ability to recognize real-world challenges such as abnormal placement, partial occlusion, and complex backgrounds in image recognition tasks, and is a key component of the intelligent railcar security inspection system for dealing with the detection of diverse prohibited items.

[0091] The intelligent recognition and classification module 130 is used to extract features and classify preprocessed images based on a convolutional neural network with symmetric rotation and equivariance, and to automatically identify and classify prohibited items.

[0092] After image preprocessing and enhancement, the intelligent recognition and classification module 130 of this system is used to intelligently identify and classify prohibited items. Therefore, this invention constructs an intelligent recognition and classification mechanism based on symmetric rotation equivariant convolutional neural network (SR-Equivariant CNN) to automatically identify whether there are prohibited items (such as knives, firearms, explosives, flammable liquids, etc.) on the railcar.

[0093] 3.1 Deep Feature Extraction Based on Fusion Rotation Isovariability

[0094] To enhance the system's recognition capabilities in practical applications under scenarios involving multiple viewpoints, rotational shooting, and occlusion, Symmetrical Rotation Equivariant Convolution (SR-Conv) technology is used to replace the standard convolutional kernel of traditional convolutional neural networks. SR-Conv can capture local visual features with rotational symmetry and spatial diversity, such as the outline shape, color distribution, material texture, and internal structure density of contraband items.

[0095] The intelligent recognition and classification module 130 of this invention is based on a meticulously designed symmetric rotational isovariant convolutional neural network. This network structure aims to effectively extract features that are insensitive to rotation and viewpoint changes from preprocessed security inspection images, thereby achieving stable recognition of prohibited items in any posture.

[0096] like Figure 3 The diagram shown is a schematic representation of the specific structure of a convolutional neural network based on symmetric rotation equivariance according to an embodiment of the present invention. The specific hierarchical structure and processing procedure of the network are as follows:

[0097] 1. Input layer

[0098] Input: Receives the output image from the image preprocessing module. The image size is standardized to 224 pixels × 224 pixels × 3 channels (RGB format).

[0099] Function: As the starting point of the network, it converts image data into tensor format for subsequent layers to process.

[0100] 2. Feature Extraction Backbone Network

[0101] This part consists of multiple cascaded symmetrical rotational equivariant convolutional blocks, which progressively extract and refine features.

[0102] 2.1) First Symmetric Rotation Equivariant Block 1

[0103] Core layer: At least one symmetric rotational equivariant convolutional layer. Convolutional kernel construction: The convolutional kernel of this layer is not a traditional matrix with freely learnable parameters, but is constructed using a discrete circular strip parameterization method. Specifically, the region of the 7×7 two-dimensional convolutional kernel is divided into... K One (e.g.) K =3) Concentric, non-overlapping circular bands (such as inner ring, middle ring, outer ring), each circular band Associated with a trainable parameter And through a predefined binary mask matrix The trainable parameter values ​​are mapped to all positions within the corresponding circular bands. Finally, the parameterized results of all circular bands are superimposed to form a complete convolutional kernel Θ, effectively extracting the contours, edges, and geometric features of contraband items. This structure inherently possesses rotational symmetry. Convolution operation: Convolution operations are performed using the constructed convolutional kernel. To capture multi-angle information, multiple (e.g., 16) of these convolutional kernels can be used in the initial stage. Isovariability guarantee: The design of this layer guarantees that when the input image undergoes discrete rotation (e.g., rotation by an integer multiple of 90°), its output feature map will undergo a predictable transformation (e.g., cyclic shift), which is known as "isovariability."

[0104] The auxiliary layers include: a batch normalization layer, which normalizes the feature maps output by the convolution to accelerate training and improve stability; and a ReLU activation layer, which introduces non-linearity.

[0105] Output: Feature map size is 112×112×16 (assuming downsampling was performed using convolution or pooling with stride of 2).

[0106] b) Second symmetric rotationally equivariant convolution block (SR-Equivariant Block 2)

[0107] Structure: Similar to the first block, but using more convolutional kernels (e.g., 32) and smaller kernel size (e.g., 5×5) to learn more complex features.

[0108] Rotational Isotropic Feature Aggregation Mechanism: This block introduces a key feature aggregation step. For each circular band scale... k Generate convolution kernels at this scale R Discrete rotation angles (e.g.) R =4 (corresponding to versions at 0°, 90°, 180°, 270°) Then, these rotated versions are merged to generate a composite convolution kernel: .use Convolutions enable the network to directly aggregate information from multiple angles, significantly enhancing its robustness to rotation.

[0109] Output: The feature map size is further reduced, for example, to 56×56×32.

[0110] c) Subsequent Blocks

[0111] One or two similar symmetric rotation convolutional blocks can be stacked to gradually increase the number of convolutional kernels (e.g., 64, 128) while reducing the feature map space size to construct deep feature representations with strong semantic information.

[0112] Max pooling layers can be selectively added after each block to further enhance the scale invariance of features and reduce computational cost.

[0113] 3. Global Feature Aggregation & Classifier

[0114] Global Adaptive Average Pooling Layer: Receives the feature map (e.g., 7 × 7 × 128) output from the last convolutional block. This layer averages all activations across each feature channel, thus compressing a feature map of arbitrary size into a fixed-length feature vector (128-dimensional in this example). A This operation enables the network to process inputs of different sizes and focus on global statistics of features.

[0115] Fully connected classification layer:

[0116] Structure: Consists of one or more fully connected layers. For example, it begins with a hidden layer that maps a 128-dimensional vector to a 64-dimensional vector, and uses the ReLU activation function and Dropout regularization (e.g., a dropout rate of 0.5) to prevent overfitting.

[0117] Output layer: The number of neurons in the last fully connected layer is equal to the number of categories of prohibited items (e.g., 5 categories: firearms, knives, flammable liquids, explosives, and non-prohibited items). This layer outputs an unnormalized score vector.

[0118] Softmax activation function: Apply the Softmax function to the score vector of the output layer, transforming it into a probability distribution. . Each element in the array represents the probability that the input image belongs to the corresponding category.

[0119] 4. Training process:

[0120] Loss function: The network is trained using the standard cross-entropy loss function used in classification tasks.

[0121] Optimizer: Optimization algorithms such as Adam or stochastic gradient descent can be used.

[0122] Weight initialization: The weights of convolutional and fully connected layers can be initialized using the Xavier or He initialization methods.

[0123] The preprocessed image is passed through the aforementioned layers sequentially to obtain the final classification probability. During the training phase, the loss is calculated based on the predicted probability and the true label, and parameters, including the parameters of each circular band, are updated via backpropagation. and fully connected layer weights , During the inference (deployment) phase, the category with the highest probability is output as the recognition result.

[0124] More specifically:

[0125] First, a rotation-sensitive parameter structure is constructed by dividing the convolution kernel into multiple non-overlapping discrete circular regions. Each circular region corresponds to a trainable parameter. And through a binary mask matrix Map it to kernel space:

[0126]

[0127] in, This represents the final convolution kernel parameter matrix. K It represents the number of discrete circular bands, with each band representing a trainable parameter. It is the first k Each band has trainable parameters that reflect the characteristics of objects at different scales and densities. It is a binary index matrix, ensuring Only in the Application within a specific area. The set concatenation operation represents the weighted summation of all weighted regions to form the final convolutional kernel. This can be viewed as a "fusion" of multiple local regions, forming an overall convolutional kernel with rotational structure perception capabilities.

[0128] Among them, the binary mask matrix Generate in the following way: for a size of × The convolution kernel is used to calculate its center coordinates. , For any position (p, q) within the kernel, calculate its Euclidean distance to the center. According to the preset circular band radius threshold [ , , ..., If d falls within the range of the k-th circular zone, , Inside, then = 1, otherwise 0.

[0129] In one embodiment, for example, for a 7×7 convolution kernel, a radius threshold [0, 1.5, 3.5, 5.0] can be set to form 3 ( K =3) Concentric circles.

[0130] This structure can effectively represent angle-sensitive features such as the sharp edge of a knife, the curved contour of a liquid bottle, and the local geometry of a firearm.

[0131] 3.2 Rotation-based Isovariant Convolution Operation

[0132] Furthermore, the intelligent recognition and classification module also includes a rotation-equivariant feature aggregation mechanism, which generates a feature aggregation function for the discrete circular band parameterized convolution kernel. R Version under a predetermined discrete rotation angle These rotated versions are then merged to form a coherent convolution kernel. It is used to perform convolution operations on the input feature map and output convolution kernels that fuse multiple rotation angles to enhance the model's feature response capability to rotated, tilted, or partially occluded objects.

[0133] The convolutional neural network consists of a feature extraction backbone network composed of multiple cascaded symmetric rotation equivariant convolutional blocks; each symmetric rotation equivariant convolutional block contains at least one symmetric rotation equivariant convolutional layer, a batch normalization layer, and a nonlinear activation function.

[0134] By parameterizing the rotating isomorphic kernel Feature maps of each channel applied to the input security inspection image Obtain the output feature map :

[0135]

[0136] in, F Output feature map; It is the first of the input feature maps j One channel; This represents the convolution operation. : No. j The rotationally equivariant convolution kernels corresponding to each channel are obtained from the previous step 3.1 and possess rotational direction awareness capabilities. The kernel parameters for each channel can be learned independently, thereby adapting to rotational changes of different types of features. This represents a non-linear activation function (such as ReLU).

[0137] A fully convolutional neural network is constructed using symmetric rotational equivariant convolutional layers. This architecture does not perform reshaping or flattening operations in the spatial dimension to maintain rotational equivariant features.

[0138] 3.3 Local Rotation Feature Aggregation

[0139] To enhance the system's response to images rotated at multiple angles, a rotation-equal feature aggregation mechanism is further defined:

[0140]

[0141] Representing the Rotational isomorphic features at various scales. Indicates rotation angle Convolution kernel under the action. Discrete rotation step size (e.g.) express (rotational isomorphism).

[0142] Among them, the convolution kernel at discrete rotation angles r The next version It is achieved by using the original convolution kernel. Around its center r This is achieved through a 90° rotation operation. This rotation operation preferably uses nearest-neighbor interpolation to preserve the properties of the binary mask.

[0143] This method enables the system to maintain accurate feature extraction capabilities when processing images such as knives placed on their sides or liquid bottles placed at an angle.

[0144] 3.4 Global Adaptive Pooling and Classification

[0145] Following the feature extraction backbone network, the convolutional neural network includes, in sequence, a global adaptive average pooling layer and a classifier.

[0146] The global adaptive average pooling layer is used to compress the 3D feature map output by the feature extraction backbone network into a 1D feature vector, thereby compressing the extracted multi-scale features into a fixed-length vector. Finally, after feature extraction, global adaptive average pooling is used to compress the multi-scale rotational features to obtain a vector of uniform length. A As input to the classifier:

[0147]

[0148] in, A The feature vector, after rotational feature extraction and global adaptive pooling, is input into the classifier. This represents the height and width of the feature map. Representative feature map at location The activation value at point 1, i.e., the output feature after symmetric rotation and equal variation convolution, is at point 2. h Okay, number w The value at the column position, i.e.:

[0149]

[0150] in, Representing the Feature maps of each input channel. It is the weighted convolution kernel obtained in step 3.3, which incorporates multiple rotation angle versions. This represents the convolution operation. This represents a non-linear activation function (such as ReLU).

[0151] The classifier consists of one or more fully connected layers, and a Softmax function is applied to the output of the last fully connected layer to generate a probability distribution of the input image belonging to each prohibited item category.

[0152] Ultimately, the classifier bases its decisions on the vectors. Z Predict the presence of prohibited items in an image and their category to achieve automatic intelligent classification and identification:

[0153]

[0154] The class probability vector output by the model represents the probability that the current image belongs to each class (such as firearms, knives, flammable liquids, non-contraband, etc.). The weight matrix of the classifier, which learns the relationship between each class and features. The mapping relationship; The classifier's bias term is used to adjust the classification threshold offset for different classes. : Activation function, which transforms a linear output into a probability distribution.

[0155] For example, if the model supports The categories (firearms, knives, flammable liquids, explosives, non-contraband) are defined in the image, and the classification result is output after identification.

[0156]

[0157] This indicates that the system identifies the substance as "flammable liquid" with an 83% probability, which is highly suspicious, and the system will trigger an automatic alarm mechanism.

[0158] This identification and classification module has good rotational robustness, multi-scale adaptability and high-precision feature perception capabilities, and can be widely used in actual railcar security inspections to detect prohibited items under various shapes, postures and occlusion conditions.

[0159] The alarm and processing module 140 is used to trigger an alarm when suspected prohibited items are identified, and push the identification results to the security personnel's terminal for further processing.

[0160] The alarm and processing module 140 performs the following steps: When the probability (confidence level) of the highest identified category exceeds a preset threshold (for example, this threshold can be set between 0.7 and 0.9), an alarm mechanism is triggered, automatically marking the item's location, type, and confidence level, and pushing alarm information to the security personnel's terminal in real time to initiate the manual verification process. Specifically:

[0161] When the system identifies a suspected contraband item in an image, it immediately triggers an alarm mechanism, automatically recording and marking the item's location in the image. Simultaneously, it pushes the item's corresponding type label, confidence score, and visual feature summary to the security personnel's terminal in real time. This mechanism, based on the intelligent identification and classification results from step three, ensures the accuracy and timeliness of the alarm.

[0162] Security personnel can manually verify suspicious items based on the detailed analysis results output by the system. This includes confirming the actual attributes of the items, verifying passenger information, and conducting further bag checks, ultimately forming a complete closed-loop security inspection process. This process not only improves security inspection efficiency but also significantly reduces the risk of missed or false detections, ensuring the safe operation of the railcar.

[0163] The embodiments of this invention construct a complete security inspection management system from image optimization to intelligent recognition. First, improved deep singular value decomposition denoising and grid-mixed data enhancement effectively improve image quality and sample diversity. Then, a network structure based on symmetric rotational equivariant convolution is designed. Through discrete circular parameterized convolution kernels and rotation feature aggregation mechanisms, the model exhibits strong robustness to complex postures such as object rotation and occlusion. Thus, this invention systematically solves the three major problems in railcar security inspection: poor image quality, limited sample size, and low recognition stability, significantly improving the accuracy, generalization ability, and practicality of prohibited item recognition.

[0164] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0165] It should also be noted that, in the embodiments of this application, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0166] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined in the embodiments of this application may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown in this application, but is to be accorded the widest scope consistent with the principles and novel features disclosed in the embodiments of this application.

Claims

1. An image recognition-based railcar contraband intelligent security check management system, characterized in that, The system includes: The image acquisition module is used to acquire high-definition images of luggage and items from multiple angles at the security checkpoint entrance of the railcar; The image preprocessing module is used to sequentially apply an improved depth singular value decomposition denoising method to the acquired images for denoising and data augmentation processing based on a hybrid network. The intelligent recognition and classification module is used to extract features and classify preprocessed images based on a convolutional neural network with symmetric rotation and other invariance, and to automatically identify and classify prohibited items. The alarm and processing module is used to trigger an alarm when suspected prohibited items are identified, and push the identification results to the security personnel's terminal for further processing; The image preprocessing module includes an image denoising submodule, used to perform the following steps: The acquired image is divided into multiple overlapping sub-blocks; Sparse coding and dictionary learning are performed on each sub-block to remove noise and preserve key structural features; Each sub-block is subjected to adaptive threshold denoising through singular value decomposition, where the threshold parameter is dynamically learned through a neural network; All denoised sub-blocks are reconstructed into a complete denoised image by overlapping weighted average; wherein the input security image is segmented into a plurality of overlapping sub-blocks each sub-block representing information of a local region in the image, the denoising task of the sub-blocks being modeled as the following optimization problem: ； In the formula, The denoised sub-block data retains key structural features, i.e., the cleaner data blocks obtained through optimized calculations; : The original sub-block data, from the baggage inspection image of the railcar, represents the i-th sub-block in the data to be denoised; D: Dictionary matrix, representing the data basis used for sparse coding, which can learn the structural features of the data in order to better reconstruct the data; : Sparse coding matrix, representing In the dictionary The sparse representation on the dictionary bases means that each sub-block of data is represented by a linear combination of dictionary bases, which preserves important information while removing noise. : Represents the reconstruction error, calculated using the Frobenius norm. rather than sparse representation The goal is to minimize this error to ensure that the data retains as much of its original information as possible after denoising. Regularization parameter, used to control sparsity, determines the sparsity during optimization. The sparsity requirement is such that the larger the value, the stronger the sparsity, thus better removing noise; Norm constraints ensure sparse representation It has fewer non-zero elements to enhance the sparsity of the data, thus making noise removal more effective; The convolutional neural network based on symmetric rotation equivariance includes at least one symmetric rotation equivariance convolutional layer; The convolution kernel of the symmetric rotation equivariant convolutional layer is constructed through discrete circular band parameterization. Specifically, the region of the two-dimensional convolution kernel is divided into K concentric, non-overlapping circular bands. Each circular band is associated with a trainable parameter, and the trainable parameter value is mapped to all positions within the corresponding circular band through a binary mask matrix. Finally, the parameterization results of all circular bands are superimposed to form a complete convolution kernel, so as to effectively extract the contour, edge and geometric features of contraband items. The intelligent recognition and classification module also includes a rotational isovariant feature aggregation mechanism; The rotation-equal feature aggregation mechanism generates R versions of the discrete circular parameterized convolution kernel at predetermined discrete rotation angles, and merges these rotation versions to form an aggregated convolution kernel, which is used to perform convolution operations on the input feature map and outputs a convolution kernel that fuses multiple rotation angles to enhance the model's feature response capability to rotated, tilted, or partially occluded objects.

2. The system of claim 1, wherein, The image denoising submodule uses a deep neural network for sparse coding and utilizes a multilayer perceptron to adaptively learn regularization parameters to adapt to different noise types and intensities.

3. The system of claim 1, wherein, The image preprocessing module further includes a data augmentation submodule, used to perform the following steps: Several images are randomly selected from a set of training images that have undergone denoising. Each selected image is downsampled and scaled to a uniform predetermined scale. The downsampled images are stitched together in a grid layout to create a new hybrid training image; Based on the multi-hot encoded labels of multiple original images, a fusion label corresponding to the new hybrid training image is generated.

4. The system of claim 3, wherein, in, The fused label is a weighted sum of the labels of each original image, and the weighting coefficients are determined based on the area ratio or spatial position relationship of each original image in the fused training image.

5. The system of claim 1, wherein, The convolutional neural network consists of a feature extraction backbone network composed of multiple cascaded symmetrical rotational equivariant convolutional blocks; Each of the symmetric rotation equivariant convolutional blocks contains at least one symmetric rotation equivariant convolutional layer, a batch normalization layer, and a nonlinear activation function.

6. The system of claim 5, wherein, The convolutional neural network, following the feature extraction backbone network, includes, in sequence: The global adaptive average pooling layer is used to compress the three-dimensional feature map output by the feature extraction backbone network into a one-dimensional feature vector, thereby compressing the extracted multi-scale features into a fixed-length vector. The classifier consists of one or more fully connected layers, and a Softmax function is applied to the output of the last fully connected layer to generate a probability distribution of the input image belonging to each prohibited item category.

7. The system of claim 1, wherein, The alarm and processing module is used to perform the following steps: When the confidence level of the identification exceeds the preset threshold, the location, type and confidence level of the item are automatically marked, and an alarm message is pushed to the security personnel's terminal in real time to initiate the manual verification process.

Images