An article real-time detection method, device, equipment and storage medium

Abstract

The application discloses an article real-time detection method and device, equipment and a storage medium, and relates to the technical field of security check information management. The method comprises the following steps: performing feature extraction on a current X-ray image, inputting a first feature value extracted to an optimized fast spatial pyramid pooling network to perform multi-scale feature fusion to obtain a second feature value; inputting the second feature value to an encoding network to perform encoding, merging the encoded feature value with the feature value in the feature extraction process, and inputting the merged feature value to a pre-created decoding network to decode the merged feature value to obtain a decoded feature value; inputting the decoded feature value to a trained feedforward neural network to detect contraband in the current X-ray image, and obtaining a contraband detection result containing a target category, a target frame and a score. The application can improve the real-time detection efficiency of contraband, improve the detection accuracy, and reduce the false detection rate.

Description

Technical Field

[0001] This application relates to the field of security inspection information management technology, and in particular to a method, device, equipment and storage medium for real-time detection of items. Background Technology

[0002] Baggage flow, passenger flow, and information flow are the three main processes ensuring the safe operation of passenger travel venues. Among them, the safety, smoothness, and efficiency of baggage flow directly affect the regularity of travel schedules and passenger satisfaction. Currently, to ensure smooth and safe travel for passengers, baggage security screening systems are usually installed at travel venues such as train stations, high-speed rail stations, and airports to detect prohibited items. X-ray prohibited item detection, as a primary technical method for baggage security inspection, is of great significance for maintaining public safety and protecting citizens' personal and property safety. Furthermore, with the rise and development of deep learning technology, combined with the practical needs of security inspection tasks, intelligent X-ray prohibited item detection has become one of the research hotspots in the field of security prevention.

[0003] Currently, X-ray imaging systems employ a line-scanning method. Their internal high-sensitivity X-ray array detectors, driven by a mechanical scanning device, scan items layer by layer. After the transmitted X-ray signals are detected and processed, the acquired data is used to reconstruct an image, thereby capturing items within luggage. However, unlike conventional target detection in RGB (Red, Green, Blue, a color standard) images, X-ray images inherently differ from RGB images due to severe class imbalance, monotonous color, and a lack of texture information. Furthermore, current security checks heavily rely on manual screening and judgment of X-ray images, making it difficult to identify deliberately concealed contraband in overlapping and cluttered luggage.

[0004] Furthermore, factors such as changes in spatial scale, complex background interference, similar backgrounds, lack of color information, and obscuring by contraband can make it difficult to detect contraband. Additionally, to facilitate convenient travel, real-time detection is necessary during baggage screening to prevent baggage accumulation. Moreover, current X-ray image detection methods rely on manual feature recognition and classifier classification, using a sliding window based on location information to distinguish between contraband and non-contraband. The high subjectivity of manual feature recognition may lead to insufficient robustness, a high false positive rate, and low efficiency. Summary of the Invention

[0005] In view of this, the purpose of this application is to provide a method, apparatus, device, and storage medium for real-time detection of contraband, which can improve the efficiency of real-time detection, increase the accuracy of detection, and reduce the false detection rate. The specific solution is as follows:

[0006] Firstly, this application discloses a method for real-time detection of articles, including:

[0007] Feature extraction is performed on the current X-ray image in the real-time received X-ray video stream, and the extracted first feature value is input into the optimized fast spatial pyramid pooling network for multi-scale feature fusion to obtain the second feature value;

[0008] The second feature value is input into a pre-created encoding network for encoding, and the encoded feature value is merged with the feature value in the feature extraction process to obtain the merged feature value;

[0009] The merged feature values ​​are input into a pre-created decoding network to decode the merged feature values, resulting in decoded feature values; both the encoding network and the decoding network are attention-based networks.

[0010] The decoded feature values ​​are input into a trained feedforward neural network to detect contraband in the current X-ray image, resulting in a contraband detection result that includes the target category, the bounding box, and the score.

[0011] Optionally, the step of extracting features from the current X-ray image in the real-time received X-ray video stream and inputting the extracted first feature value into an optimized fast spatial pyramid pooling network for multi-scale feature fusion to obtain a second feature value includes:

[0012] Acquire X-ray images from the real-time received X-ray video stream to obtain the current X-ray image;

[0013] The current X-ray image is input into a feature extraction network to extract features from the current X-ray image and obtain a first feature value; the feature extraction network is a network created based on a first number of convolutional networks and a second number of C3 networks;

[0014] The first feature value is input into the optimized fast spatial pyramid pooling network to perform multi-scale feature fusion on the first feature value to obtain the second feature value.

[0015] Optionally, the step of inputting the current X-ray image into a feature extraction network to extract features from the current X-ray image and obtain a first feature value includes:

[0016] The current X-ray image is scaled to a preset size to obtain a scaled image;

[0017] The scaled image is input into the feature extraction network to extract features from the scaled image and obtain a first feature value;

[0018] Wherein, the number of channels corresponding to the first feature value is greater than the number of channels of the scaled image, and the pixel value corresponding to the first feature value is less than the pixel value of the scaled image.

[0019] Optionally, the step of inputting the second feature value into a pre-created encoding network for encoding, and merging the encoded feature value with the feature value in the feature extraction process to obtain the merged feature value includes:

[0020] The second feature value is input into a pre-created encoding network to encode the second feature value, thereby obtaining the encoded feature value;

[0021] The encoded feature value is concatenated with multiple feature values ​​from the feature extraction process to obtain multiple concatenated feature values. These multiple concatenated feature values ​​are then merged with the encoded feature value to obtain merged feature values.

[0022] Optionally, the step of concatenating the encoded feature value with multiple feature values ​​obtained in the feature extraction process to obtain multiple concatenated feature values, and merging the multiple concatenated feature values ​​with the encoded feature value to obtain merged feature values, includes:

[0023] The encoded feature value is concatenated with the third feature value output by the first C3 network during the feature extraction process to obtain the first concatenated feature value; the third feature value has the same number of channels or pixel value as the encoded feature value.

[0024] The first concatenated feature value is convolved with a pre-created first convolutional network and a second convolutional network respectively to obtain the first convolutional feature value and the second convolutional feature value.

[0025] The first convolutional feature value and the second convolutional feature value are added point by point to obtain the first summed feature value;

[0026] The first summed feature value is concatenated with the fourth feature value output by the second C3 network during the feature extraction process to obtain the second concatenated feature value; the fourth feature value has the same number of channels or pixel value as the first summed feature value.

[0027] The concatenated feature values ​​are convolved with the pre-created third and fourth convolutional networks respectively to obtain the feature values ​​after the third and fourth convolutions.

[0028] The feature values ​​after the third convolution and the feature values ​​after the fourth convolution are added point by point to obtain the second summed feature value. The first summed feature value, the second summed feature value and the encoded feature value are then merged to obtain the merged feature value.

[0029] Optionally, the real-time item detection method further includes:

[0030] Collect historical X-ray images to obtain an X-ray image dataset;

[0031] The initial feedforward neural network is trained using the X-ray image dataset to obtain the trained feedforward neural network;

[0032] The feedforward neural network consists of a fully connected network layer, a linear rectified function layer, and a fully connected network layer.

[0033] Optionally, the real-time item detection method further includes:

[0034] Contraband items in each historical X-ray image of the X-ray image dataset are tagged to obtain a real label dataset;

[0035] The predicted value dataset generated after the initial feedforward neural network training is matched with the real label dataset to obtain the matching result;

[0036] A loss function is calculated based on the matching results to evaluate the performance of the trained feedforward neural network.

[0037] Secondly, this application discloses a real-time article detection device, comprising:

[0038] The feature extraction module is used to extract features from the current X-ray image in the real-time received X-ray video stream;

[0039] The feature fusion module is used to input the extracted first feature value into the optimized fast spatial pyramid pooling network for multi-scale feature fusion to obtain the second feature value.

[0040] The encoding module is used to input the second feature value into a pre-created encoding network for encoding;

[0041] The feature merging module is used to merge the encoded feature values ​​with the feature values ​​in the feature extraction process to obtain merged feature values;

[0042] The decoding module is used to input the merged feature values ​​into a pre-created decoding network to decode the merged feature values ​​and obtain decoded feature values; both the encoding network and the decoding network are attention mechanism networks;

[0043] The contraband detection module is used to input the decoded feature values ​​into a trained feedforward neural network to detect contraband in the current X-ray image and obtain a contraband detection result including target category, target box and score.

[0044] Thirdly, this application discloses an electronic device, including a processor and a memory; wherein, when the processor executes a computer program stored in the memory, it implements the aforementioned real-time item detection method.

[0045] Fourthly, this application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned real-time item detection method.

[0046] As can be seen, this application first extracts features from the current X-ray image in the real-time received X-ray video stream, and inputs the extracted first feature value into an optimized fast spatial pyramid pooling network for multi-scale feature fusion to obtain a second feature value. Then, the second feature value is input into a pre-created encoding network for encoding, and the encoded feature value is merged with the feature value in the feature extraction process to obtain a merged feature value. The merged feature value is then input into a pre-created decoding network to decode the merged feature value to obtain a decoded feature value. Finally, the decoded feature value is input into a trained feedforward neural network to detect contraband in the current X-ray image and obtain a contraband detection result including target category, target box, and score. This application utilizes a fast spatial pyramid pooling network to perform multi-scale feature fusion on X-ray images after feature extraction. The fused feature values ​​are then encoded and merged with sub-feature values ​​from the feature extraction process. The merged feature values ​​are then decoded and input into a feedforward neural network to obtain a contraband detection result containing the category, location box, and score. This method enables automated real-time detection of contraband via X-rays, saving labor and time costs and improving detection efficiency. Furthermore, the application employs feature extraction, multi-feature fusion, and feature merging techniques, enabling comprehensive analysis of contraband during detection, thereby improving detection accuracy and reducing false positive rates. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0048] Figure 1 This is a flowchart of a real-time article detection method disclosed in this application;

[0049] Figure 2This is a specific SPPF network structure block diagram disclosed in this application;

[0050] Figure 3 This is a specific C3 network structure diagram disclosed in this application;

[0051] Figure 4 This is a block diagram of a specific convolutional network structure disclosed in this application;

[0052] Figure 5 This is a specific B1 network structure diagram disclosed in this application;

[0053] Figure 6 This is a specific B2 network structure diagram disclosed in this application;

[0054] Figure 7 This is a specific coding network structure block diagram disclosed in this application;

[0055] Figure 8 This is a block diagram of a specific decoding network structure disclosed in this application;

[0056] Figure 9 This is a flowchart of a specific real-time article detection method disclosed in this application;

[0057] Figure 10 This is a block diagram of a specific real-time article detection method disclosed in this application;

[0058] Figure 11 This is a schematic diagram of the structure of a real-time article detection device disclosed in this application;

[0059] Figure 12 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0060] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0061] This application discloses a method for real-time detection of items. See [link to relevant documentation] Figure 1 As shown, the method includes:

[0062] Step S11: Extract features from the current X-ray image in the real-time received X-ray video stream, and input the extracted first feature value into the optimized fast spatial pyramid pooling network for multi-scale feature fusion to obtain the second feature value.

[0063] In this embodiment, X-ray images from the X-ray video stream captured and transmitted by the X-ray camera are first received in real time. Then, features are extracted from the received current X-ray image to obtain a first feature value. The extracted first feature value is then input into an optimized Spatial Pyramid Pooling Fast (SPPF) network for multi-scale feature fusion to obtain a second feature value. It should be noted that this application optimizes the model structure of the traditional SPPF network, further simplifying the SPPF network model structure and reducing the original number of max pooling operations, thereby improving the overall processing efficiency of the model. For details, see [link to documentation]. Figure 2 As shown, Figure 2 This paper demonstrates a specific optimized SPPF network. When using this network for real-time contraband detection, the first input feature value is first processed through a convolutional network (Conv network, parameters k=1, s=1, p=0, c=512). The resulting convolution is then copied twice: one copy is fed to a feature fusion node, and the other is fed to a Maxpool2d layer (a two-dimensional max pooling layer, parameters k=3, s=1, p=2). The feature fusion node then performs multi-scale feature fusion on the two processed results. Finally, a Conv network (parameters k=1, s=1, p=0, c=1024) performs a convolution operation on the merged result to obtain the second feature value. In the convolutional network, k represents the kernel size, c represents the column number, s represents the stride, and p represents the padding.

[0064] Specifically, the step of extracting features from the current X-ray image in the real-time received X-ray video stream and inputting the extracted first feature value into an optimized fast spatial pyramid pooling network for multi-scale feature fusion to obtain a second feature value may include: acquiring an X-ray image from the real-time received X-ray video stream to obtain a current X-ray image; inputting the current X-ray image into a feature extraction network to extract features from the current X-ray image to obtain a first feature value; the feature extraction network is a network created based on a first number of convolutional networks and a second number of C3 networks; and inputting the first feature value into the optimized fast spatial pyramid pooling network to perform multi-scale feature fusion on the first feature value to obtain a second feature value. In this embodiment, the X-ray image in the real-time received X-ray video stream can be extracted first. Then, the extracted current X-ray image is input into a feature extraction network created based on a first number of convolutional networks and a second number of C3 networks to extract features from the current X-ray image, obtaining a first feature value. This first feature value is then input into an optimized fast spatial pyramid pooling network to perform multi-scale feature fusion to obtain a second feature value. The first and second numbers can be selected according to actual application requirements. For details, see [link to documentation]. Figure 3 and Figure 4 As shown, a specific C3 network and a convolutional network are illustrated, respectively. Figure 3 The C3 network in the code includes 3 convolutional layers (with parameters k=1 and s=1), 1 B1 or B2 network, and 1 feature value merging node. Figure 4 The convolutional network in the code includes one Conv2d layer (a two-dimensional convolutional layer), one Batch Normalization (BN) layer, and one SiLU (Sigmoid Linear Unit, an improved activation function) layer. See also... Figure 5 and Figure 6 As shown, respectively Figure 3 The network structures of B1 and B2 networks, among which Figure 5 The B1 network in the example includes two convolutional layers and one feature value merging node. Figure 6 The B2 network in the model includes two convolutional layers. By combining multiple C3 networks and convolutional networks, the accuracy of feature extraction can be improved.

[0065] Step S12: Input the second feature value into a pre-created encoding network for encoding, and merge the encoded feature value with the feature value in the feature extraction process to obtain the merged feature value.

[0066] In this embodiment, after the extracted first feature value is input into an optimized fast spatial pyramid pooling network for multi-scale feature fusion to obtain a second feature value, the second feature value is further input into a pre-created encoding network for encoding to obtain a corresponding encoded feature value. This encoded feature value is then merged with the sub-feature values ​​from the feature extraction process to obtain a merged feature value. The encoding network is a network based on the Transformer architecture's attention mechanism. It should be noted that the encoding network used for feature extraction can be repeatedly stacked, and the specific number of stacks can be selected according to actual application requirements. For example, the number can be set to N=6, and the number of self-attention modules can be set to M=12.

[0067] For details, see Figure 7 As shown, Figure 7 A specific encoding network is illustrated. The encoding process using this network includes: first, flattening the input features (i.e., the second feature value); then, performing layer normalization on the flattened input features to obtain normalized features; next, dividing the normalized features into three identical sets of values: feature V (Value), feature Q (Query), and feature K (Key); further, using Transformer encoding, positionally encoding features Q and K, and then performing matrix multiplication on the encoded features Q and K. The result of multiplying features Q and K is then scaled, which can be done by dividing by a specific value. Scaling prevents the dot product value from becoming too large. Without scaling, as the vector dimension increases, the dot product value may become very large, leading to a very small gradient in the softmax function, resulting in gradient vanishing and affecting the model's training performance. Additionally, scaling stabilizes the gradient, making the dot product result more stable and ensuring the softmax function operates within an appropriate range, thus guaranteeing the model's training stability. Next, the scaled result is passed through the softmax activation function. The feature values ​​after softmax are then multiplied by the feature V to obtain the output feature; wherein the output feature... The formula can be expressed as:

[0068] ;

[0069] In the formula, Let K be the dimension value of feature K.

[0070] Next, the above output features The input feature (i.e., the second feature value) is added point by point, and the added feature is then normalized by layer. The normalized feature is then passed through a fully connected layer. Finally, the feature after passing through the fully connected layer is added point by point to the added feature to obtain the encoded feature value.

[0071] Specifically, the step of inputting the second feature value into a pre-created encoding network for encoding, and merging the encoded feature value with the feature value obtained in the feature extraction process to obtain a merged feature value, may include: inputting the second feature value into a pre-created encoding network to encode the second feature value, obtaining an encoded feature value; concatenating the encoded feature value with multiple feature values ​​obtained in the feature extraction process to obtain multiple concatenated feature values, and merging the multiple concatenated feature values ​​with the encoded feature value to obtain a merged feature value. That is, first, the second feature value is encoded using a pre-created encoding network, and the encoded feature value is concatenated with multiple feature values ​​obtained in the feature extraction process to obtain multiple concatenated feature values; then, the multiple concatenated feature values ​​are merged with the encoded feature value to obtain a merged feature value.

[0072] Step S13: Input the merged feature value into a pre-created decoding network to decode the merged feature value and obtain the decoded feature value; both the encoding network and the decoding network are attention mechanism networks.

[0073] In this embodiment, the merged feature values ​​can be input into a pre-created decoding network to decode the merged feature values ​​and obtain the corresponding decoded feature values; wherein, the decoding network is a network based on the attention mechanism of the Transformer architecture.

[0074] It should be noted that this application also includes an object localization component, which is input into the decoding network after position encoding (using Transformers' position encoding format). The object localization component is defined based on the output network of the decoding network. Preferably, the number of detected objects can be set to 50, and each detected object includes three detection results: category, bounding box, and score. By employing the object localization component, the training convergence speed of the model can be increased, ensuring a one-to-one correspondence between the model's output category, bounding box, and score.

[0075] In addition, the decoding network in this application outputs all the results of the input X-ray image at once, instead of using the method of rotating and recursively inserting the output results into the network (that is, the first output result is used as input and then enters the network until all results are traversed).

[0076] For details, see Figure 8 As shown, Figure 8 This paper illustrates a specific decoding network. The decoding process using this network includes: first, obtaining the feature values ​​of the object encoding components; then, performing layer normalization on these feature values ​​and dividing the normalized features into three identical groups: feature V, feature Q, and feature K; next, using Transformer encoding to perform positional encoding on features Q and K, and then performing matrix multiplication on the encoded features Q and K; then, scaling the result of features Q and K, which can be achieved by dividing by a specific value. Scaling prevents the dot product value from becoming too large, stabilizes the gradient, and makes the dot product result more stable, ensuring that the softmax function works within an appropriate range, thereby guaranteeing the training stability of the model; further, the scaled result is then processed by the softmax activation function. The feature values ​​after the softmax activation function are multiplied by feature V to obtain the first output feature. Next, the first output feature and the input feature (i.e., the encoded feature value) are added point-by-point, and the resulting first feature is normalized. The normalized value is then positionally encoded as the Q feature of the decoder. The encoded feature value is then used as the features K and V of the decoder. Preferably, features K, V, and Q have the same dimension. Further, features Q and K in the encoded feature value are multiplied by matrix, and the result is scaled (by dividing by a specific value). The scaled result is then passed through the softmax activation function, and the softmax-processed feature value is multiplied by feature V to obtain the second output feature. The second output feature is then added point-by-point to the normalized value, and the resulting second feature is normalized. The normalized value is then passed through a fully connected layer. Finally, the feature after the fully connected layer is added point-by-point to the second feature to obtain the decoded feature value. It should be noted that the number of decoding networks can be repeatedly added, and the specific number can be selected according to the actual application requirements. For example, if the number is set to N=6, the number of self-attention modules can be set to M=12.

[0077] Step S14: Input the decoded feature values ​​into the trained feedforward neural network to detect contraband in the current X-ray image and obtain a contraband detection result including target category, target box and score.

[0078] In this embodiment, after decoding the merged eigenvalue to obtain the decoded eigenvalue, further, the decoded eigenvalue is input into a trained feed-forward neural network (FFN, Feed-Forward Neural Network) to detect contraband in the current X-ray image, and a contraband detection result including the target category, target box, and score is obtained.

[0079] It should be noted that before real-time object detection, it specifically further includes: collecting historical X-ray images to obtain an X-ray image dataset; training an initial feed-forward neural network using the X-ray image dataset to obtain the trained feed-forward neural network; where the network structure of the feed-forward neural network is a fully connected network layer, a rectified linear unit layer, and a fully connected network layer. In this embodiment, historical X-ray images can be collected first to obtain an X-ray image dataset, and then the initial feed-forward neural network is trained using the X-ray image dataset to obtain the trained feed-forward neural network; where the network structure of this feed-forward neural network is a fully connected network layer + rectified linear unit (ReLU) layer + fully connected network layer.

[0080] Further, it may further include: tagging the contraband in each historical X-ray image in the X-ray image dataset to obtain a true label dataset; matching the predicted value dataset generated after training the initial feed-forward neural network with the true label dataset to obtain a matching result; calculating a loss function based on the matching result to evaluate the performance of the trained feed-forward neural network. Specifically, when there are m datasets for predicted targets and n datasets for true labels, m >= n should be satisfied; when m < n, null padding is required for the elements in m to ensure m >= n; then, a cost matrix C is constructed, and the expression of C is:

[0081] ;

[0082] In the formula, represents the cost of matching the i-th element in the predicted target dataset A with the j-th element in the true number dataset B, The calculation method of

[0083] includes , where represents the label number of the category, represents the x coordinate value of the center of the target box of the category, represents the y coordinate value of the center of the target box of the category, The target box width of the category, express The target bounding box height of the category, This indicates the predicted score for the target.

[0084] include ,in express Category number, express The x-coordinate of the center of the target bounding box for the category. express The y-coordinate of the center of the target bounding box for the category. express The target box width of the category, express The target bounding box height of the category, This indicates the predicted score for the target; the default value is 1.

[0085] ;

[0086] In the formula, , , Hyperparameters for controlling the weights of each element;

[0087] Next, normalization is performed:

[0088] ;

[0089] ;

[0090] Furthermore, find the minimum number of lines k that cover all zeros, if Then the optimal solution is obtained.

[0091] It should be noted that if no perfect match is found, the minimum value of the uncovered elements is calculated and adjusted accordingly:

[0092] ;

[0093] ;

[0094] ;

[0095] Repeat the above steps until... Finally, based on the adjusted cost matrix, the optimal matching pair is extracted. And calculate the loss function for the corresponding matching pair:

[0096] ;

[0097] Where i and j are all the matching pairs, Indicates positional deviation loss. The class loss is represented by the following formula:

[0098] ;

[0099] .

[0100] By calculating the aforementioned loss function, the performance of the trained feedforward neural network can be evaluated, thereby ensuring the network's reliability and improving the accuracy of contraband detection. Furthermore, matching predicted samples with real-time samples using the cost matrix, followed by loss function calculation, improves the model's training efficiency.

[0101] As can be seen, in this embodiment, features are first extracted from the current X-ray image in the real-time received X-ray video stream. The extracted first feature value is then input into an optimized fast spatial pyramid pooling network for multi-scale feature fusion to obtain a second feature value. The second feature value is then input into a pre-created encoding network for encoding. The encoded feature value is then merged with the feature value from the feature extraction process to obtain a merged feature value. The merged feature value is then input into a pre-created decoding network to decode the merged feature value to obtain a decoded feature value. Finally, the decoded feature value is input into a trained feedforward neural network to detect contraband in the current X-ray image, obtaining a contraband detection result that includes the target category, target box, and score. This application embodiment uses a fast spatial pyramid pooling network to perform multi-scale feature fusion on the feature-extracted X-ray image. The fused feature values ​​are then encoded and merged with the sub-feature values ​​from the feature extraction process. The merged feature values ​​are then decoded and input into a feedforward neural network to obtain a contraband detection result containing the category, location box, and score. This method achieves automated real-time detection of contraband via X-rays, saving labor and time costs and improving detection efficiency. Furthermore, this application embodiment employs feature extraction, multi-feature fusion, and feature merging techniques, enabling comprehensive analysis of contraband during detection, thereby improving detection accuracy and reducing false positive rates.

[0102] This application discloses a specific method for real-time item detection. (See also...) Figure 9 As shown, the method includes:

[0103] Step S201: Obtain the X-ray image from the real-time received X-ray video stream to obtain the current X-ray image.

[0104] Step S202: Scale the current X-ray image to a preset size to obtain a scaled image, and input the scaled image into the feature extraction network to extract features from the scaled image to obtain a first feature value; the feature extraction network is a network created based on a first number of convolutional networks and a second number of C3 networks, the number of channels corresponding to the first feature value is greater than the number of channels of the scaled image, and the pixel value corresponding to the first feature value is less than the pixel value of the scaled image.

[0105] In this embodiment, after obtaining the X-ray image from the X-ray video stream, it can be preprocessed, such as by scaling. The scaled image is then input into a feature extraction network created by multiple convolutional networks and multiple C3 networks to extract features from the scaled image and obtain a first feature value. It should be noted that the number of channels corresponding to the first feature value is greater than the number of channels in the scaled image, and the pixel value corresponding to the first feature value is less than the pixel value in the scaled image.

[0106] Specifically, X-ray images can be scaled to a specific size, preferably 640*640. It should be noted that during image scaling, the aspect ratio is not changed; the largest side is scaled to 640, while the smaller sides are filled with grayscale. After scaling, feature values ​​of [1, 3, 640, 640] can be obtained; where the four elements in [1, 3, 640, 640] represent the number of images, the number of channels, and the pixel length and width, respectively. Further details can be found in [link to documentation]. Figure 10As shown, the system comprises 5 convolutional networks and 4 C3 networks. The processing flow of each network is as follows: The feature value [1, 3, 640, 640] is input into Conv network 1 to obtain the feature value [1, 64, 320, 320]; the feature value [1, 64, 320, 320] is input into Conv network 2 to obtain the feature value [1, 128, 160, 160]; the feature value [1, 128, 160, 160] is input into C3 network 1 to obtain the feature value [1, 128, 160, 160]; the feature value [1, 128, 160, 160] is input into Conv network 3 to obtain the feature value [1, 256, 80, 80]; the feature value [1, 256, 80, 80] is input into C3 network 2. The eigenvalues ​​[1,256,80,80] are obtained. The eigenvalues ​​[1,256,80,80] are input into Conv network 4 to obtain the eigenvalues ​​[1,512,40,40]. The eigenvalues ​​[1,512,40,40] are input into C3 network 3 to obtain the eigenvalues ​​[1,512,40,40]. The eigenvalues ​​[1,512,40,40] are input into Conv network 5 to obtain the eigenvalues ​​[1,1024,20,20]. The eigenvalues ​​[1,1024,20,20] are input into C3 network 4 to obtain the eigenvalues ​​[1,1024,20,20].

[0107] Step S203: Input the first feature value into the optimized fast spatial pyramid pooling network to perform multi-scale feature fusion on the first feature value to obtain the second feature value.

[0108] Step S204: Input the second feature value into a pre-created encoding network to encode the second feature value and obtain the encoded feature value.

[0109] Step S205: The encoded feature value is concatenated with the third feature value output by the first C3 network during the feature extraction process to obtain the first concatenated feature value; the third feature value has the same number of channels or pixel value as the encoded feature value.

[0110] In this embodiment, see Figure 10 As shown, the encoded feature values ​​can be... The feature values ​​[1,512,40,40] obtained from the C3 network 3 are concatenated to obtain the first concatenated feature value; wherein the feature value [1,512,40,40] is the same as the number of channels or pixel value corresponding to the encoded feature value.

[0111] Step S206: Perform convolution operations on the first concatenated feature values ​​using the pre-created first convolutional network and the second convolutional network respectively to obtain the first convolutional feature values ​​and the second convolutional feature values.

[0112] In this embodiment, see Figure 10 As shown, two convolutional networks can be used to perform convolution operations on the first concatenated feature values ​​to obtain two corresponding convolutional feature values, which are the first convolutional feature values. Eigenvalues ​​after second convolution .

[0113] Step S207: Add the first convolutional feature value and the second convolutional feature value point by point to obtain the first summed feature value.

[0114] In this embodiment, see Figure 10 As shown, the feature values ​​after the two convolutions can be summed point by point to obtain the first summed feature value. ,in, .

[0115] Step S208: The first summed feature value is concatenated with the fourth feature value output by the second C3 network during the feature extraction process to obtain the second concatenated feature value; the fourth feature value has the same number of channels or pixel value as the first summed feature value.

[0116] Further, see Figure 10 As shown, the first summed eigenvalues ​​can be... The feature values ​​[1,256,80,80] output by C3 network 2 in the above feature extraction process are concatenated to obtain the second concatenated feature values.

[0117] Step S209: Perform convolution operations on the second concatenated feature values ​​using the pre-created third and fourth convolutional networks respectively to obtain the third convolutional feature values ​​and the fourth convolutional feature values.

[0118] In this embodiment, a pre-created third convolutional network and a fourth convolutional network can be used to perform convolution operations on the second concatenated feature values ​​to obtain the third convolutional feature values. and the eigenvalues ​​after the fourth convolution .

[0119] Step S210: Add the third convolutional feature value and the fourth convolutional feature value point by point to obtain the second summed feature value, and merge the first summed feature value, the second summed feature value and the encoded feature value to obtain the merged feature value.

[0120] Further, see Figure 10 As shown, the eigenvalues ​​after the third and fourth convolutions are summed point by point to obtain the second summed eigenvalues. ,in Next, respectively and Flatten the surface to obtain the corresponding first flattened eigenvalues. Second flattened eigenvalues The first summed feature value, the second summed feature value, and the encoded feature value are then combined to obtain the combined feature value. ,in .

[0121] Step S211: Input the merged feature value into a pre-created decoding network to decode the merged feature value and obtain the decoded feature value; both the encoding network and the decoding network are attention mechanism networks.

[0122] Step S212: Input the decoded feature values ​​into the trained feedforward neural network to detect contraband in the current X-ray image and obtain a contraband detection result including target category, target box and score.

[0123] For more detailed processing procedures of steps S201, S203, S204, S211, and S212, please refer to the corresponding content disclosed in the foregoing embodiments, which will not be repeated here.

[0124] As can be seen, the embodiments of this application abandon traditional object detection networks such as YOLO or Faster-RCNN, and adopt NMS (non-maximum suppression) to process the target boxes, which improves the detection efficiency of the model. Furthermore, through the optimized SPPF network and the pre-created encoding and decoding networks based on the attention mechanism, the feature fusion is further increased. Moreover, the decoding network is added only in the highest dimension of the model framework, and only the high-dimensional features with the smallest size are extracted, thereby increasing the execution efficiency of the model and solving the problems of slow detection speed and low detection accuracy in the X-ray contraband detection process.

[0125] Accordingly, this application also discloses a real-time item detection device, see [link to relevant documentation]. Figure 11 As shown, the device includes:

[0126] Feature extraction module 11 is used to extract features from the current X-ray image in the real-time received X-ray video stream;

[0127] Feature fusion module 12 is used to input the extracted first feature value into the optimized fast spatial pyramid pooling network for multi-scale feature fusion to obtain the second feature value;

[0128] Encoding module 13 is used to input the second feature value into a pre-created encoding network for encoding;

[0129] The feature merging module 14 is used to merge the encoded feature values ​​with the feature values ​​in the feature extraction process to obtain merged feature values;

[0130] Decoding module 15 is used to input the merged feature values ​​into a pre-created decoding network to decode the merged feature values ​​and obtain decoded feature values; both the encoding network and the decoding network are attention mechanism networks;

[0131] The contraband detection module 16 is used to input the decoded feature values ​​into the trained feedforward neural network to detect contraband in the current X-ray image and obtain a contraband detection result including target category, target box and score.

[0132] The specific workflow of each of the above modules can be found in the relevant content disclosed in the foregoing embodiments, and will not be repeated here.

[0133] As can be seen, in this embodiment, features are first extracted from the current X-ray image in the real-time received X-ray video stream. The extracted first feature value is then input into an optimized fast spatial pyramid pooling network for multi-scale feature fusion to obtain a second feature value. The second feature value is then input into a pre-created encoding network for encoding. The encoded feature value is then merged with the feature value from the feature extraction process to obtain a merged feature value. The merged feature value is then input into a pre-created decoding network to decode the merged feature value to obtain a decoded feature value. Finally, the decoded feature value is input into a trained feedforward neural network to detect contraband in the current X-ray image, obtaining a contraband detection result that includes the target category, target box, and score. This application embodiment uses a fast spatial pyramid pooling network to perform multi-scale feature fusion on the feature-extracted X-ray image. The fused feature values ​​are then encoded and merged with the sub-feature values ​​from the feature extraction process. The merged feature values ​​are then decoded and input into a feedforward neural network to obtain a contraband detection result containing the category, location box, and score. This method achieves automated real-time detection of contraband via X-rays, saving labor and time costs and improving detection efficiency. Furthermore, this application embodiment employs feature extraction, multi-feature fusion, and feature merging techniques, enabling comprehensive analysis of contraband during detection, thereby improving detection accuracy and reducing false positive rates.

[0134] In some specific embodiments, the feature extraction module 11 may specifically include:

[0135] The image acquisition unit is used to acquire X-ray images from the real-time received X-ray video stream to obtain the current X-ray image;

[0136] The first feature extraction unit is used to input the current X-ray image into the feature extraction network to extract features from the current X-ray image and obtain a first feature value; the feature extraction network is a network created based on a first number of convolutional networks and a second number of C3 networks;

[0137] Accordingly, the feature fusion module 12 may specifically include:

[0138] The feature fusion unit is used to input the first feature value into the optimized fast spatial pyramid pooling network to perform multi-scale feature fusion on the first feature value to obtain the second feature value.

[0139] In some specific embodiments, the first feature extraction unit may specifically include:

[0140] An image scaling unit is used to scale the current X-ray image to a preset size to obtain a scaled image;

[0141] The second feature extraction unit is used to input the scaled image into the feature extraction network to extract features from the scaled image and obtain a first feature value.

[0142] Wherein, the number of channels corresponding to the first feature value is greater than the number of channels of the scaled image, and the pixel value corresponding to the first feature value is less than the pixel value of the scaled image.

[0143] In some specific embodiments, the encoding module 13 may specifically include:

[0144] The encoding unit is used to input the second feature value into a pre-created encoding network to encode the second feature value and obtain the encoded feature value.

[0145] Accordingly, the feature merging module 14 may specifically include:

[0146] The first feature value concatenation unit is used to concatenate the encoded feature value with multiple feature values ​​in the feature extraction process to obtain multiple concatenated feature values.

[0147] The first feature value merging unit is used to merge multiple concatenated feature values ​​with the encoded feature values ​​to obtain merged feature values.

[0148] In some specific embodiments, the first feature value splicing unit may specifically include:

[0149] The second feature value concatenation unit is used to concatenate the encoded feature value with the third feature value output by the first C3 network during the feature extraction process to obtain the first concatenated feature value; the third feature value has the same number of channels or pixel value as the encoded feature value.

[0150] The first convolutional unit is used to perform convolution operations on the first concatenated feature value using a pre-created first convolutional network and a second convolutional network respectively, to obtain the first convolutional feature value and the second convolutional feature value.

[0151] The first eigenvalue summation unit is used to add the first convolutional eigenvalue and the second convolutional eigenvalue point by point to obtain the first summed eigenvalue.

[0152] The third feature value concatenation unit is used to concatenate the first summed feature value with the fourth feature value output by the second C3 network during the feature extraction process to obtain the second concatenated feature value; the fourth feature value has the same number of channels or pixel value as the first summed feature value.

[0153] The second convolutional unit is used to perform convolution operations on the second concatenated feature value using the pre-created third and fourth convolutional networks respectively, to obtain the third convolutional feature value and the fourth convolutional feature value.

[0154] The second eigenvalue summation unit is used to add the third convolutional eigenvalue and the fourth convolutional eigenvalue point by point to obtain the second summed eigenvalue.

[0155] Accordingly, the first feature value merging unit may specifically include:

[0156] The second feature value merging unit is used to merge the first summed feature value, the second summed feature value, and the encoded feature value to obtain the merged feature value.

[0157] In some specific embodiments, the real-time item detection device may further include:

[0158] The image acquisition unit is used to acquire historical X-ray images and obtain an X-ray image dataset.

[0159] The network training unit is used to train the initial feedforward neural network using the X-ray image dataset to obtain the trained feedforward neural network.

[0160] The feedforward neural network consists of a fully connected network layer, a linear rectified function layer, and a fully connected network layer.

[0161] In some specific embodiments, the real-time item detection device may further include:

[0162] The tagging unit is used to tag contraband in each historical X-ray image in the X-ray image dataset to obtain a real tag dataset;

[0163] The matching unit is used to match the predicted value dataset generated after the initial feedforward neural network training with the real label dataset to obtain the matching result;

[0164] The loss function calculation unit is used to calculate a loss function based on the matching result, so as to evaluate the performance of the trained feedforward neural network based on the loss function.

[0165] Furthermore, embodiments of this application also disclose an electronic device, Figure 12 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0166] Figure 12 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the real-time item detection method disclosed in any of the foregoing embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0167] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0168] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0169] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the real-time article detection method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include a computer program capable of performing other specific tasks.

[0170] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned real-time item detection method. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0171] Furthermore, embodiments of this application also disclose a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the real-time item detection method disclosed above.

[0172] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0173] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0174] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0175] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0176] The above provides a detailed description of a real-time article detection method, apparatus, device, and storage medium provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for real-time detection of an item, characterized in that, include: Feature extraction is performed on the current X-ray image in the real-time received X-ray video stream, and the extracted first feature value is input into the optimized fast spatial pyramid pooling network for multi-scale feature fusion to obtain the second feature value; The second feature value is input into a pre-created encoding network for encoding, and the encoded feature value is merged with the feature value in the feature extraction process to obtain the merged feature value; The merged feature values ​​are input into a pre-created decoding network to decode the merged feature values, resulting in decoded feature values; both the encoding network and the decoding network are attention-based networks. The decoded feature values ​​are input into a trained feedforward neural network to detect contraband in the current X-ray image, resulting in a contraband detection result that includes the target category, the target box, and the score. The step of inputting the second feature value into a pre-created encoding network for encoding, and merging the encoded feature value with the feature value in the feature extraction process to obtain a merged feature value, includes: inputting the second feature value into a pre-created encoding network to encode the second feature value to obtain an encoded feature value; concatenating the encoded feature value with multiple feature values ​​in the feature extraction process to obtain multiple concatenated feature values; and merging the multiple concatenated feature values ​​with the encoded feature value to obtain a merged feature value. The step of concatenating the encoded feature value with multiple feature values ​​from the feature extraction process to obtain multiple concatenated feature values, and merging the multiple concatenated feature values ​​with the encoded feature value to obtain a merged feature value, includes: concatenating the encoded feature value with a third feature value output by the first C3 network during the feature extraction process to obtain a first concatenated feature value; the third feature value has the same number of channels or pixel value as the encoded feature value; performing convolution operations on the first concatenated feature value using a pre-created first convolutional network and a second convolutional network respectively to obtain a first convolutional feature value and a second convolutional feature value; and then performing a convolution operation on the first concatenated feature value and the second convolutional feature value... The first summed feature value is obtained by adding the features point by point. The first summed feature value is then concatenated with the fourth feature value output by the second C3 network during the feature extraction process to obtain the second concatenated feature value. The fourth feature value has the same number of channels or pixel value as the first summed feature value. The second concatenated feature value is then convolved with the pre-created third and fourth convolutional networks to obtain the third and fourth convolutional feature values. The third and fourth convolutional feature values ​​are then added point by point to obtain the second summed feature value. Finally, the first summed feature value, the second summed feature value, and the encoded feature value are merged to obtain the merged feature value.

2. The real-time detection method for items according to claim 1, characterized in that, The step involves extracting features from the current X-ray image in the real-time received X-ray video stream, and inputting the extracted first feature value into an optimized fast spatial pyramid pooling network for multi-scale feature fusion to obtain a second feature value, including: Acquire X-ray images from the real-time received X-ray video stream to obtain the current X-ray image; The current X-ray image is input into a feature extraction network to extract features from the current X-ray image and obtain a first feature value; the feature extraction network is a network created based on a first number of convolutional networks and a second number of C3 networks; The first feature value is input into the optimized fast spatial pyramid pooling network to perform multi-scale feature fusion on the first feature value to obtain the second feature value.

3. The real-time detection method for items according to claim 2, characterized in that, The step of inputting the current X-ray image into a feature extraction network to extract features from the current X-ray image and obtain a first feature value includes: The current X-ray image is scaled to a preset size to obtain a scaled image; The scaled image is input into the feature extraction network to extract features from the scaled image and obtain a first feature value. Wherein, the number of channels corresponding to the first feature value is greater than the number of channels of the scaled image, and the pixel value corresponding to the first feature value is less than the pixel value of the scaled image.

4. The real-time article detection method according to any one of claims 1 to 3, characterized in that, Also includes: Collect historical X-ray images to obtain an X-ray image dataset; The initial feedforward neural network is trained using the X-ray image dataset to obtain the trained feedforward neural network; The feedforward neural network consists of a fully connected network layer, a linear rectified function layer, and a fully connected network layer.

5. The real-time article detection method according to claim 4, characterized in that, Also includes: Contraband items in each historical X-ray image of the X-ray image dataset are tagged to obtain a real label dataset; The predicted value dataset generated after the initial feedforward neural network training is matched with the real label dataset to obtain the matching result; A loss function is calculated based on the matching results to evaluate the performance of the trained feedforward neural network.

6. A real-time item detection device, characterized in that, include: The feature extraction module is used to extract features from the current X-ray image in the real-time received X-ray video stream; The feature fusion module is used to input the extracted first feature value into the optimized fast spatial pyramid pooling network for multi-scale feature fusion to obtain the second feature value. The encoding module is used to input the second feature value into a pre-created encoding network for encoding; The feature merging module is used to merge the encoded feature values ​​with the feature values ​​in the feature extraction process to obtain merged feature values; The decoding module is used to input the merged feature values ​​into a pre-created decoding network to decode the merged feature values ​​and obtain decoded feature values; both the encoding network and the decoding network are attention mechanism networks; The contraband detection module is used to input the decoded feature values ​​into a trained feedforward neural network to detect contraband in the current X-ray image and obtain a contraband detection result including target category, target box and score; The encoding module is specifically used to input the second feature value into a pre-created encoding network to encode the second feature value and obtain the encoded feature value. The feature merging module is specifically used to concatenate the encoded feature value with multiple feature values ​​in the feature extraction process to obtain multiple concatenated feature values, and to merge the multiple concatenated feature values ​​with the encoded feature value to obtain merged feature values; The feature merging module is further configured to concatenate the encoded feature value with the third feature value output by the first C3 network during the feature extraction process to obtain the first concatenated feature value; The third feature value has the same number of channels or pixel value as the encoded feature value; the first concatenated feature value is convolved with the first convolutional network and the second convolutional network respectively to obtain the first convolutional feature value and the second convolutional feature value. The first convolutional feature value and the second convolutional feature value are added point by point to obtain the first summed feature value; The first summed feature value is concatenated with the fourth feature value output by the second C3 network during the feature extraction process to obtain the second concatenated feature value. The fourth feature value has the same number of channels or pixel value as the first summed feature value; the second concatenated feature value is convolved using the pre-created third and fourth convolutional networks respectively to obtain the third convolutional feature value and the fourth convolutional feature value; The feature values ​​after the third convolution and the feature values ​​after the fourth convolution are added point by point to obtain the second summed feature value. The first summed feature value, the second summed feature value and the encoded feature value are then merged to obtain the merged feature value.

7. An electronic device, characterized in that, It includes a processor and a memory; wherein, when the processor executes a computer program stored in the memory, it implements the real-time article detection method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, Used to store a computer program; wherein, when the computer program is executed by a processor, it implements the real-time article detection method as described in any one of claims 1 to 5.

Images