Subway safety detection method based on TPU processor improved YOLOv8 algorithm

Abstract

A subway safety detection method based on a TPU processor improved YOLOv8 algorithm adopts a Bifpn architecture, introduces a small target detection head in P2, adds an LSKA attention mechanism module in a backbone network, and fuses an LSKA module and an SPPF module to form an SPPF-LSKA module; meanwhile, a heavy parameter convolution OREPA module and a C2f module are adopted to be fused to form a C2f-OREPA module, and a loss function SIOU is redefined; in addition, the model pruning operation is carried out to reduce the model parameter quantity, and the X-ray image detection precision and detection efficiency are remarkably improved. And the system is successfully deployed to a TPU hardware processor, so that the practicability of the model is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence applications, and in particular to an X-ray image detection method based on an improved YOLOv8 model, which is successfully deployed on a TPU device. Background Art

[0002] In subway security inspection scenarios, X-ray image object detection technology is widely used to identify potential threats in luggage, parcels, and personal belongings, such as weapons or explosives. With the development of artificial intelligence technology, more and more security inspection systems have added deep learning models such as convolutional neural networks (CNNs) to improve the accuracy and speed of object detection. Advanced security inspection systems can monitor X-ray image streams in real time and quickly detect suspicious items during the inspection process, thereby improving the ability to respond to potential threats. Given the characteristics of X-ray images, the complexity of their detection tasks is usually higher than that of traditional RGB image object detection. Therefore, based on the YOLOv8 model, improvements have been made to the characteristics of X-ray images, using a lighter BIFPN architecture (Bidirectional Feature Pyramid Network), LSKA (Large Separable Kernel Attention) module, and OREPA (Online Convolutional Reparameterization) technology.

[0003] The BIFPN architecture is a neural network structure for target detection, which aims to improve the efficiency and performance of the feature pyramid network (FPN). The architecture introduces bidirectional feature fusion and weighted feature fusion. By performing bidirectional feature fusion in the upsampling and downsampling processes, the representation capability of multi-scale features is enhanced. At the same time, a weighted feature fusion mechanism is introduced, so that features at different levels can be adjusted according to their importance, thereby better utilizing feature information at different levels. In addition, an efficient feature fusion module is used to reduce the amount of calculation and parameters, while improving the performance and speed of the model.

[0004] LSKA is a lightweight attention mechanism designed to improve the accuracy of target detection. It enhances the performance of important areas in feature maps by focusing on spatial key points while maintaining low computational complexity and a small number of parameters, making it suitable for resource-limited environments. LSKA can effectively highlight key information in feature maps, thereby improving the model's target recognition ability. Figure 3 ) does not change the basic structure of the model and can also speed up the inference speed. The introduction of LSKA improves the inference speed because it decomposes the original K×K convolution operation into 1×K horizontal convolution and K×1 vertical convolution.

[0005] OREPA is a reparameterization technology for improving the efficiency and accuracy of convolutional neural networks. By adjusting the parameters of the convolutional layer online, OREPA can more accurately control the weight of the convolution operation and improve model performance. In the reparameterization process, OREPA reduces the computational complexity and makes the convolution operation more efficient, especially in large-scale networks. In addition, OREPA enhances the feature extraction capability and improves the accuracy and robustness of the model by optimizing the parameter configuration of the convolutional layer. It can reduce the additional training time and GPU memory usage by 65% ​​to 75%, and speed up the training process by 1.5 to 2.3 times. The present invention combines the C2f module with the OREPA module to form a C2f-OREPA module, and applies it in network layers P2, P4 and P6 to improve the reasoning speed and feature extraction effect of the model, thereby enhancing the training efficiency and overall performance of the model.

[0006] Model pruning is a method for efficiently pruning deep learning models. The goal of this technique is to reduce the complexity and computational overhead of the model while preserving the performance of the model as much as possible; at the same time, pruning can help save computing resources and storage space when deploying deep learning models on various devices, especially in environments with limited computing power. This pruning operation is performed step by step by layer, rather than on the entire model at the same time. This method can more accurately adjust the structure of each layer to minimize performance loss.

[0007] These technologies focus on the problems of weak features of contraband in X-ray images, severe background interference, and distortion in small target detection. However, in actual security inspection scenarios, X-ray image detection still faces challenges such as detection speed and complexity of application scenarios. Specific issues include:

[0008] (1) X-ray projection causes object overlap and deformation. Among common contraband items (such as knives), the shapes and sizes of the items vary. Since X-ray scanning is fixed, the projection of the item on the X-ray image may be deformed, and X-ray image target detection is extremely sensitive to the shape information of the object. Compared with the rich information of RGB images, the feature information of X-ray images is often too monotonous.

[0009] (2) X-ray imaging has color differences. The coloring algorithms used by different X-ray equipment manufacturers may cause the same object to appear different in color and brightness on different devices. In addition, even if the equipment is from the same manufacturer, color differences may occur due to different usage time and parameter settings. Summary of the invention

[0010] In order to solve the problems in the prior art, the present invention proposes an X-ray image detection method based on a large separable kernel attention module, a heavily parameterized convolutional neural network and a lightweight BIFPN architecture. This method effectively improves the detection speed and accuracy, especially when dealing with object deformation problems caused by X-ray projection.

[0011] In order to achieve the above-mentioned invention object, the technical solution adopted by the present invention is as follows:

[0012] The subway safety detection method based on the improved YOLOv8 algorithm of the TPU processor described in the present invention comprises the following steps:

[0013] S1. Obtain the X-ray image data set detected by the X-ray machine and convert it into a TXT format that can be used for training. Each line of this format file is the target index, the upper left corner coordinate, and the lower right corner coordinate. The specific steps are as follows:

[0014] Data reading and labeling: read the data image through the X-ray machine, and use the Labeling tool to manually label the data. Data preprocessing: preprocess the labeled data, including rotation, segmentation, and brightness adjustment operations to form an X-ray object data set. Data enhancement: preprocess the images and apply Mosaic data enhancement technology during model training; the preprocessing steps are to load the processed data set into a local file, load the label file from the local file, and adjust the input image and its label. According to the model target, classify the extremely small targets, small targets, medium targets, and large targets in the image, and segment and enlarge the image according to the target position so that the model can learn better.

[0015] S2, YOLOv8 algorithm improvement steps:

[0016] (1) BiFPN architecture is used to improve YOLOv8, covering three modules: Backbone, Neck and Head. Considering that the dataset is mainly composed of small targets, a P2 small target detection head is added to the framework. Since the texture and edge details of the original image are gradually lost during the convolution operation, the information features of small targets are reduced. SPPF and attention mechanism LSKA are introduced for fusion to form the SPPF-LSKA module. This improvement not only enhances the information feature extraction capability and detection accuracy of the backbone, but also improves the computational efficiency of the model.

[0017] In addition, the OREPA convolution reparameterization module is fused with the C2f module features and named C2f-OREPA module. The OREPA convolution reparameterization module effectively reduces the GPU resource cost required for model training, while speeding up training and improving detection performance. Through model pruning experiments, it is found that the number of model parameters is successfully reduced, thereby improving the detection speed, and by reducing the feature data generated by the C2f module, the parameter advantage of the model is maintained.

[0018] Finally, in order to expand the detection range of the model, four detection heads are added in the improvement, targeting extremely small, small, medium and large-scale targets respectively, so that the model has a wider range of detection capabilities.

[0019] (2) With the continuous development of target detection networks, model training time and GPU memory overhead are gradually increasing. In order to solve this problem, the present invention adopts the fusion of the reparameterizable module OREPA and the C2f module. OREPA can reduce the additional training time and GPU memory cost by 65% ​​to 75%, and speed up the training process by 1.5 to 2.3 times. Therefore, the present invention uses C2f-OREPA modules in network layers P2, P4 and P6 to increase the reasoning speed of the model. At the same time, this fusion also improves the feature extraction effect of the model, greatly improves the training efficiency of the reparameterized model, and thus provides stronger support for model performance.

[0020] (3) To improve the detection accuracy of the model, LSKA (attention mechanism) is introduced into the backbone network module. LSKA can capture the key information of multiple detection channels and effectively reduce the detection difficulty of small targets in X-ray images. Figure 3 ) does not change the basic structure of the model and can speed up the model reasoning. The introduction of LSKA significantly improves the reasoning efficiency of the model by decomposing the K×K convolution operation into 1×K horizontal convolution and K×1 vertical convolution.

[0021] S3. Use the training data set to train the improved YOLOv8 target detection model, where the improved YOLOv8 target detection model includes a backbone network feature extraction network, a neck feature fusion network, and a head prediction network; and input the X-ray image as an input module;

[0022] S4, X-ray images first enter the feature extraction layer, extract image features through P0 and P1 convolution layers, then enter the P20 convolution model through the C2f module in P2, and then perform feature fusion through P21, and input the P22 reparameterized convolution to detect shallow and extremely small targets. Then, further extract features through P3 convolution, after reparameterized convolution, output to P10 neck, and then pass through P17 feature fusion, and pass through P18 reparameterized convolution, P24 feature fusion and P25 reparameterized convolution in turn to detect shallow small targets. Subsequently, through P5 convolution operation and C2f_OREPA module, and then through P11 convolution model, P14 feature fusion Fusion module, P15 reparameterized convolution C2f_OREPA module, P27 feature fusion Fusion module and P28 reparameterized convolution C2f_OREPA module to output medium target box. Finally, the large target frame is output through the P7 convolution operation, P8 re-parameterized convolution C2f_OREPA module, SPPF_LSKA module, P12 convolution module, P31 feature fusion Fusion module and P32 re-parameterized convolution C2f_OREPA module. Finally, the four detection heads output the extremely small target, small target, medium target and large target respectively.

[0023] S5. Adjust the parameters according to the output of the model and redefine the loss function SIOU, which comprehensively considers the angle loss, distance loss, shape loss and IOU loss. By comparing the difference between the predicted bounding box of the training model and the original bounding box in angle, distance and shape, the prediction effect is comprehensively evaluated. Finally, the location information and classification information of the region of interest can be accurately represented.

[0024] For the IOU loss function problem, CIOU loss is used in YOLOv8. IOU is an important indicator for measuring the accuracy of target detection in a dataset. Suppose there are two bounding boxes, one is the real object bounding box, and B gt =[x gt ,y gt ,ω gt ,h gt ], and the other is the predicted box, represented by b = [x, y, ω, h], where the four values ​​represent the center coordinates (x, y), width (ω) and height (H) of the bounding box.

[0025] In this case, the calculation of the original model loss function CIOU loss is:

[0026]

[0027] Among them B gt is the area of ​​the true target box, B prd is the area of ​​the predicted target box. ρ(B gt, B prd ) represents the distance between two target boxes. c and h c is the width and height of the minimum enclosed space covering the two target boxes. The height and width of the predicted box and the real box are h respectively. prd ,ω prd 、h gt and ω gt Since CIOU defines the aspect ratio as a relative value, there is an ambiguous situation. When the predicted box has the same aspect ratio as the true box, but different height and width, the loss function values ​​between the above target boxes are the same, which means that these loss functions lose their effectiveness, thus limiting the convergence accuracy.

[0028] Therefore, considering the limitations of the existing loss function CIOU, the loss function SIOU is redefined, which comprehensively considers multiple aspects such as angle loss, distance loss, shape loss and IOU loss. The specific definitions are as follows:

[0029]

[0030] Where L represents the loss function, IoU represents the ratio of the intersection and union of the predicted box and the true box, Δ is the distance loss, and Ω is the distance loss.

[0031] The distance loss is:

[0032]

[0033] Among them, ρ x , ρ y Use power-of-2 weights for the x-axis distance loss, represents the bounding box loss at the center of the regression box in the x-axis and y-axis, represents the border loss of the center point of the ground-truth box; c w , c h Represents the Euclidean distance between the center point of the regression box and the true box in terms of width and height.

[0034] The shape loss is:

[0035]

[0036] Where Ω represents the shape loss, ω, ω gt Indicates the width distance between the real box and the predicted box; h, h gt Represents the height distance between the real box and the predicted box; θ represents the angle between the predicted box and the center point of the real box.

[0037] By introducing the distance and angle loss between the real box and the predicted box into the loss function, SIOU can comprehensively consider multiple aspects. This method avoids the problem of loss function failure under the same aspect ratio, thereby achieving faster convergence speed and more accurate positioning results.

[0038] Through the above improvements, the model accuracy was improved from the original 0.686 to 0.721, an increase of 5%.

[0039] S6. Prune the model according to the prediction results. First, the trained model is evaluated using the SIOU loss function, and the importance of each channel in the model is evaluated by the absolute value of the weight. According to the evaluation results, the channels with lower importance are removed to reduce the model size. Subsequently, the pruned model is retrained to restore performance and adapt to the new structure. Finally, the pruned model is further adjusted to optimize performance, while maintaining high accuracy while reducing computational and storage overhead. Through data set verification, the accuracy of the pruned model is slightly reduced, but the number of model parameters is reduced by 48%, and the detection speed is increased by 30%.

[0040] S7. Deploy the model on the TPU platform BM1684 device. The BM1684 chip supports a variety of deep learning algorithm frameworks, but the trained model needs to be compiled to adapt to the efficient operation of the TPU. The BM1684 development board provides a complete set of deep learning processor compilation tool chains—TPU-MLIR, which can convert pre-trained neural network models under different frameworks into bmodel files that can be efficiently operated on the BM1684 processor.

[0041] Here are the steps:

[0042] (1) Convert the trained weight file into an intermediate file in ONNX format.

[0043] (2) Convert the intermediate file in ONNX format in the docker environment or Sophon MLIR SDK environment provided by Suanneng, and call the model conversion API to convert the intermediate file in ONNX format into an MLIR format file. Call the model deployment API to convert the MLIR file into a bmodel file that can be accelerated on the TPU platform. If you need to get an int8 type bmodel file, you need to call the quantization table generation API to get the quantization table before calling the model deployment API to use the quantization table to assist in getting an int8 type bmodel file.

[0044] (3) Install the Sophon-sail toolkit that supports the YOLOv8 algorithm on the TPU platform.

[0045] (4) Write code to call the acceleration interface and use the bmodel file converted by YOLOv8 to perform inference analysis on the RTSP data stream input to the TPU device.

[0046] S8. The hardware system obtains the RTSP data stream. RTSP is a client-server protocol that supports multiple codecs and is suitable for different audio and video formats. The client can send commands such as play, pause, and stop to the server. Build an RTSP streaming server, push the video streams collected by the X-ray machine and the camera to the RTSP streaming server through FFmpeg, and then the BM1684 device obtains the data stream for real-time reasoning and display. Furthermore, the video data of all security equipment in the entire subway station can be pushed to the streaming server through RTSP, and then the BM1684 server completes reasoning, detection, and display, thereby greatly saving labor costs.

[0047] Here are the steps:

[0048] (1): Start the RTSP streaming server.

[0049] (2): Use the FFmpeg tool to obtain the RTSP data stream of the X-ray security scanner and camera, and push the data stream to the RTSP streaming server.

[0050] (3): The TPU platform pulls the RTSP data stream for real-time reasoning detection and achieves real-time monitoring.

[0051] In view of the shortcomings of the prior art, the present invention proposes a method for intelligently detecting prohibited items and realizing visual data management. This method greatly reduces the interference of background information on the model, solves the problem of low accuracy caused by color differences in the images of security inspection machines in X-ray images; compared with the problems of low efficiency, slow speed and waste of human resources in traditional manual identification of prohibited items, it can meet the needs of intelligent security management. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a schematic diagram of the process of the present invention.

[0053] Figure 2 The present invention is a schematic diagram of the network model improvement process.

[0054] Figure 3 The figure is a schematic diagram of the network structure involved in the present invention.

[0055] Figure 4 Figure 2 is a schematic diagram of the structure of the C2f-OREPA module.

[0056] Figure 5 Schematic diagram of the structural decomposition of the SPPF-LSKA module.

[0057] Figure 6 Comparison chart of model pruning channels.

[0058] Figure 7 This is a comparison chart of model detection effect parameters, where a is the YOLOv8 model, b is the improved YOLOv8 model, and c is the pruned improved YOLOv8 model.

[0059] Figure 8 This is a comparison chart of the improved YOLOv8 model. Among them, a is the detection accuracy chart of the YOLOv8 model, and b is the detection accuracy chart of the improved pruned model.

[0060] Fig. 9 A deployment diagram for converting an algorithm model into an executable model for TPU devices.

[0061] Fig.10 Schematic diagram of the hardware terminal data transmission system based on the RTSP protocol. DETAILED DESCRIPTION

[0062] The embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, but the present invention is not limited to these embodiments. Those skilled in the art can fully understand the present invention without these detailed descriptions.

[0063] In the following paragraphs, the present invention will be described in more detail through the accompanying drawings. It should be noted that the accompanying drawings adopt a simplified description method and use an inaccurate scale, which is intended to facilitate and clearly assist in explaining the embodiments of the present invention.

[0064] like Figure 1 and Figure 2 As shown, the subway safety detection system based on the TPU processor improved YOLOv8 algorithm includes the following steps:

[0065] S1, obtaining an X-ray image to be detected through an X-ray machine;

[0066] S2. Preprocess the X-ray image dataset, including image rotation, flipping, and cropping, and manually label the data using the Labeling tool to convert the note file into the YOLO format;

[0067] S3, by improving the YOLOv8 model, including: modifying the Bifpn network architecture, using the SPPF-LSKA module as the downsampling module, using the C2f-OREPA module as the P2, P4, P6 and P8 backbone feature extraction module and adding the small target detection head P2;

[0068] S4, taking the X-ray image to be detected as an input to the foreground feature module, and obtaining four foreground features corresponding to the X-ray image to be detected; wherein the foreground feature comparison module includes a YOLOv8 backbone feature extraction network;

[0069] S5, concatenating the outputs of the four shape enhancement modules with the four outputs of the BiFPN feature extraction pyramid network to obtain four concatenated features;

[0070] The S6 and YOLOv8 network structures combine FPN and PAN to design the BiFPN structure. However, after analyzing the X-ray dataset, it was found that the position information of small-sized targets was not effective. To further improve the accuracy of target detection, this paper uses the BiFPN structure as a feature fusion method. When inputting BiFPN in the feature layer, a 3×3 convolution layer is added with a step size of 2 and a channel number of 256. In addition, the three-layer fusion structure of BiFPN is expanded to a four-layer fusion structure, and a very small target detection layer is introduced in the P2 layer.

[0071] S7, such as Figure 4 As shown in the figure, this paper introduces OREPA into the C2f module and optimizes the Bottleneck module. First, we apply the principle of linear stage blocks, delete all nonlinear BN layers in the Bottleneck module, and replace them with linear scaling layers to maintain the diversity of optimization directions. To stabilize the training process, a BN layer is added after the branch. Next, using the principle of compressed stage structure, the linear scaling layer is compressed into an OREPA module, thereby reducing the training cost while maintaining high performance. Finally, we introduce this module into the C2f module and name it the C2f-OREPA module. By reparameterizing the auxiliary structure into a single convolutional layer, this improvement not only maintains high performance but also significantly reduces the training cost.

[0072] S8. LSKA attention is introduced into the SPPF downsampling module. The LSKA attention mechanism first uses 1D convolution kernels for spatial separation convolution, then generates an attention map using the obtained feature map, and finally applies the attention map to enhance the original feature map. When LSKA attention is integrated into the SPPF structure, the ability to pattern recognize X-ray images is enhanced without adding too much computational burden.

[0073] S9. Send the four spliced ​​features to the prediction layer respectively, obtain the prediction box category information corresponding to the X-ray image to be detected, and complete the detection.

[0074] According to the image input, the first convolution layer, the second convolution layer, the first C2f-OREPA layer, the third convolution layer, the second C2f-OREPA layer, the fourth convolution layer, the third C2f-OREPA layer, the fifth convolution layer, the fourth C2f-OREPA layer and the SPPF module are connected in sequence; wherein the output of the first C2f-OREPA layer is the first layer output of the foreground feature contrast module, the output of the second C2f-OREPA layer is the second layer output of the foreground feature contrast module, the output of the third C2f-OREPA layer is the third layer output of the foreground feature contrast module and the SPPF output is the fourth layer output of the foreground feature contrast module;

[0075] The input end of the first morphological bounding box module is connected to the first output end of the foreground feature comparison module, and the output of the first shape enhancement module is connected to the first output of the BiFPN feature extraction pyramid network through a connection layer, finally forming the first target box output layer;

[0076] The input end of the second morphological bounding box module is connected to the second output end of the foreground feature contrast module, and the output of the second shape enhancement module is connected to the second output of the BiFPN feature extraction pyramid network through a connection layer, finally forming a second target box output layer;

[0077] The input end of the third morphological bounding box module is connected to the third output end of the foreground feature contrast module, and the output of the first shape enhancement module is connected to the third output of the BiFPN feature extraction pyramid network through a connection layer, finally forming the first target box output layer;

[0078] The input of the fourth morphological bounding box module is connected to the fourth output of the foreground feature contrast module, and the output of the first shape enhancement module is connected to the fourth output of the BiFPN feature extraction pyramid network through a connection layer, finally forming the first target box output layer.

[0079] S10. Finally, the CIOU loss function of the original YOLOv8 model is converted into the SIOU loss function which is more in line with the scene changes.

[0080] In the specific implementation process, the network model involved in this method is first trained, and the original model YOLOv8 is trained to compare the regression curves before and after the improvement. In one embodiment of the present invention, by obtaining the color invariance characteristics of the X-ray image, the same X-ray machine output data is used for experiments, the original training image is used to train the network, and the test image is used to test the network. The model has color differences in real security inspection scenarios. Among them, the training set includes 4092 X-ray images, the test set includes 722 X-ray images, and the ratio of training data to test data is 11:2. The input of the entire network is the X-ray image, and the output is the location type of contraband and the target location information.

[0081] In this embodiment, this method is compared with other methods using the YOLOv8 model as the benchmark model. Figure 8 As shown in Table 1. Figure 8 As shown in Table 1, on the test set after model improvement, the mAP of this method is 3.5 percentage points higher than that of the baseline model. Except for the computer category, the detection accuracy of other categories has been improved.

[0082] Table 1

[0083] YOLOv8 Model Improving the YOLOv8 model Pruning to improve the YOLOv8 model FPS 79.4 45.7 72.2 GFLOS 8.1 39.1 19.6 Map 0.686 0.721 0.699

[0084] S11. In this embodiment, we perform Lamp pruning on the improved trained model file pt, evaluate the weight coefficients of different network layers through the algorithm, judge the importance of the channel, and perform pruning operations; then, fine-tune the pruned model to verify and adjust its performance. Figure 6 ,By comparing the amount of parameters between each channel before and after pruning, the ,model parameters are reduced by 48% and the detection speed is increased by 30%.

[0085] S12, such as Fig. 9As shown in the figure, the deployment of the model on the BM1684 development board: 1. Convert the trained weight file into an intermediate file in ONNX format. 2. Convert the intermediate file in ONNX format through the Docker environment or Sophon MLIR SDK environment. First, call the model conversion API to convert the intermediate file in ONNX format into an MLIR format file. Then, use the model deployment API to convert the MLIR file into a bmodel file suitable for accelerated operation on the TPU platform. If you need to generate an int8 type bmodel file, you need to first call the generate quantization table API to generate a quantization table, and then use the model deployment API to combine the quantization table to generate an int8 type bmodel file. 3. Install the sophon-sail toolkit that supports the operation of the YOLOv8 algorithm on the TPU platform. 4. Write code to call the acceleration interface and use the bmodel file converted by YOLOv8 to perform inference analysis on the RTSP data stream input to the TPU device.

[0086] S13, such as Fig.10 As shown in the figure, the hardware system obtains the RTSP data stream. 1. Start the RTSP streaming server. 2. Use the FFmpeg tool to obtain the RTSP data stream of the X-ray security scanner and camera, and push the data stream to the RTSP streaming server. 3. The TPU platform pulls the RTSP data stream for real-time reasoning detection and achieves real-time monitoring.

Claims

1. A subway safety detection method based on the improved YOLOv8 algorithm of TPU processor, characterized by The following steps are involved: S1. Obtain an X-ray image dataset detected by an X-ray machine and convert it into a TXT format that can be used for training. Each line of the file in this format is the target index number, the upper left corner coordinate, and the lower right corner coordinate in sequence; S2. Improve the YOLOv8 algorithm: (1) The BiFPN architecture is used to improve YOLOv8, covering three modules: backbone, neck, and head. The P2 small target detection head is added to the framework, and SPPF and attention mechanism LSKA are introduced for fusion to form the SPPF-LSKA module. In addition, the OREPA convolution reparameterization module is fused with the C2f module features and named C2f-OREPA module; Finally, a very small target detection head is added to the original model, and the detection heads are respectively for very small, small, medium and large-scale targets; (2) Use C2f-OREPA modules in network layers P2, P4, and P6 to improve the inference speed of the model; (3) Introduce the attention mechanism LSKA in the backbone network module; S3. Use the training data set to train the improved YOLOv8 target detection model, where the improved YOLOv8 target detection model includes a backbone network feature extraction network, a neck feature fusion network, and a head prediction network; and input the X-ray image as an input module; S4, X-ray images first enter the feature extraction layer, extract image features through P0 and P1 convolution layers, then enter the P20 convolution model through the C2f module in P2, and then perform feature fusion through P21, and input the P22 re-parameterized convolution to detect shallow and extremely small targets; then, further extract features through P3 convolution, after re-parameterized convolution, output to P10 neck, and then pass through P17 feature fusion, and successively pass through P18 re-parameterized convolution, P24 feature fusion and P25 re-parameterized convolution to detect shallow small targets; then, through P5 convolution operation and C2f_OREPA module, and then through P11 convolution The model, P14 feature fusion Fusion module, P15 re-parameterized convolution C2f_OREPA module, P27 feature fusion Fusion module and P28 re-parameterized convolution C2f_OREPA module are used to output the medium target frame; finally, the large target frame is output through the P7 convolution operation, P8 re-parameterized convolution C2f_OREPA module, SPPF_LSKA module, P12 convolution module, P31 feature fusion Fusion module and P32 re-parameterized convolution C2f_OREPA module; finally, the four detection heads are used to output the extremely small target, small target, medium target and large target respectively; S5. Adjust the parameters according to the output results of the model, comprehensively consider the angle loss, distance loss, shape loss and IOU loss, and redefine the loss function SIOU: Among them, L represents the loss function, IoU represents the ratio of the intersection and union of the predicted box and the true box, Δ is the distance loss, and Ω is the distance loss; Among them, the distance loss is: Among them, ρ x , ρ y Use power-of-2 weights for the x-axis distance loss, represents the bounding box loss at the center of the regression box in the x-axis and y-axis, represents the border loss of the center point of the ground-truth box; c w , c h Represents the Euclidean distance between the width and height of the center point of the regression box and the true box; Among them, shape loss: Where Ω represents the shape loss, w, w gt Indicates the width distance between the real box and the predicted box; h, h gt Represents the height distance between the real box and the predicted box; θ represents the angle between the predicted box and the center point of the real box; S6. Prune the model according to the prediction results. First, evaluate the trained model using the redefined SIOU loss function, and evaluate the importance of each channel in the model by the absolute value of the weight. According to the evaluation results, remove the channels with lower importance to reduce the model size. Then, retrain the pruned model to restore the performance and adapt to the new structure. Finally, further adjust the pruned model to optimize the performance and reduce the computational and storage overhead while maintaining high accuracy. S7. Deploy the model on the TPU platform BM1684 device: (1) Convert the trained weight file into an intermediate file in ONNX format; (2) Convert the intermediate file in ONNX format in the docker environment or Sophon MLIR SDK environment provided by Suanneng, call the model conversion API to convert the intermediate file in ONNX format into an MLIR format file; call the model deployment API to convert the MLIR file into a bmodel file that can be accelerated on the TPU platform. If you need to get an int8 type bmodel file, you need to call the generate quantization table API to get the quantization table before calling the model deployment API to use the quantization table to assist in getting an int8 type bmodel file; (3) Install the Sophon-sail toolkit that supports the YOLOv8 algorithm on the TPU platform; (4) Write code to call the acceleration interface and use the bmodel file converted by YOLOv8 to perform reasoning analysis on the RTSP data stream input to the TPU device; S8. Hardware system obtains RTSP data stream: Build an RTSP streaming server, push the video stream collected by the X-ray machine and camera to the RTSP streaming server through FFmpeg, and then the BM1684 device obtains the data stream for real-time reasoning and display. The steps are as follows: (1) Start the RTSP streaming media server; (2) Use FFmpeg to obtain the RTSP data stream of the X-ray security scanner and camera, and push the data stream to the RTSP streaming server; (3) The TPU platform pulls the RTSP data stream for real-time reasoning detection and achieves real-time monitoring.

2. According to claim 1, a subway safety detection method based on a TPU processor improved YOLOv8 algorithm is characterized in that Step 1 includes the following steps: (1) Data reading and labeling: read the data image through the X-ray machine and use the Labeling tool to manually label the data; (2) Data preprocessing: perform preprocessing operations including rotation, segmentation and brightness adjustment on the labeled data to form an X-ray object data set; (3) Data enhancement: during model training, preprocess the images and apply Mosaic data enhancement technology; (4) Data preprocessing: load the processed data set into a local file, load the label file from the local file, and adjust the input image and its label. According to the model target, classify the extremely small targets, small targets, medium targets and large targets in the image, and segment and enlarge the image according to the target position so that the model can learn better.

Images