Method and device for monitoring human and vehicle invasion

Abstract

The invention provides a method and device for monitoring human-vehicle intrusion, and the method comprises the steps: dividing a human-vehicle intrusion detection data set into a training set and a verification set, and carrying out the preprocessing of the training set, and obtaining a marked training set image; clustering the annotation boxes of the training set images to generate a prior block diagram; an improved YOLOv7 target detection network model is constructed, multi-scale features are extracted, a neck network is fused with the multi-scale features, a detection head is obtained, and a loss function is constructed; based on the training set image and the verification set, using a transfer learning mode to train the improved YOLOv7 target detection network model; and inputting a monitoring image into the trained model for reasoning to obtain a reasoning result, carrying out frame filtering on the reasoning result, storing the filtered reasoning result into a time sequence table, and outputting an object intrusion alarm when the occurrence frequency of the same object in the time sequence table reaches a set threshold value. The method is high in real-time performance, high in robustness to the late night environment and high in practical application value.

Description

Technical Field

[0001] The present invention relates to the field of intelligent video monitoring, and in particular to a method and device for monitoring human and vehicle intrusion. Background Art

[0002] With the rapid development of deep learning and machine vision technology, neural network-based object detection technology has achieved increasingly higher accuracy, replacing traditional image detection algorithms and becoming the main technology in the field of video surveillance. And with the continuous improvement of edge computing chip computing power and the gradual reduction of costs, the efficiency of running neural network models in surveillance camera chips is also getting higher and higher. However, even so, the cost of running neural network models in surveillance camera chips is still very high, and in many scenarios it cannot meet the real-time requirements, so it is crucial to use some lightweight technologies to reduce the number of parameters and calculations of neural network models.

[0003] Human and vehicle intrusion detection is an important demand in the field of video surveillance. Many confidential and prohibited places prohibit people and vehicles from entering. Once it is determined that a person or vehicle has intruded, an alarm needs to be issued in time. Generally, the cameras used to monitor such large-scale sites are mostly long-distance telephoto cameras. Therefore, the target object occupies a small proportion of the entire image, which belongs to the category of small target detection. The basic target detection network is generally not suitable for detecting small targets, so some special designs are also needed to modify the target detection network to increase its accuracy in detecting small targets.

[0004] At the same time, in response to the common environmental interference and false alarm problems in video surveillance, some filtering strategies and judgment strategies are also needed to reduce the impact of environmental interference and false alarms. Summary of the invention

[0005] In view of the problems of human and vehicle intrusion detection, environmental interference and false alarm, the present invention proposes a method and device for monitoring human and vehicle intrusion.

[0006] The present invention provides a method for monitoring human and vehicle intrusion, which comprises the steps of: The human-vehicle intrusion detection dataset is divided into a training set and a validation set, and the training set data is preprocessed to obtain labeled training set images; Use KMeans++ clustering method to cluster the annotation boxes of the training set images and generate a priori box diagram; An improved YOLOv7 target detection network model is constructed, which includes a backbone network, a neck network, a detection head, and a loss function. The backbone network extracts multi-scale features from the prior frame map, the neck network fuses multi-scale features, the detection head is obtained by detecting feature maps at multiple scales, and the loss function is constructed based on bounding box loss, confidence loss, category loss, and their corresponding adjustable weight coefficients. Based on the preprocessed training set images and validation set, the improved YOLOv7 target detection network model is trained using transfer learning. The preprocessed images collected by the surveillance camera are input into the trained improved YOLOv7 target detection network model for inference to obtain the inference results, and then the inference results are filtered by borders. The filtered inference results are then saved in a time series table. When the number of times the same object appears in the time series table reaches the set threshold, an intrusion alarm for the object is output.

[0007] Furthermore, the method for acquiring the human and vehicle intrusion detection dataset is: selecting an experimental scene, setting the height and incident angle of the surveillance camera, continuously recording video under dynamic conditions in the morning, noon, evening, and on sunny, cloudy, and rainy days, outputting the video data at a set frame rate to generate image data, and obtaining the human and vehicle intrusion detection dataset after removing irrelevant data and annotation.

[0008] Furthermore, preprocessing the training set data includes: enhancement processing; which specifically includes: Use HSV enhancement to randomly transform the hue, brightness and saturation of the original image; Affine transformation enhancement is used to randomly flip, rotate and scale the original image; Mixup enhancement is used to mix two different human and vehicle intrusion detection images in a certain proportion to generate a new image; Mosaic-4 data enhancement is used to randomly select 4 images from the human-vehicle intrusion detection dataset, perform geometric transformation enhancement, and then splice them into 1 image; The enhanced images and annotations are aligned to the original image size and added to the training set.

[0009] Furthermore, the improved YOLOv7 target detection network model adopts deep separable convolution; The backbone network downsamples the prior frame graph to obtain feature graphs of a total of 5 scales: P1, P2, P3, P4 and P5; The neck network fuses multi-scale feature maps to form an FPN network and a PAN network, the FPN network is used to fuse the deep semantic information of the network to the shallow detail information, and the PAN network is used to fuse the shallow detail information to the deep semantic information; The detection head performs detection on the feature maps P1, P2, and P3 to obtain the corresponding detection head.

[0010] Furthermore, the bounding box loss is CIoU loss, and the confidence loss and category loss are BCE loss.

[0011] Furthermore, the improved YOLOv7 target detection network model is trained by using transfer learning, specifically including: The improved YOLOv7 target detection network model is first pre-trained on the COCO dataset, and then trained on the human-vehicle intrusion detection dataset. The training is repeated in rounds. After each round of iteration, the weights are saved and the accuracy is tested on the validation set. Finally, the weight with the highest accuracy on the validation set is taken as the weight of the final model.

[0012] The present invention provides a device for monitoring human and vehicle intrusion, which comprises: The preprocessing module divides the human-vehicle intrusion detection dataset into a training set and a validation set, and preprocesses the training set data to obtain labeled training set images; The clustering module uses the KMeans++ clustering method to cluster the annotation boxes of the training set images and generate a priori box diagram; Module construction module, build an improved YOLOv7 target detection network model, which includes a backbone network, a neck network, a detection head and a loss function; the backbone network extracts multi-scale features from the prior frame map, the neck network fuses multi-scale features, the detection head is obtained by detecting feature maps at multiple scales, and the loss function is constructed based on bounding box loss, confidence loss, category loss and their corresponding adjustable weight coefficients; The training module uses transfer learning to train the improved YOLOv7 target detection network model based on the preprocessed training set images and validation set; The detection module pre-processes the images collected by the surveillance camera and inputs them into the trained improved YOLOv7 target detection network model for inference to obtain the inference results, then performs border filtering on the inference results, and then saves the filtered inference results in the draft time series table. When the number of times the same object appears in the time series table reaches the set threshold, an alarm for the intrusion of the object is output.

[0013] Furthermore, preprocessing the training set data includes: enhancement processing; which specifically includes: Use HSV enhancement to randomly transform the hue, brightness and saturation of the original image; Affine transformation enhancement is used to randomly flip, rotate and scale the original image; Mixup enhancement is used to mix two different human and vehicle intrusion detection images in a certain proportion to generate a new image; Mosaic-4 data enhancement is used to randomly select 4 images from the human-vehicle intrusion detection dataset, perform geometric transformation enhancement, and then splice them into 1 image; The enhanced images and annotations are aligned to the original image size and added to the training set.

[0014] Furthermore, the improved YOLOv7 target detection network model adopts deep separable convolution; The backbone network downsamples the prior frame graph to obtain feature graphs of a total of 5 scales: P1, P2, P3, P4 and P5; The neck network fuses multi-scale feature maps to form an FPN network and a PAN network, the FPN network is used to fuse the deep semantic information of the network to the shallow detail information, and the PAN network is used to fuse the shallow detail information to the deep semantic information; The detection head performs detection on the feature maps P1, P2, and P3 to obtain the corresponding detection head.

[0015] Furthermore, the improved YOLOv7 target detection network model is trained by using transfer learning, specifically including: The improved YOLOv7 target detection network model is first pre-trained on the COCO dataset, and then trained on the human-vehicle intrusion detection dataset. The training is repeated in rounds. After each round of iteration, the weights are saved and the accuracy is tested on the validation set. Finally, the weight with the highest accuracy on the validation set is taken as the weight of the final model.

[0016] The target detection network model used in the present invention can quickly obtain human and vehicle intrusion detection results using less than 10M of storage space, and has high detection accuracy; compared with the existing human and vehicle intrusion monitoring solutions, the solution of the present invention has high detection accuracy for small targets at a long distance, a simple network structure design of the model, is hardware-friendly, fast and real-time, and has strong robustness in late-night environments and high practical application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a flow chart of the method of the present invention; Figure 2 A schematic diagram of the network structure of the method of the present invention; Figure 3 A schematic diagram of a network module used in the method of the present invention; Figure 4 This is a schematic diagram of the human and vehicle intrusion detection effect of the improved target detection network model of the present invention and the YOLOv7 benchmark model. DETAILED DESCRIPTION

[0018] The specific implementation of the present invention is described below to facilitate the understanding of the present invention by those skilled in the art. It should be noted that, for those skilled in the art, several improvements and modifications may be made to the present invention without departing from the principles of the present invention, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

[0019] The present invention provides a method and device for monitoring human and vehicle intrusion. The method of the present invention comprises the steps of: The human-vehicle intrusion detection dataset is divided into a training set and a validation set, and the training set data is preprocessed to obtain labeled training set images.

[0020] The KMeans++ clustering method is used to cluster the annotation boxes of the training set images to generate a priori box diagram.

[0021] An improved YOLOv7 target detection network model is constructed, which includes a backbone network, a neck network, a detection head and a loss function; the backbone network extracts multi-scale features from the prior frame graph, the neck network fuses multi-scale features, the detection head is obtained by detecting feature maps at multiple scales, and the loss function is constructed based on bounding box loss, confidence loss, category loss and their corresponding adjustable weight coefficients.

[0022] Based on the preprocessed training set images and validation set, the improved YOLOv7 target detection network model is trained using transfer learning.

[0023] The preprocessed images collected by the surveillance camera are input into the trained improved YOLOv7 target detection network model for inference to obtain the inference results, and then the inference results are filtered by borders. The filtered inference results are then saved in a time series table. When the number of times the same object appears in the time series table reaches the set threshold, an intrusion alarm for the object is output.

[0024] like Figure 1 As shown, the detailed steps of the method for monitoring human and vehicle intrusion of the present invention include: Step 1: Select an experimental scene, place the surveillance camera equipment at a height of 3 m and an incident angle of 75°, and record continuously for 3 hours under dynamic conditions in three time periods: morning, noon, and evening, and in three weather environments: sunny, cloudy, and rainy. Export the obtained video data into image data at a rate of 1 second per frame, and then organize, clean, and annotate it to obtain the human and vehicle intrusion detection dataset.

[0025] Step 2: Divide the human-vehicle intrusion detection dataset obtained in step 1 into a training set, a validation set, and a test set. Then enhance the images in the training set, align the enhanced images and annotations to the original size and add them to the training set.

[0026] Step 3: Use the KMeans++ clustering algorithm to calculate the annotation boxes of the training set images and generate four groups of prior boxes of different sizes.

[0027] Step 4: Figure 2As shown, an improved YOLOv7 target detection network model is constructed. The improved YOLOv7 target detection network includes a backbone network, a neck network, a detection head and a loss function, which corresponds to the backbone network Backbone, the neck network Neck, the head network Head and the Loss loss function part in the figure, where: The backbone network is used to extract features and is a stack of CBS, ELAN and MPASC modules. A total of 32 times downsampling is performed to obtain feature maps of a total of 5 scales: P1, P2, P3, P4 and P5.

[0028] The neck network is used to fuse multi-scale features, which is a bottom-up feature pyramid FPN network and a top-down path hybrid PAN network composed of CBS, SPPFCSP, ELAN, MPASC and RepVGG modules.

[0029] The CBS, ELAN, MPASC, SPPFCSP and RepVGG modules are universal modules with different structures. The specific composition of each module is as follows: Figure 3 shown.

[0030] The FPN and PAN networks are two components of the neck network, such as Figure 2 As shown, they are used for the fusion of deep semantic information to shallow detail information and the fusion of shallow detail information to deep semantic information.

[0031] The P1, P2, P3, P4 and P5 feature maps are five scale levels of the network that gradually decrease by a factor of 2 and are used to capture features at different levels of the image. The corresponding parts of each feature map are as follows: Figure 2 shown.

[0032] The detection head is used to output multi-scale feature maps for detection, and detection is performed on P1, P2 and P3 feature maps, that is, feature maps downsampled by 2 times, 4 times and 8 times, respectively, to obtain detection heads of 3 scales.

[0033] The loss function is composed of the bounding box loss L bbox , confidence loss L obj Class loss L cls The overall loss function constructed is calculated as follows: ; Where λ1, λ2, and λ3 are all adjustable weight coefficients.

[0034] In the constructed loss function, the bounding box loss is CIoU loss, and the confidence loss and category loss are BCE loss, that is, binary cross entropy loss.

[0035] Step 5: Use transfer learning to train the improved YOLOv7 target detection network model in step 4, that is, first pre-train on the large target detection dataset COCO dataset, and then fine-tune it on the human and vehicle intrusion detection dataset. Train for 100 rounds of iterations. After each round of iteration, save the weights and test the accuracy on the validation set. Finally, take the weight with the highest accuracy on the validation set as the weight of the final model.

[0036] Step 6: Build an image reasoning pipeline. First, filter the image captured by the surveillance camera for clarity, then correct the image brightness to alleviate overexposure and underexposure, then input the trained YOLOv7 target detection network model in step 5 for reasoning, filter the reasoning results for large frames, filter out the bounding boxes whose width or height is greater than 80% of the width and height of the original image, and then save the reasoning results into a time series table. When the number of times the same object appears in the time series table reaches the set threshold, an alarm of human or vehicle intrusion is output.

[0037] In step 1, the selected experimental scene is a scene with dense human and vehicle activities, which can provide relatively rich sample data; the different time periods and weather conditions are selected to increase the diversity of data, so as to enhance the generalization ability and robustness of the model after training.

[0038] In step 1, the edge computing chip used is RK3568 produced by Rockchip Microelectronics, and the surveillance camera is HF500 produced by Jierui Microelectronics.

[0039] In step 2, the image enhancement methods used include the following four methods: HSV enhancement is used to randomly transform the hue, brightness and saturation of the original image.

[0040] Affine transformation enhancement is used to randomly flip, rotate and scale the original image.

[0041] Mixup enhancement is used to mix two different human and vehicle intrusion detection images in a certain proportion to generate a new image.

[0042] Mosaic-4 data enhancement is used to randomly select 4 images in the human-vehicle intrusion detection dataset, perform geometric transformation enhancement, and then splice them into 1 image.

[0043] In step 3, the steps of the Kmeans++ clustering algorithm are as follows: since the total number of prior boxes is 12, 12 cluster centers are set; first, a sample point is randomly selected from the data set as the first initial cluster center; then, the shortest distance between each sample and the current existing cluster center is calculated, and the probability of each sample point being selected as the next cluster center is calculated based on this distance, and the sample point with the largest probability is selected as the next cluster center; this process is repeated until 12 cluster centers are selected.

[0044] In step 4, the improved YOLOv7 target detection network is improved compared to the original YOLOv7 target detection network in that it adopts deep separable convolution, which greatly reduces the number of parameters and the amount of calculation compared to the standard convolution. The specific steps are: for an input feature map of size , the convolution kernel size is , the number of input channels is c i , the number of output channels is c o , the output feature map size is the same as the input, then the computational amount MACs of the standard convolution is: ; The computational amount MACs of depth-separable convolution is: ; The ratio of the computational effort of standard convolution to depth-wise separable convolution is: In the YOLOv7 target detection network, the convolution kernel size is 3×3, so the computational complexity of the depthwise separable convolution is about 1 / 9 of that of the standard convolution.

[0045] The number of parameters for standard convolution is: ; The number of parameters of depth-wise separable convolution is: ; The ratio of the computational effort of standard convolution and depth-wise separable convolution is: In the YOLOv7 target detection network, the convolution kernel size is 3×3, so the number of parameters of the depthwise separable convolution is about 1 / 9 of the standard convolution.

[0046] In step 4, the improvement of the improved YOLOv7 target detection network compared to the original YOLOv7 target detection network is that the FPN network and PAN network of its neck network input the P1, P2, P3, P4 and P5 feature maps into the feature fusion network, enhancing the network's information fusion of small targets; its multi-scale detection head outputs P1, P2 and P3 detection heads, enhancing the network's detection capability for small targets.

[0047] In step 4, the calculation formula of the CIoU loss is: ; In the formula, represents the Euclidean distance between the center points of the predicted box and the true box; c represents the diagonal distance of the minimum enclosing area that can contain both the predicted box and the true box; the calculation formulas for α and v are: ; ; In the formula, w, h and w gt 、h gt Represent the width and height of the predicted box and the width and height of the real box respectively; The CIoU loss not only considers the overlap degree IoU between the predicted box and the true box, but also considers their center point distance and aspect ratio, making the predicted box more consistent with the true box, improving the accuracy of the model and accelerating the convergence of the model.

[0048] In step 6, the clarity filtering is as follows: Laplacian filtering is performed on the image, and then the variance of the filtered image is calculated. If the variance value is too small, it means that the image is more blurred, so it is filtered; otherwise, the image is retained. The large frame filtering is as follows: if the frame or height of the output boundary box exceeds the set threshold, it is filtered; otherwise, the output result is retained.

[0049] In step 6, the brightness correction method is to perform histogram equalization on the image, and the steps are: first, calculate the cumulative histogram and count the number of times each gray level appears in the histogram; then, perform interval conversion on the cumulative histogram and map the value of the cumulative histogram to a new pixel value range; in the cumulative histogram, original values ​​with similar probabilities will be processed as the same value, so that the pixel values ​​originally concentrated in the distribution are evenly distributed to the range of all possible values, thereby effectively alleviating the problems of overexposure and underexposure.

[0050] The present invention also provides a device for monitoring human and vehicle intrusion, which comprises: The preprocessing module divides the human-vehicle intrusion detection dataset into a training set and a validation set, and preprocesses the training set data to obtain labeled training set images; The clustering module uses the KMeans++ clustering method to cluster the annotation boxes of the training set images and generate a priori box diagrams; Module construction module, build an improved YOLOv7 target detection network model, which includes a backbone network, a neck network, a detection head and a loss function; the backbone network extracts multi-scale features from the prior frame map, the neck network fuses multi-scale features, the detection head is obtained by detecting feature maps at multiple scales, and the loss function is constructed based on bounding box loss, confidence loss, category loss and their corresponding adjustable weight coefficients; The training module uses transfer learning to train the improved YOLOv7 target detection network model based on the preprocessed training set images and validation set; The detection module inputs the pre-processed images collected by the surveillance camera into the trained improved YOLOv7 target detection network model for inference to obtain the inference results, then filters the inference results by borders, and then saves the filtered inference results in a time series table. When the number of times the same object appears in the time series table reaches the set threshold, an alarm for the intrusion of the object is output.

[0051] Figure 4 It is a schematic diagram of the human and vehicle intrusion detection effect of the improved target detection network model of the present invention and the YOLOv7 benchmark model, which gives a comparison of the detection effects of the improved model of the present invention and the benchmark model.

[0052] In summary, the present invention relates to a method for monitoring human and vehicle intrusion in the field of video surveillance. The method includes three parts. First, based on the video data collected by the surveillance camera, a set of human and vehicle intrusion detection data sets is constructed and an image data preprocessing method is proposed; second, based on the YOLOv7 target detection network model, a small target detection layer is added to increase the detection capability of small targets at a long distance, and the backbone network is lightweight to reduce the number of parameters and the amount of calculation of the model; third, in the real road monitoring environment, for the low quality of pictures collected at certain times and false alarm phenomena, a method for image quality filtering and repair and a false alarm filtering strategy are proposed, which realizes high-precision monitoring of human and vehicle intrusion, and has real-time and robustness in late-night environments.

Claims

1. A method for monitoring human and vehicle intrusion, characterized in that: It includes the steps of: The human-vehicle intrusion detection dataset is divided into a training set and a validation set, and the training set data is preprocessed to obtain labeled training set images; Use KMeans++ clustering method to cluster the annotation boxes of the training set images and generate a priori box diagram; An improved YOLOv7 target detection network model is constructed, which includes a backbone network, a neck network, a detection head, and a loss function. The backbone network extracts multi-scale features from the prior frame map, the neck network fuses multi-scale features, the detection head is obtained by detecting feature maps at multiple scales, and the loss function is constructed based on bounding box loss, confidence loss, category loss, and their corresponding adjustable weight coefficients. Based on the preprocessed training set images and validation set, the improved YOLOv7 target detection network model is trained using transfer learning. The images collected by the surveillance camera are preprocessed and input into the trained improved YOLOv7 target detection network model for inference to obtain the inference results. The inference results are then filtered by borders and saved in a time series table. When the number of times the same object appears in the time series table reaches the set threshold, an intrusion alarm for the object is output.

2. The method according to claim 1, characterized in that: The method for obtaining the human-vehicle intrusion detection dataset is as follows: The experimental scene was selected, the height and incident angle of the surveillance camera were set, and videos were continuously recorded under dynamic conditions in the morning, noon, evening, sunny, cloudy, and rainy days. The video data was output at a set frame rate to generate image data. After removing irrelevant data and labeling, a human and vehicle intrusion detection dataset was obtained.

3. The method according to claim 2, characterized in that Preprocessing the training set data includes: enhancement processing; which specifically includes: Use HSV enhancement to randomly transform the hue, brightness and saturation of the original image; Affine transformation enhancement is used to randomly flip, rotate and scale the original image; Mixup enhancement is used to mix two different human and vehicle intrusion detection images in a certain proportion to generate a new image; Mosaic-4 data enhancement is used to randomly select 4 images from the human-vehicle intrusion detection dataset, perform geometric transformation enhancement, and then splice them into 1 image; The enhanced images and annotations are aligned to the original image size and added to the training set.

4. The method according to claim 3, characterized in that: The improved YOLOv7 target detection network model adopts deep separable convolution; The backbone network downsamples the prior frame graph to obtain feature graphs of a total of 5 scales: P1, P2, P3, P4 and P5; The neck network fuses multi-scale feature maps to form an FPN network and a PAN network, the FPN network is used to fuse the deep semantic information of the network to the shallow detail information, and the PAN network is used to fuse the shallow detail information to the deep semantic information; The detection head performs detection on the feature maps P1, P2, and P3 to obtain the corresponding detection head.

5. The method according to claim 1, characterized in that The bounding box loss is CIoU loss, and the confidence loss and category loss are BCE loss.

6. The method according to claim 1, characterized in that The method of using transfer learning to train the improved YOLOv7 target detection network model specifically includes: The improved YOLOv7 target detection network model is first pre-trained on the COCO dataset, and then trained on the human-vehicle intrusion detection dataset. The training is repeated in rounds. After each round of iteration, the weights are saved and the accuracy is tested on the validation set. Finally, the weight with the highest accuracy on the validation set is taken as the weight of the final model.

7. A device for monitoring human and vehicle intrusion, characterized in that: It includes: The preprocessing module divides the human-vehicle intrusion detection dataset into a training set and a validation set, and preprocesses the training set data to obtain labeled training set images; The clustering module uses the KMeans++ clustering method to cluster the annotation boxes of the training set images and generate a priori box diagram; Module construction module, build an improved YOLOv7 target detection network model, which includes a backbone network, a neck network, a detection head and a loss function; the backbone network extracts multi-scale features from the prior frame map, the neck network fuses multi-scale features, the detection head is obtained by detecting feature maps at multiple scales, and the loss function is constructed based on bounding box loss, confidence loss, category loss and their corresponding adjustable weight coefficients; The training module uses transfer learning to train the improved YOLOv7 target detection network model based on the preprocessed training set images and validation set; The detection module pre-processes the images collected by the surveillance camera and inputs them into the trained improved YOLOv7 target detection network model for inference to obtain the inference results, then performs border filtering on the inference results, and then saves the filtered inference results in the draft time series table. When the number of times the same object appears in the time series table reaches the set threshold, an alarm for the intrusion of the object is output.

8. The device according to claim 7, characterized in that Preprocessing the training set data includes: enhancement processing; which specifically includes: Use HSV enhancement to randomly transform the hue, brightness and saturation of the original image; Affine transformation enhancement is used to randomly flip, rotate and scale the original image; Mixup enhancement is used to mix two different human and vehicle intrusion detection images in a certain proportion to generate a new image; Mosaic-4 data enhancement is used to randomly select 4 images from the human-vehicle intrusion detection dataset, perform geometric transformation enhancement, and then splice them into 1 image; The enhanced images and annotations are aligned to the original image size and added to the training set.

9. The device according to claim 7, characterized in that The improved YOLOv7 target detection network model adopts deep separable convolution; The backbone network downsamples the prior frame graph to obtain feature graphs of a total of 5 scales: P1, P2, P3, P4 and P5; The neck network fuses multi-scale feature maps to form an FPN network and a PAN network, the FPN network is used to fuse the deep semantic information of the network to the shallow detail information, and the PAN network is used to fuse the shallow detail information to the deep semantic information; The detection head performs detection on the feature maps P1, P2, and P3 to obtain the corresponding detection head.

10. The device according to claim 7, characterized in that The method of using transfer learning to train the improved YOLOv7 target detection network model specifically includes: The improved YOLOv7 target detection network model is first pre-trained on the COCO dataset, and then trained on the human-vehicle intrusion detection dataset. The training is repeated in rounds. After each round of iteration, the weights are saved and the accuracy is tested on the validation set. Finally, the weight with the highest accuracy on the validation set is taken as the weight of the final model.

Images