Unmanned aerial vehicle fire fighting access occupation detection method based on YOLOv8 and SAM

Abstract

The invention discloses an unmanned aerial vehicle fire fighting access occupation detection method based on YOLOv8 and SAM, and the method comprises the steps: S1, preparing a data set: making a to-be-detected data set, and taking the to-be-detected data set as an input detection image; meanwhile, making a model training data set; s2, training a YOLOv8-P2 model: carrying out model training by adopting the model training data set to obtain an improved YOLOv8-P2 model; s3, fire fighting access and vehicle detection: using an improved YOLOv8-P2 target detection algorithm in the improved YOLOv8-P2 model to detect and analyze the to-be-detected image P, and identifying all vehicles and fire fighting accesses contained in the image; and S4, reporting information. According to the method, the detection speed and accuracy of the small target vehicle and the fire fighting access under the view angle of the unmanned aerial vehicle can be remarkably improved, and the fire fighting access area in an irregular shape is accurately segmented by using the SAM, so that the IOU of the vehicle area and the fire fighting access area is calculated.

Description

Technical Field

[0001] The present invention belongs to the field of novel artificial intelligence technology, and in particular relates to a method for detecting the occupancy of a fire channel of an unmanned aerial vehicle based on YOLOv8 and SAM. Background Art

[0002] At present, there is a phenomenon of fire truck walkways being occupied and blocked in residential areas, industrial parks, large buildings and other places. Once an emergency such as a fire occurs and fire trucks need to pass through, if these roads are occupied or blocked by other vehicles, the fire cannot be handled in time, which will further increase and aggravate the losses caused by the fire. Therefore, ensuring the smooth passage of fire trucks is a must for each park management and fire inspection.

[0003] Traditionally, fixed cameras are used to perform analysis and judgment using traditional image processing methods. Traditional image similarity algorithms are used to detect the similarity between normal channel images and blocked channel images to determine whether the channel is blocked, and then push alarm information to the server background for processing. At the same time, there is no special road occupation recognition system in the existing image recognition system. Most of the existing vehicle recognition systems are image classification systems that can recognize vehicles and roads, but cannot handle all problems in a refined manner; in the existing system, if the vehicle accounts for a small proportion of the image, the vehicle will not be recognized, and the road will also not be recognized when the image accounts for a small proportion; fire channels are more difficult to recognize in the picture, and the existing system cannot count the time a vehicle stays at a certain location.

[0004] Since most fire lanes are irregular in shape, the predicted road rectangle may contain a large part of the area that does not belong to the fire lane, or even an area where parking is allowed. If the judgment is made only by calculating the overlapping area of ​​the rectangular boxes, the probability of false detection will be significantly increased. In addition, traditional image segmentation tasks usually have relatively clear boundaries and limitations. For example, in this process, the input data is a picture, and the task of the model is to segment the picture according to the specific classification criteria learned during the training process. The core of this segmentation method is that the training data set of the model already contains all the object categories that need to be identified and segmented. The segmentation ability of the model is based on the limited categories it has been exposed to. When faced with a new segmentation task, especially when it involves classifications that the model has never seen, this traditional segmentation model will appear to be powerless.

[0005] Chinese patent document CN 117576080A discloses a method and device for detecting rockfill gradation based on drone aerial survey and instance segmentation, including: relying on the local surface rockfill image before the density pit excavation and the rockfill screening result of the density pit, establishing the conversion relationship between the surface gradation and the overall gradation; using drone close-range photogrammetry technology to obtain the image of the filling bin surface of the rockfill dam; generating orthophotos of the filling bin surface; making image data sets based on the orthophotos; training the instance segmentation model jointly constructed by YOLOv8 and Unet to obtain the instance segmentation model for rockfill segmentation; obtaining the orthophotos of the filling bin surface of the rockfill dam to be predicted and cutting them into small blocks, using the instance segmentation model for prediction, realizing pixel-level instance segmentation and morphological extraction; splicing the predicted blocks, performing morphological quantitative analysis on the predicted rockfill instance, and obtaining the rockfill gradation of the entire bin surface after gradation conversion. In this technical solution, the SAM model is used to annotate data, and Unet is used for segmentation.

[0006] Chinese patent document CN 118172605 A A semantic segmentation label annotation method for RGB-T dual-modal images, used for labeling image data sets, including the following steps: Step 1 performs bounding box annotation for the image data set to be processed, Step 2 uses bounding box annotation and SAM model to obtain the first round of prediction masks of visible light image and thermal image respectively as RGB label and thermal label; Step 3 For the image data set to be optimized, calculate the enhancement point prompt, use SAM model to obtain the second round of prediction mask as the optimized RGB label and thermal label; Step 4 reviews through annotation software, corrects the existing errors and omissions and merges the labels to obtain the final RGB-T label. This technical solution is aimed at annotating data sets, using SAM to improve the quality of data annotation, rather than targeting specific target segmentation tasks.

[0007] Chinese patent document CN116543353A discloses a fire lane road occupation identification method, system, electronic device and storage medium, the method includes: patrolling in the fire truck lane area in the patrol environment by a robot; the robot is equipped with a camera; obtaining shooting information in the fire truck lane area, the shooting information includes pictures and video streams; pushing the shooting information to the background, and using a deep learning method to analyze and judge the shooting information to detect whether there is a vehicle in the shooting information; detecting the vehicle that appears in the shooting information, and processing the detection result with a tracking algorithm according to the time sequence to determine the vehicle's bounding box and the tracked vehicle category; judging the vehicle's state according to the vehicle's bounding box and the vehicle category to determine whether the vehicle is illegally parked and occupying the road. This method uses the YOLOv5 method in deep learning to locate the banner in the image, and modifies the original backbone network of YOLOV5 into a lightweight network such as Mobilenetv3. Part of the model's training data is open source on the Internet, and the other part is collected and labeled by ourselves; this method can detect the occupancy of fire passages in fixed scenarios, but if the scenario is complex and changeable, the algorithm deployment will be very difficult. Therefore, in actual detection, this method is difficult to deploy quickly, and the professional and technical requirements of the deployment personnel are very high, and it is not suitable for large-scale deployment.

[0008] Therefore, it is necessary to provide a detection method for UAV fire channel occupancy based on YOLOv8 and SAM. When identifying the occupancy of irregular-shaped fire channels from the perspective of a drone, the main advantages of the improved YOLOv8-P2 and SAM over traditional methods are reflected in real-time performance, detection accuracy and segmentation quality. The improved YOLOv8-P2 has significant advantages in small target detection and processing speed, while the SAM model performs well in accurate segmentation and processing of complex backgrounds. Summary of the invention

[0009] The technical problem to be solved by the present invention is to provide a detection method for the occupancy of a UAV fire passage based on YOLOv8 and SAM, which can significantly improve the detection speed and accuracy of small target vehicles and fire passages from the perspective of the UAV, and use SAM to accurately segment the irregularly shaped fire passage area, thereby calculating the IOU of the vehicle area and the fire passage area, which is suitable for irregularly shaped fire passages.

[0010] In order to solve the above technical problems, the technical solution adopted by the present invention is: the detection method of the occupancy of the UAV fire channel based on YOLOv8 and SAM specifically comprises the following steps:

[0011] S1: Prepare the dataset: Prepare the dataset to be tested as the input test image; prepare the model training dataset at the same time;

[0012] S2 trains the YOLOv8-P2 model: an improved YOLOv8-P2 model is obtained by using the model training dataset for model training;

[0013] S3 Fire lane and vehicle detection: Use the improved YOLOv8-P2 target detection algorithm in the improved YOLOv8-P2 model to detect and analyze the image P to be detected, and identify all vehicles and fire lanes contained in the image;

[0014] S4 Report information: Collect and report information on illegal vehicles.

[0015] Using the above technical solution, firstly, by collecting image data of different vehicle categories and fire lanes, the improved YOLOv8-P2 is incrementally learned, and the predicted rectangular frame position area of ​​the vehicle and fire lane in the image to be detected P is identified with its help; then the center coordinate point a and any point coordinate b in the rectangular frame position area of ​​the fire road are selected as the prompt input method of the SAM model; the mask FD of the fire road in the image P is accurately segmented by the provided two-point coordinates and the SAM model. Finally, the fire road mask FD and the rectangular frame mask Car of the vehicle in the image P are IOU-calculated for the overlapping area. If the overlapping area exceeds the threshold, it is determined to occupy the fire lane and take corresponding feedback measures. The YOLO series algorithms in the existing scheme have low efficiency and low recognition accuracy for fire lanes and vehicles. Therefore, the improved YOLOv8-P2 algorithm is used to improve the detection accuracy and speed of the algorithm for fire lanes and small target vehicle recognition. At the same time, compared with the ViT model, the improved YOLOv8-P2 has faster real-time detection capability, higher small target accuracy and lower computing resource requirements than the ViT model. First, the improved YOLOv8-P2 uses the improved CSPDarknet as the backbone network and optimizes the feature extraction process using the cross-stage part (CSPNet). This structure improves the efficiency of feature extraction by separating and optimizing the feature extraction steps. For the recognition of different vehicle categories, this optimization can help the model better capture the subtle differences between different vehicles, thereby improving the classification accuracy. Secondly, the improved YOLOv8-P2 uses PANet as a feature fusion module. This pyramid network aggregates features through bottom-up paths, so that features at different levels can be better fused. For fire channel recognition, this effective fusion of multi-scale features enhances the detection capability of channels of various sizes and shapes, enabling the model to more accurately identify and locate fire channels and reduce false detection rates. In the detection head part, the improved YOLOv8-P2 has been improved by combining the tasks of target classification, positioning regression and confidence prediction. Through efficient convolutional layers and attention mechanisms, especially the Squeeze-and-Excitation module, the improved YOLOv8-P2 can better focus on key features, further improving detection accuracy. For different vehicle categories, especially those that may be similar in appearance, the model's fine feature extraction and attention mechanism can effectively distinguish them. For fire lane recognition, the enhanced feature expression capability helps the model accurately capture the boundaries and locations of fire lanes. For complex scenarios, such as traffic environments with multiple vehicle types or fire lanes with multiple features, this optimization of the improved YOLOv8-P2 can provide more accurate detection results.Therefore, the improved YOLOv8-P2 mainly benefits from its advanced network structure design, optimized feature fusion mechanism, fine feature extraction capability and powerful attention mechanism in different vehicle identification and fire channel identification. These factors work together to enable it to achieve higher detection accuracy when dealing with complex scenes and tasks. In addition, the present invention uses the improved YOLOv8-P2 combined with the SAM precise segmentation technology, and significantly reduces the false detection rate by calculating the intersection and union (IOU) of the fire channel and the vehicle. Compared with the traditional rectangular box IOU calculation, this method uses SAM to accurately segment the fire channel, which can more accurately identify and locate the fire channel, thereby effectively reducing false detection.

[0016] Preferably, the specific steps in step S1 are:

[0017] S11: Using a drone to photograph a fixed area where the fire passage occupancy situation needs to be detected, to obtain an image P to be detected as an input detection image;

[0018] S12: Use ffmpeg tool to perform frame sampling on the collected drone video, extract key frames, and obtain fire channel image data;

[0019] S13: collecting vehicle image data of different categories in the gallery;

[0020] S14: Filter the obtained vehicle image data and fire passage image data respectively to obtain a vehicle image data set and a fire passage image data set.

[0021] Preferably, the specific steps of step S2 are:

[0022] S21: Segment the collected vehicle image data and fire passage image data, and manually label the collected vehicle image data and fire passage image data using the LabelImg tool to ensure that the target objects in each image are accurately labeled; after labeling, divide the data set into a training set and a validation set to facilitate subsequent model training and evaluation;

[0023] S22: Build an improved YOLOv8-P2 algorithm model, input the labeled data of the training set and perform model training to obtain the algorithm model weight file best.pt.

[0024] Preferably, the SE-Block attention mechanism is introduced in step S22, and the loss function CIoU in the algorithm is replaced by the HIoU loss function. The HIoU loss function introduces a dynamic adjustment mechanism and integrates multiple IoU metrics. The HIoU loss function is expressed as a weighted combination of the following forms:

[0025] HIoU=α·IoU+β·GIoU+γ·DIoU;

[0026] Among them, α and β are weights in the total loss; IoU is the most basic metric, which is used to calculate the overlap ratio between the predicted box and the real box; Generalized IoU is an extension of IoU, which introduces the minimum closure area of ​​the predicted box and the real box; Distance IoU is further improved on the basis of generalized IoU, increasing the distance between the center point of the predicted box and the real box. By using HIoU as the loss function of YOLOv8-P2 and combining it with rotating target detection, the detection speed and accuracy of small target vehicles and fire channels from the perspective of drones can be significantly improved.

[0027] With the above technical solutions, YOLOv3 has low accuracy in detecting small targets, while the improved YOLOv8-P2 has greatly improved its recognition ability for small targets through an improved feature fusion mechanism and a deeper network architecture, which is particularly important for detecting small obstacles or distant vehicles in fire lanes. Secondly, YOLOv3's target recognition accuracy is insufficient in complex backgrounds. The improved YOLOv8-P2 introduces attention mechanisms such as SE-Block and improves the loss function, which enhances its adaptability to complex backgrounds and occluded targets and improves its detection performance in complex environments. For rotated and tilted targets, YOLOv3 has limited detection capabilities, while the improved feature extraction and detection strategies of the improved YOLOv8-P2 significantly improve the recognition capabilities of these targets and adapt to the diverse postures of targets in fire lanes. Finally, YOLOv3 performs poorly in dealing with targets with large scale changes, while the improved feature pyramid network (FPN) of the improved YOLOv8-P2 improves the detection capabilities of targets of different scales. In addition, in the fire channel and vehicle recognition tasks, the improved YOLOv8-P2 shows significant advantages over the VisionTransformer (ViT) model. First, the design of the improved YOLOv8-P2 focuses on real-time performance and detection accuracy. Its improved feature fusion technology (CSPNet) and optimized inference algorithm enable it to perform well in dealing with complex scenes and small targets, and is suitable for real-time application requirements. In contrast, the ViT model has a large computational overhead based on the self-attention mechanism, which usually requires more computing time and resources, resulting in it being less efficient than the improved YOLOv8-P2 in real-time target detection tasks. First, the improved YOLOv8-P2 uses an improved CSPDarknet as the backbone network. This structure optimizes the feature extraction process through the cross-stage part (CSPNet), reducing redundant calculations and memory usage. CSPNet effectively reduces the computational overhead by dividing the feature extraction process into multiple stages and optimizing each stage, so that the model can run efficiently on lower-performance hardware. Compared with the traditional network structure, this design significantly improves the processing speed and reduces the computing requirements. Secondly, the feature fusion module of the improved YOLOv8-P2 uses PANet. PANet enhances the fusion capability of multi-scale features through a bottom-up feature aggregation path. This feature fusion mechanism can effectively optimize the calculation process and avoid unnecessary computational redundancy, thereby improving the real-time processing capability of the model. PANet effectively combines feature information at different levels, enabling the improved YOLOv8-P2 to process target information in complex scenes faster.

[0028] The known technical contents underlying the present invention include: YOLOv8-P2 target detection algorithm and SAM target segmentation algorithm.

[0029] (1) Improved YOLOv8-P2 target detection algorithm

[0030] The YOLO algorithm was originally proposed by Joseph Redmon and others. Its core idea is to complete target detection and classification simultaneously in a single network forward pass, which significantly improves the detection speed and real-time performance. As the latest version of the YOLO series, YOLOv8 is mainly due to the continuous optimization of the accuracy and speed of previous generations of algorithms. Its purpose is to further improve the accuracy of target detection, reduce false positives and false negatives, and improve computational efficiency and processing speed to adapt to more complex scenarios and application requirements. YOLOv8-P2 is a variant of the YOLOv8 model designed to improve the recognition performance of small objects. Compared with the original YOLOv8 model, it contains additional feature pyramid levels, which enables it to detect and locate smaller objects more accurately.

[0031] (2) SAM target segmentation algorithm

[0032] The SAM algorithm was proposed to address the challenges in the object segmentation task. Object segmentation requires accurate segmentation of the object region of interest from the image, which is very important for various computer vision tasks. Traditional segmentation methods often have restrictions on the type of objects and backgrounds, while the SAM algorithm aims to solve these problems through a general deep learning model, enabling accurate segmentation in a variety of different scenarios and applications. Its purpose is to improve the versatility and accuracy of segmentation, thereby expanding its application scope and improving efficiency.

[0033] Preferably, the specific steps of step S3 are:

[0034] S31: performing preprocessing, normalization processing and data enhancement processing on the image P to be detected in sequence;

[0035] S32: Input the image P to be detected, load the algorithm model weight file best.pt to perform prediction analysis on the image P to be detected, and obtain the rectangular frame position area of ​​the vehicle and the fire bucket;

[0036] S33: Use the deep learning SAM target segmentation algorithm to segment the fire passage area, that is, segment the fire passage area M in the image to be detected P and generate a mask FD;

[0037] S34: Calculate the IOU of the fire road mask FD and the rectangular frame mask Car of the vehicle in the image to be detected P, and determine the occupation of the fire lane through IOU. If the overlapping area exceeds the threshold, it is determined to be occupied by the fire lane, and then go to step S4; if it is determined to be unoccupied, then end the detection. Since the main advantage of SAM compared to traditional models (FCN, UNet, MaskR-CNN, etc.) lies in its versatility and flexibility, it can perform well in a variety of tasks and can work effectively without a large amount of labeled data. The SAM model is directly used for segmentation tasks, but the SAM model alone cannot classify specific tasks. Therefore, yolov8 is used to provide SAM with point prompt coordinates, so that it can accurately segment the large detection object area, thereby segmenting the fire lane area. The SAM model is used to accurately segment the fire lane area, and it is calculated by IOU with the vehicle rectangular frame area to improve the recognition accuracy of vehicles occupying fire lanes.

[0038] Preferably, in step S31, the image P to be detected is reduced to the same size through preprocessing, and then normalized to scale the pixel values ​​to the range of [0, 1], and then data enhancement processing is performed on the image P to be detected, including random cropping, rotation, flipping, brightness and contrast adjustment.

[0039] Preferably, in step S32, YOLOv8-P2 rotating target detection is used as the recognition algorithm to obtain the rectangular frame position area of ​​the vehicle and the fire bucket, wherein the calculation formula of the rotating rectangular frame coordinates is:

[0040]

[0041] Among them, x i and i Represents the coordinates of the four corner points, where i is 1 to 4, corresponding to the four corner points of the rectangular box; x c and c Respectively represent the horizontal and vertical coordinates of the center point of the rectangular frame, w and h are the width and height of the rectangular frame, respectively, which are used to calculate the corner point position from the center point; θ is the rotation angle of the rectangular frame relative to the horizontal line, expressed in radians, and the positions of the four corner points after rotation are accurately calculated by combining the rotation angle and trigonometric functions. In order to accurately identify and locate tilted or rotated vehicles and fire channel targets in images, adapt to complex outdoor scenes, improve the accuracy and robustness of detection, and reduce false detection and missed detection, the present invention uses improved YOLOv8-P2 rotation target detection as the recognition algorithm based on the improved YOLOv8-P2.

[0042] Preferably, the specific steps of step S33 are:

[0043] S331: According to the obtained rectangular frame position area of ​​the vehicle and the fire bucket, the center point coordinate a and any point coordinate b in the rectangular frame area of ​​the fire passage are selected as model prompt input;

[0044] S332: Build a SAM segmentation model to segment the image P to be detected;

[0045] S333: According to the center point coordinates a and b, use the SAM segmentation model to segment the fire passage area M in the image to be detected P and generate a mask FD.

[0046] Preferably, the step S331 is specifically as follows: the coordinate information of the four vertices of the rectangular frame area of ​​the fire passage obtained by the step S32 can determine the coordinate a of the center point of the rectangular frame area, and select any point therein as the coordinate b; the calculation formula of the center point coordinate a is:

[0047]

[0048] Among them, (x c ,y c ) represents the selection of the coordinates a of the center point in the rectangular frame area of ​​the fire passage; (x1, y1), (x2, y2), (x3, y3), (x4, y4) are the coordinate information of the four vertices of the rectangular frame area of ​​the fire passage; a random point b is generated in the Cartesian coordinate system in the rectangular area of ​​the fire passage, and its coordinate calculation formula is:

[0049] x rand =x1+(x2-x1)×r x ;

[0050] y rand =y1+(y2-y1)×r y ;

[0051] Among them, (x rand ,y rand ) represents the selection of a random coordinate point b within the rectangular frame area of ​​the fire passage; (x1, y1) and (x2, y2) are the coordinate information of two vertices on the main diagonal line of the rectangular frame area of ​​the fire passage, r x and r y is a random number in the range [0,1].

[0052] Preferably, the specific steps of step S34 are:

[0053] S341: First, read the rectangular frame coordinate information of all vehicles in the image to be detected P analyzed by the improved YOLOv8-P2, and generate a vehicle mask Car according to the rectangular frame coordinate information;

[0054] The main features of the generated mask Car involve several key aspects. First, the mask image is binary and contains only two pixel values: the foreground area is marked as white, and the background area is marked as black, which makes it possible to clearly distinguish between the target and the background. Secondly, the mask image generated based on the bounding box is rectangular in shape, covering the area within the bounding box. The internal pixels are marked as foreground, and the rest of the area is background. In addition, the size of the mask image is consistent with the original image (the original image size is 1000×750) to ensure accurate alignment and processing at the pixel level.

[0055] S342: then read the fire channel mask FD segmented by the SAM segmentation model;

[0056] S343: Finally, the pixels at the same position in the vehicle mask Car and the fire channel mask FD are bitwise ANDed to obtain the mask result_mask of the overlapping area;

[0057] S344: Use the mask result_mask to calculate the number of pixels true_count in the overlapping area, and similarly calculate the total number of pixels total_elements of the fire channel mask FD, so as to obtain the IOU, and use the IOU to determine whether the vehicle illegally occupies the fire channel. Use SAM to accurately segment the irregularly shaped fire channel area, and calculate the IOU of the vehicle area and the fire channel area. This method effectively solves the problem of false detection of fire channels caused by relying only on rectangular boxes to calculate IOU.

[0058] Preferably, in step S344, the IOU is calculated according to the number of pixels true_count and the total number of pixels total_elements in the overlapping area, and the formula is:

[0059]

[0060] Among them, A intersection is the area of ​​the intersection, which is equivalent to the number of pixels true_count in the overlapping area; A union is the area of ​​the union, which is equivalent to the total number of pixels total_elements; the IOU value ranges from 0 to 1, and the higher the value, the higher the overlap between the predicted box and the real box; the IOU value is the judgment result, that is, the proportion of the overlapping area in the fire channel mask is calculated; if the judgment result result value is greater than the threshold 0, it is determined that the vehicle illegally occupies the fire channel.

[0061] Compared with the prior art, the present invention has the following beneficial effects:

[0062] (1) High accuracy: The high accuracy of the improved YOLOv8-P2 is due to its advanced network structure design and optimization of key components, which are particularly significant in different vehicle recognition and fire lane recognition tasks;

[0063] (2) Real-time performance: The reason why the improved YOLOv8-P2 performs well in real-time performance is mainly due to its specific network structure and optimization scheme. These designs enable it to achieve a good balance between processing speed and computing resources; the improvements in the detection head of the improved YOLOv8-P2 also have a significant impact on real-time performance. The detection head combines target classification, positioning regression, and confidence prediction tasks, and optimizes the calculation process through efficient convolutional layers. This design reduces processing time while maintaining high-precision detection results, allowing the model to respond quickly when performing target detection. Therefore, these network structures and optimization schemes enable the improved YOLOv8-P2 to run efficiently on low-consumption hardware devices while ensuring detection accuracy and processing speed;

[0064] (3) Generalization ability: The superiority of SAM is mainly reflected in its innovative components and design, which makes it superior to traditional models in segmentation tasks. The SAM model shows significant advantages over traditional CNN models in terms of global context capture, long-range dependency processing, and task generalization.

[0065] (4) Versatility: The target detection and target segmentation method based on the improved YOLOv8-P2 and SAM has wide applicability. This method can easily cope with various types of cameras and diverse scenarios, and can demonstrate its strong versatility and adaptability in fields such as urban monitoring, traffic management, and fire safety. This method can detect and segment targets more accurately and quickly, providing strong technical support for various practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 It is a flow chart of the method for detecting the occupancy of the UAV fire channel based on YOLOv8 and SAM of the present invention;

[0067] Figure 2 It is a schematic diagram of fire channel segmentation in the method for detecting fire channel occupancy of unmanned aerial vehicle based on YOLOv8 and SAM of the present invention;

[0068] Figure 3 It is the judgment result of the partial fire channel occupancy algorithm in the detection method of the UAV fire channel occupancy based on YOLOv8 and SAM of the present invention. DETAILED DESCRIPTION

[0069] The embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and are not intended to limit the protection scope of the present invention.

[0070] Example: Figure 1 As shown, the detection method of drone fire channel occupancy based on YOLOv8 and SAM specifically includes the following steps:

[0071] S1 Prepare the data set: create a data set to be tested as the input test image; create a model training data set at the same time; use Pycharm to crawl different categories of vehicle image data, and use the ffmpeg tool to perform frame sampling on the drone video to obtain the fire channel image data;

[0072] The specific steps in step S1 are:

[0073] S11: The drone takes a picture of the fixed-point area where the fire passage occupancy situation needs to be detected, and obtains an image P to be detected as an input detection image; the drone takes a picture of the fixed-point area where the fire passage occupancy situation needs to be detected, and the obtained image resolution is 4000×3000 pixels; and scales these images to 1000×750 pixels to serve as the input image P to be detected;

[0074] S12: Use ffmpeg tool to perform frame sampling on the collected drone video, extract key frames, and obtain fire channel image data. These key frames usually contain high image quality. The extracted frames will be saved as independent image files for subsequent processing and analysis;

[0075] S13: Collect vehicle image data of different categories in the gallery; analyze the web page structure of Baidu Gallery and the web page loading method of 360 Gallery, and check the robots.txt protocol of both; write a script to collect vehicle image data based on the analysis results; first use the "Octopus Collector" tool to analyze the data of Baidu Gallery and 360 Gallery in order to collect the download address of vehicle image data. In the Octopus tool, a new crawling task was created, the URLs of Baidu and 360 Gallery were entered, and crawling rules were set to extract the download addresses of the images. In order to ensure that all images can be crawled, paging rules were configured, and the HTML structure and JavaScript dynamic loading method of the web page were checked. After the crawling is completed, the Octopus tool aggregates all the image download addresses into a local file in CSV format for subsequent processing. Then a script was written to download these images in batches. In this process, the local file generated previously was read, the download address of each image was parsed from it, and then HTTP requests were sent using Python and the requests library to download these images one by one and save them to the specified local directory;

[0076] S14: Filter the acquired vehicle image data and fire passage image data respectively to obtain a vehicle image dataset and a fire passage image dataset; perform manual screening on the acquired vehicle and fire passage data to remove image data of poor quality or image data that does not meet the definition of the detection task; the acquired vehicle images need to be manually screened to ensure the quality and relevance of the acquired images; the screening process includes removing blurred or improperly exposed images, and ensuring that the images clearly display the vehicle information and do not contain background clutter or non-vehicle images. For the fire passage image data, manual screening is also required to check the clarity and quality of the images, remove blurred or unclear images, and confirm whether the images clearly display the information of the fire passage, and remove those images that do not meet the task definition; through these steps, ensure that the final dataset meets both the quality standards and the predetermined task requirements;

[0077] S2 trains the YOLOv8-P2 model: an improved YOLOv8-P2 model is obtained by using the model training dataset for model training;

[0078] The specific steps of step S2 are:

[0079] S21: Segment the collected vehicle image data and fire passage image data, and use the LabelImg tool to manually annotate the collected vehicle image data and fire passage image data to ensure that the target objects in each image are accurately labeled; after the annotation is completed, divide the data set into a training set and a validation set to facilitate subsequent model training and evaluation; this data set covers various categories in different scenarios, including different types of vehicle images and fire passage images, to ensure that the model can perform effective detection in a variety of environments;

[0080] S22: Build an improved YOLOv8-P2 algorithm model, input the labeled data of the training set and perform model training to obtain the algorithm model weight file best.pt; this weight file best.pt contains the trained and optimized model parameters, which can be used for subsequent target detection tasks to achieve accurate identification of vehicles and fire passages;

[0081] The SE-Block attention mechanism is introduced in step S22, and the loss function CIoU in the algorithm is replaced with the HIoU loss function, so as to further improve the YOLOv8-P2 model and improve the accuracy of small target detection; the HIoU loss function introduces a dynamic adjustment mechanism and integrates multiple IoU metrics to improve the accuracy of small target detection. This innovative method enables the model to better regress small targets during training, thereby improving detection performance. The traditional IoU loss function has the problem of insufficient regression accuracy when dealing with small targets, because its weights and calculation methods are relatively fixed during the training process; HIoU solves this problem by dynamically adjusting the weights, so that the various parts of the loss function can be flexibly adapted in different training stages, thereby improving the detection accuracy of small targets; in addition, HIoU combines multiple IoU metrics, which can more comprehensively evaluate the accuracy of the prediction box, especially in the scene of small targets. This innovative design makes HIoU more advantageous than traditional loss functions when dealing with small target detection tasks, and can provide more accurate regression and better detection results; the HIoU loss function is expressed as a weighted combination of the following forms:

[0082] HIoU=α·IoU+β·GIoU+γ·DIoU;

[0083] Among them, α and β are weights in the total loss; Intersection over Union (IoU) is the most basic metric, which is used to calculate the overlap ratio between the predicted box and the true box; Generalized Intersection over Union (GIoU) ​​is an extension of Intersection over Union (IoU), which introduces the minimum closure area of ​​the predicted box and the true box; Distance Intersection over Union (DIoU) is further improved on the basis of Generalized Intersection over Union (GIoU), which increases the distance between the center points of the predicted box and the true box; YOLOv3 has low accuracy in small target detection, while the improved YOLOv8-P2 greatly improves the recognition ability of small targets through an improved feature fusion mechanism and a deeper network architecture, which is particularly important for detecting small obstacles in fire lanes or distant vehicles;

[0084] S3 Fire lane and vehicle detection: Use the improved YOLOv8-P2 target detection algorithm in the improved YOLOv8-P2 model to detect and analyze the image P to be detected, and identify all vehicles and fire lanes contained in the image;

[0085] The specific steps of step S3 are:

[0086] S31: performing preprocessing, normalization processing and data enhancement processing on the image P to be detected in sequence;

[0087] In the step S31, the image P to be detected is reduced to the same size by preprocessing. The image P to be detected is first adjusted to the input size of 640×640 pixels required by the improved YOLOv8-P2 model, so that the image P to be detected meets the input requirements of the model; then normalization is performed to scale the pixel values ​​to the range of [0,1] so that the model can process it effectively; in addition, in order to improve the generalization ability and robustness of the model, data enhancement processing is performed on the image P to be detected, including random cropping, rotation, flipping, brightness and contrast adjustment;

[0088] S32: Input the image P to be detected, load the algorithm model weight file best.pt and the model configuration file YOLOv8-P2.yaml; predict and identify the processed image P to be detected, that is, perform in-depth analysis on the image P to be detected, identify all vehicles and fire channels contained in the image, and obtain the rectangular frame location area of ​​the vehicles and fire buckets;

[0089] In order to accurately identify and locate tilted or rotated vehicles and fire channel targets in the image, adapt to complex outdoor scenes, improve the accuracy and robustness of detection, and reduce false detection and missed detection, the present invention adopts the improved YOLOv8-P2 rotation target detection as the recognition algorithm based on the improved YOLOv8-P2; in the step S32, the YOLOv8-P2 rotation target detection is used as the recognition algorithm to obtain the rectangular frame position area of ​​the vehicle and the fire bucket, wherein the calculation formula of the rotation rectangular frame coordinates is:

[0090]

[0091] Among them, x i and i Represents the coordinates of the four corner points, where i is 1 to 4, corresponding to the four corner points of the rectangular box; x c and c Represent the horizontal and vertical coordinates of the center point of the rectangular box, w and h are the width and height of the rectangular box, respectively, which are used to calculate the corner point position from the center point; θ is the rotation angle of the rectangular box relative to the horizontal line, expressed in radians. By combining the rotation angle and trigonometric functions, the positions of the four corner points after rotation can be accurately calculated;

[0092] S33: Segment the fire passage area using the deep learning SAM target segmentation algorithm; that is, segment the fire passage area M in the image to be detected P and generate a mask FD;

[0093] The specific steps of step S33 are:

[0094] S331: According to the obtained rectangular frame position area of ​​the vehicle and the fire bucket, the center point coordinate a and any point coordinate b in the rectangular frame area of ​​the fire passage are selected as model prompt input;

[0095] The step S331 is specifically as follows: the coordinate information of the four vertices of the rectangular frame area of ​​the fire passage obtained in the step S32 can determine the coordinate a of the center point of the rectangular frame area, and select any point therein as the coordinate b; the calculation formula of the coordinate a of the center point is:

[0096]

[0097] Among them, (x c ,y c ) represents the selection of the coordinates a of the center point in the rectangular frame area of ​​the fire passage; (x1, y1), (x2, y2), (x3, y3), (x4, y4) are the coordinate information of the four vertices of the rectangular frame area of ​​the fire passage; a random point b is generated in the Cartesian coordinate system in the rectangular area of ​​the fire passage, and its coordinate calculation formula is:

[0098] x rand =x1+(x2-x1)×r x ;

[0099] y rand =y1+(y2-y1)×r y ;

[0100] Among them, (x rand ,y rand ) represents the selection of a random coordinate point b within the rectangular frame area of ​​the fire passage; (x1, y1) and (x2, y2) are the coordinate information of two vertices on the main diagonal line of the rectangular frame area of ​​the fire passage, r x and r y is a random number in the range [0,1];

[0101] S332: Build a SAM segmentation model to segment the image P to be detected; first, the SAM model adopts the Transformer architecture, and captures global context and long-range dependencies through the self-attention mechanism, overcoming the limitation of the traditional convolutional neural network (CNN) that is limited to the local receptive field. Secondly, the SAM model divides the image into small patches (patches) and uses position encoding to maintain spatial information, which enables the model to process image data efficiently. In addition, SAM is pre-trained on large-scale datasets to obtain rich feature representations and excellent generalization capabilities, especially in tasks with less data. Its multi-task learning ability enables SAM to handle a variety of segmentation tasks, with strong flexibility and adaptability, avoiding the need to retrain the model for each task. In terms of modular design, SAM supports backbone networks of different sizes, adjusts according to computing resources and task requirements, and integrates information at different levels, thereby improving the ability to handle complex segmentation tasks. Finally, SAM performs well in boundary detection and detail recognition, and can dynamically adapt to objects of different shapes and sizes to provide more accurate segmentation results;

[0102] S333: According to the coordinates a and b of the center point, the SAM segmentation model is used to segment the fire passage area M in the image to be detected P and generate a mask FD; the SAM model is used to segment the fire passage area in the image to be detected P based on the coordinate information of the point a and the point b;

[0103] The SAM model has three segmentation modes: point prompt, rectangular frame prompt and text prompt. Figure 2 As shown, the improved YOLOv8-P2 is used to predict the rectangular frame area of ​​the fire passage and select the point coordinates therein, so as to provide the SAM model with more accurate point prompts to segment the fire passage area, solving the problem that the SAM model cannot accurately screen out the fire passage area due to segmentation in the "segment everything (full point prompts)" mode; the SAM model has three model backbones: ViT-H, ViT-L and ViT-B. Among them, ViT-B has the least number of parameters, lower computational complexity than the other two models, and faster calculation speed, which meets the real-time application scenarios of fast response and is suitable for devices with limited computing resources such as drones. Therefore, the backbone network model type adopted in the present invention is ViT-B, and GPU is used for reasoning;

[0104] The fire channel mask FD generated by the SAM model has the following characteristics:

[0105] First, the mask is binarized. Each pixel in the image has only two possible values: foreground (fire passage area) and background. The foreground area is marked with one value (white) and the background area is marked with another value (black). This binarization method can clearly separate the fire passage from other areas, which is convenient for subsequent analysis and processing.

[0106] Secondly, the fire escape mask is generated by providing coordinate points to the SAM model, rather than based on a rectangular bounding box. Specifically, the generation of the mask depends on the input coordinate points, and the model uses these coordinate points to accurately segment the fire escape area and mark it as the foreground, and the rest of the area is marked as the background. The mask generated in this way can accurately conform to the shape of the actual fire escape, rather than a simple rectangular area;

[0107] In addition, the size of the generated mask image is consistent with the original input image (1000×750), which ensures accurate alignment of the mask with the original image at the pixel level. This size consistency is crucial for image segmentation and subsequent processing to avoid any possible distortion or alignment errors;

[0108] Finally, before generating the mask, the input data is converted into a PyTorch tensor and moved to the specified computing device (GPU) to speed up the calculation process and ensure the efficiency and accuracy of mask generation. These features together ensure the effectiveness and reliability of the mask in the task of segmenting the fire channel area;

[0109] S34: Perform IOU calculation on the fire lane mask FD and the rectangular frame mask Car of the vehicle in the image to be detected P. If the overlapping area exceeds the threshold, it is determined that the fire lane is occupied, and then go to step S4; if it is determined that it is not occupied, then end this detection;

[0110] The specific steps of step S34 are:

[0111] S341: First, read the rectangular frame coordinate information of all vehicles in the image to be detected P analyzed by the improved YOLOv8-P2, and generate a vehicle mask Car according to the rectangular frame coordinate information;

[0112] The main features of the generated mask Car involve several key aspects. First, the mask image is binary and contains only two pixel values: the foreground area is marked as white, and the background area is marked as black, which makes it possible to clearly distinguish between the target and the background. Secondly, the mask image generated based on the bounding box is rectangular in shape, covering the area within the bounding box. The internal pixels are marked as foreground, and the rest of the area is background. In addition, the size of the mask image is consistent with the original image (the original image size is 1000×750) to ensure accurate alignment and processing at the pixel level.

[0113] S342: then read the fire channel mask FD segmented by the SAM segmentation model;

[0114] S343: Finally, the pixels at the same position in the vehicle mask Car and the fire channel mask FD are bitwise ANDed to obtain the mask result_mask of the overlapping area;

[0115] S344: The number of pixels true_count in the overlapping area is calculated using the mask result_mask, and the total number of pixels total_elements of the fire channel mask FD is calculated in the same way, so as to obtain the IOU, and the IOU is used to determine whether the vehicle illegally occupies the fire channel;

[0116] In step S344, the IOU is calculated according to the number of pixels true_count and the total number of pixels total_elements in the overlapping area, and the formula is:

[0117]

[0118] Among them, A intersection is the area of ​​the intersection, which is equivalent to the number of pixels true_count in the overlapping area; A union is the area of ​​the union, which is equivalent to the total number of pixels total_elements; the IOU value ranges from 0 to 1, and the higher the value, the higher the overlap between the predicted box and the real box; the IOU value is the judgment result, that is, the proportion of the overlapping area in the fire channel mask is calculated; if the judgment result result value is greater than the threshold 0, it is determined that the vehicle illegally occupies the fire channel; Figure 3 As shown;

[0119] S4 Report information: Collect and report the information of illegal vehicles, and end the detection.

[0120] For ordinary technicians in this field, the specific embodiments are only illustrative descriptions of the present invention. It is obvious that the specific implementation of the present invention is not limited to the above-mentioned methods. As long as various non-substantial improvements are made using the method concepts and technical solutions of the present invention, or the concepts and technical solutions of the present invention are directly applied to other occasions without improvement, they are all within the protection scope of the present invention.

Claims

1. A method for detecting the occupancy of a UAV fire channel based on YOLOv8 and SAM, characterized in that: The specific steps include: S1: Prepare the dataset: Prepare the dataset to be tested as the input test image; prepare the model training dataset at the same time; S2 trains the YOLOv8-P2 model: an improved YOLOv8-P2 model is obtained by using the model training dataset for model training; S3 Fire lane and vehicle detection: Use the improved YOLOv8-P2 target detection algorithm in the improved YOLOv8-P2 model to detect and analyze the image P to be detected, and identify all vehicles and fire lanes contained in the image; S4 Report information: Collect and report information on illegal vehicles.

2. The method for detecting the occupancy of a UAV fire channel based on YOLOv8 and SAM according to claim 1, characterized in that: The specific steps in step S1 are: S11: Using a drone to photograph a fixed area where the fire passage occupancy situation needs to be detected, to obtain an image P to be detected as an input detection image; S12: Frame sampling is performed on the collected drone video, key frames are extracted, and fire channel image data is obtained; S13: collecting vehicle image data of different categories in the gallery; S14: Filter the obtained vehicle image data and fire passage image data respectively to obtain a vehicle image data set and a fire passage image data set.

3. The method for detecting the occupation of the UAV fire channel based on YOLOv8 and SAM according to claim 1, characterized in that: The specific steps of step S2 are: S21: segmenting the collected vehicle image data and fire passage image data, annotating the collected vehicle image data and fire passage image data, and after the annotation is completed, dividing the data set into a training set and a validation set; S22: Build an improved YOLOv8-P2 algorithm model, input the labeled data of the training set and perform model training to obtain the algorithm model weight file best.pt.

4. The method for detecting the occupation of a UAV fire channel based on YOLOv8 and SAM according to claim 3, characterized in that: The SE-Block attention mechanism is introduced in step S22, and the loss function CIoU in the algorithm is replaced by the HIoU loss function. The HIoU loss function introduces a dynamic adjustment mechanism and integrates multiple IoU metrics. The HIoU loss function is expressed as a weighted combination of the following forms: HIoU=α·IoU+β·GIoU+γ·DIoU; Among them, α and β are the weights in the total loss; the intersection over union (IoU) is the most basic metric, which is used to calculate the overlap ratio between the predicted box and the true box; the generalized intersection over union (GIoU) ​​is an extension of the intersection over union (IoU), which introduces the minimum closure area of ​​the predicted box and the true box; the distance intersection over union (DIoU) is further improved on the basis of the generalized intersection over union (GIoU) ​​by increasing the distance between the center points of the predicted box and the true box.

5. The method for detecting the occupation of a UAV fire channel based on YOLOv8 and SAM according to claim 4, characterized in that: The specific steps of step S3 are: S31: performing preprocessing, normalization processing and data enhancement processing on the image P to be detected in sequence; S32: Input the image P to be detected, load the algorithm model weight file best.pt to perform prediction analysis on the image P to be detected, and obtain the rectangular frame position area of ​​the vehicle and the fire bucket; S33: Segment the fire passage area using the deep learning SAM target segmentation algorithm; that is, segment the fire passage area M in the image to be detected P and generate a mask FD; S34: Perform IOU calculation on the fire road mask FD and the rectangular frame mask Car of the vehicle in the image to be detected P, and determine the occupation of the fire lane through IOU. If the overlapping area exceeds the threshold, it is determined that the fire lane is occupied, and then go to step S4; if it is determined that it is not occupied, then end this detection.

6. The method for detecting the occupation of a UAV fire channel based on YOLOv8 and SAM according to claim 5, characterized in that: In step S32, YOLOv8-P2 rotating target detection is used as the recognition algorithm to obtain the rectangular frame position area of ​​the vehicle and the fire bucket, wherein the calculation formula of the rotating rectangular frame coordinates is: Among them, x i and i Represents the coordinates of the four corner points, where i is 1 to 4, corresponding to the four corner points of the rectangular box; x c and c They represent the horizontal and vertical coordinates of the center point of the rectangular box, w and h are the width and height of the rectangular box, respectively, which are used to calculate the corner point positions from the center point; θ is the rotation angle of the rectangular box relative to the horizontal line, expressed in radians. By combining the rotation angle and trigonometric functions, the positions of the four corner points after rotation are calculated.

7. The method for detecting the occupation of a UAV fire channel based on YOLOv8 and SAM according to claim 5, characterized in that: The specific steps of step S33 are: S331: According to the obtained rectangular frame position area of ​​the vehicle and the fire bucket, the center point coordinate a and any point coordinate b in the rectangular frame area of ​​the fire passage are selected as model prompt input; S332: Build a SAM segmentation model to segment the image P to be detected; S333: According to the center point coordinates a and b, use the SAM segmentation model to segment the fire passage area M in the image to be detected P and generate a mask FD.

8. The method for detecting the occupation of a UAV fire channel based on YOLOv8 and SAM according to claim 7, characterized in that: The step S331 is specifically as follows: the coordinate information of the four vertices of the rectangular frame area of ​​the fire passage obtained in the step S32 can determine the coordinate a of the center point of the rectangular frame area, and select any point therein as the coordinate b; the calculation formula of the coordinate a of the center point is: Among them, (x c ,y c ) represents the selection of the coordinates a of the center point in the rectangular frame area of ​​the fire passage; (x1, y1), (x2, y2), (x3, y3), (x4, y4) are the coordinate information of the four vertices of the rectangular frame area of ​​the fire passage; a random point b is generated in the Cartesian coordinate system in the rectangular area of ​​the fire passage, and its coordinate calculation formula is: x rand =x1+(x2-x1)×r x ; yes rand =y1+(y2-y1)×r y ; Among them, (x rand ,y rand ) represents the selection of a random coordinate point b within the rectangular frame area of ​​the fire passage; (x1, y1) and (x2, y2) are the coordinate information of two vertices on the main diagonal line of the rectangular frame area of ​​the fire passage, r x and r y is a random number in the range [0,1].

9. The method for detecting the occupancy of a UAV fire channel based on YOLOv8 and SAM according to claim 8, characterized in that: The specific steps of step S34 are: S341: First, read the rectangular frame coordinate information of all vehicles in the image to be detected P analyzed by the improved YOLOv8-P2, and generate a vehicle mask Car according to the rectangular frame coordinate information; S342: then read the fire channel mask FD segmented by the SAM segmentation model; S343: Finally, the pixels at the same position in the vehicle mask Car and the fire channel mask FD are bitwise ANDed to obtain the mask result_mask of the overlapping area; S344: The number of pixels true_count in the overlapping area is calculated using the mask result_mask. Similarly, the total number of pixels total_elements of the fire lane mask FD is calculated to obtain the IOU. The IOU is used to determine whether the vehicle illegally occupies the fire lane.

10. The method for detecting occupancy of a UAV fire channel based on YOLOv8 and SAM according to claim 9, characterized in that: In step S344, the IOU is calculated according to the number of pixels true_count and the total number of pixels total_elements in the overlapping area, and the formula is: Among them, A intersection is the area of ​​the intersection, which is equivalent to the number of pixels true_count in the overlapping area; A union It is the area of ​​the union, which is equivalent to the total number of pixels total_elements; the IOU value ranges from 0 to 1, and the IOU value is the judgment result, that is, the proportion of the overlapping area in the fire channel mask is calculated; if the judgment result result value is greater than the threshold 0, it is determined that the vehicle illegally occupies the fire channel.

Images