Unmanned aerial vehicle visual angle infrastructure identification method based on SAM and YOLOV8 models

Abstract

The invention relates to the field of computer vision and intelligent transportation, in particular to an SAM and YOLOV8 model-based unmanned aerial vehicle visual angle infrastructure identification method, which comprises the following steps of: SI, segmenting a scene area by using an SAM, performing semantic segmentation on a picture shot by an unmanned aerial vehicle by using the SAM, segmenting a road area, and calibrating the road area; s2, target detection is carried out by using YOLOv8, infrastructures on pictures shot by the unmanned aerial vehicle are all detected, and a Neck part of a YOLOv8 model adopts a three-layer PAFPN structure; s3, if the target position area detected by the YOLOv8 and the road area segmented by the SAM have a boundary, indicating that the target is on the road or on the edge of the road; according to the method, semantic segmentation can be quickly performed on the road by using the SAM model, the target can be quickly detected by using the YOLOv8 model, whether the recognized target is the road infrastructure or not can be judged in different scenes through a judgment method, and real-time recognition and real-time segmentation can be realized by combining the SAM model and the YOLOv8 model.

Description

Technical Field

[0001] The present invention relates to the fields of computer vision and intelligent transportation, and in particular to a method for identifying infrastructure from the perspective of an unmanned aerial vehicle (UAV) based on SAM and YOLOV8 models. Background Art

[0002] With the rapid development of modern transportation, the construction and management of highway transportation infrastructure have become crucial. As an important carrier of transportation, the integrity of its infrastructure is directly related to the safety and smoothness of traffic. In this context, accurate and efficient identification and monitoring of highway transportation infrastructure has become an urgent need.

[0003] On the one hand, the traditional inspection method of highway traffic infrastructure mainly relies on manual labor, which has many disadvantages. First, manual inspection is inefficient and requires a lot of time and manpower costs. Faced with the huge highway network, manual inspection is difficult to cover all areas and is prone to omissions. Secondly, the accuracy of manual inspection is affected by the experience and subjective judgment of the inspectors, and there may be misjudgments and omissions. Moreover, in some complex terrain and environmental conditions, manual inspection also faces safety risks.

[0004] On the other hand, with the continuous advancement of science and technology, drone technology has gradually been applied to various fields. In the field of road traffic, drones have unique advantages. Drones can quickly and efficiently obtain image information of roads without being restricted by terrain and environment. By carrying high-resolution cameras, drones can take clear images, providing a rich data source for the identification of road traffic infrastructure.

[0005] However, there are also problems with manual recognition based solely on images taken by drones. The large number of images and the huge workload of manual processing make it difficult to meet real-time requirements. In order to improve the recognition efficiency and accuracy of highway traffic infrastructure, the introduction of advanced image recognition technology has become an inevitable choice.

[0006] The SAM model is a powerful semantic segmentation model that can accurately segment different areas in an image. In the identification of highway traffic infrastructure, the SAM model can be used to quickly segment highway areas from images taken by drones. By training on a large number of pictures with highway boundary markers, the SAM model can learn the characteristics of the highway and accurately identify the highway area, which provides an important foundation for subsequent infrastructure identification.

[0007] The YOLOV8 model is an efficient target detection model that can quickly detect various target objects in images. In the identification of road traffic infrastructure, the YOLOV8 model can detect infrastructure in pictures taken by drones. By training on a large number of pictures marked with infrastructure, the YOLOV8 model can learn the characteristics of different types of infrastructure and accurately detect these targets.

[0008] Combining the SAM model and the YOLOV8 model for highway traffic infrastructure identification from a drone’s perspective has many advantages.

[0009] First of all, this combination can give full play to the advantages of the two models and improve the accuracy of recognition. The SAM model can accurately segment the road area, while the YOLOV8 model can quickly detect infrastructure targets. By combining the two, the SAM model can be used to determine the road area first, and then the YOLOV8 model can be used to detect infrastructure targets in the road area, thereby reducing misidentification and missed identification.

[0010] Secondly, this combination can improve the efficiency of recognition. Both models have a high processing speed and can quickly process images taken by drones. Through real-time recognition and real-time segmentation, information on highway traffic infrastructure can be obtained in a timely manner, providing timely decision-making support for highway management departments.

[0011] In addition, in complex scenarios such as urban roads, data on roads and data on non-roads are often mixed together and difficult to distinguish. Through the method of the present invention, by comparing the target location area detected by YOLOV8 with the road area segmented by SAM, it is possible to accurately determine whether the target is on the road or on the edge of the road, which is of great significance for the management and maintenance of urban roads. Summary of the invention

[0012] In order to overcome the problem that the traditional road traffic infrastructure inspection method mainly relies on manual labor, which is time-consuming and labor-intensive, manual identification based on images taken by drones also has the problem of a large number of images, a huge workload of manual processing, and difficulty in meeting real-time requirements.

[0013] The technical solution of the present invention is: a method for identifying infrastructure from the perspective of a drone based on the SAM and YOLOV8 models, and the steps are as follows:

[0014] SI: Use SAM to segment the scene area, use SAM to perform semantic segmentation on the pictures taken by the drone, segment the road area and mark it as the road area;

[0015] S11: Collect pictures of the scene area, mark the road boundaries in each picture, and use these pictures as training data sets;

[0016] S12: Use the collected dataset to train the SAM model, which consists of an image encoder, a hint encoder, and a mask decoder;

[0017] S121: Before inputting the image into the encoder of the SAM model, the image is first divided and converted into a series of image blocks. For each image block, a corresponding relative position encoding vector is initialized. These position encoding vectors are initialized based on preset rules. Assuming that the image is divided into N image blocks, the dimension of the position encoding vector corresponding to each image block is D, and the position encoding matrix P∈R N×D , after the image block is converted into the initial feature vector through linear projection and other operations, these feature vectors are added to the corresponding position encoding vector element by element, and the image encoding is self-encoded using the self-attention mechanism to obtain the feature vector of the image. This feature vector contains the global and local information of the image. The calculation formula is:

[0018]

[0019] where Q and K are query and key matrices respectively, d k is the dimension of the key vector, Used to scale points to prevent gradient problems;

[0020] S13: Input the image to be processed into the trained SAM model. The SAM model will perform semantic segmentation on the roads in the image. The road area is represented by white, and other areas are represented by black. The coordinates of the pixels in all white areas are saved. That is, the SAM model learns these intrinsic structural features by itself through a large amount of unlabeled image data, learns to extract useful features from the image, and applies them to new unlabeled images for segmentation.

[0021] S2: Use YOLOv8 for target detection to detect all infrastructure in the pictures taken by the drone. The Neck part of the YOLOv8 model uses a three-layer PAFPN structure:

[0022] S21: Collect pictures of the scene area, mark the infrastructure in each picture, record the pixel coordinates of the infrastructure in each picture, and use these pictures as training data sets;

[0023] S22: Train the YOLOv8 model using the collected dataset;

[0024] S23: Input the image to be processed into the trained YOLOv8 model. The SAM model will detect the infrastructure in the image, frame the detected target area, and record the four vertices of each detection frame.

[0025] S3: If the target location area detected by YOLOv8 intersects with the road area segmented by SAM, it means that the target is on the road or on the edge of the road;

[0026] S31: Take out the four vertices of each target detection box recorded in S23 to determine whether they are in the pixel point coordinates recorded in S13;

[0027] S32: According to the judgment in S31, if at least one of the four vertices is in the coordinates of the pixel points in the highway area recorded in S13, it means that the detected target is a highway infrastructure;

[0028] S4: Obtain infrastructure targets on the highway through step S3, and do not process data outside the highway area;

[0029] S5: Perform semantic segmentation on the infrastructure target area, semantically segment each detected entity, and then cancel the YOLOV8 detection border;

[0030] S51: giving the pixel coordinates of each highway infrastructure target determined in S32 to SAM for semantic segmentation;

[0031] S52: SAM performs semantic segmentation and returns the coordinates of the segmented pixels;

[0032] S53: Display the pixel coordinates returned by S52 in the original target detection frame;

[0033] S54: Cancel the original borders of the target detection box, leaving only the semantic segmentation edges of each entity.

[0034] Preferably, the image encoder of SAM in S13 converts the input image data into a feature representation that the model can understand. This feature vector can capture the key features and information of the image, and the model can effectively understand and process the image data. The mask decoder in SAM decodes the feature vector generated by the encoder into the final segmentation mask to achieve accurate segmentation of the image.

[0035] Preferably, in SI, relative position encoding is integrated into the SAM model during the semantic segmentation process.

[0036] Preferably, a humidity sensor is used in S14 to monitor the humidity of the output gas in real time, and the value is displayed on a display screen. When the humidity exceeds the set value, an alarm is automatically sounded, and the air that does not meet the dehydration standard is recirculated into the dryer until the moisture is completely removed.

[0037] Preferably, the mask decoder in S12 decodes the feature vector generated by the encoder into a two-bit mask array of the same size as the original image. The mask array is used to perform pixel-level segmentation on the image, indicating whether each pixel in the image should be retained or deleted, and performing fine-grained control and operation on the image.

[0038] Preferably, in SI, 20%-30% of the training data set is separated as a validation set. These pictures also contain marked road boundaries. The trained SAM model is used to segment the road areas of the pictures in the validation set. The model performance is evaluated by comparing the overlap between the model prediction results and the true labels, including using IoU, i.e. Intersection over Union, to evaluate the model performance. A threshold is set, IoU>0.7, and predictions above this threshold are considered correct. If it is lower than this threshold, the model parameters are adjusted or retrained according to the evaluation indicators until satisfactory performance is achieved.

[0039] Preferably, in S2, a part of the training data set is separated as a validation set. These pictures contain labeled infrastructure. The trained YOLOv8 model is used to detect infrastructure on the pictures in the validation set. The model performance is evaluated using indicators such as precision, recall and F1 score. These indicators are calculated by comparing the model prediction results with the actual labels. Finally, the model parameters are adjusted or retrained according to the evaluation indicators until satisfactory performance is achieved.

[0040] Preferably, during the recognition process, the pictures taken by the drone are preprocessed. In the preprocessing, the image files stored on the drone or ground station are first read and loaded into the image processing software. The noise points are removed by a denoising algorithm, including a mean filter, a median filter or a Gaussian filter, to improve the clarity of the image. Then, the contrast is enhanced by adjusting the histogram of the image and using a contrast stretching algorithm or an adaptive contrast enhancement algorithm to make the target in the image more prominent, which is convenient for subsequent feature extraction and recognition. Machine learning or deep learning models are also applied for processing. These models automatically extract features from images by learning a large amount of image data, and realize preprocessing and optimization of the images. At the same time, the quality of the pictures taken by the drone is evaluated to ensure that the image quality meets the recognition requirements.

[0041] Preferably, the method also includes a visualization interface module to display the identified location and information of road traffic infrastructure to facilitate user viewing and operation. At the same time, the visualization interface module is provided with a user feedback function to allow the user to mark erroneous or omitted infrastructure and interact with other traffic management systems to achieve information sharing and collaborative management. At the same time, a parameter adjustment function is set to optimize and adjust the model parameters according to different scenarios and needs.

[0042] As a preference, multi-target tracking algorithms, including SORT and DeepSORT, are introduced based on the YOLOV8 model, and a behavior analysis algorithm is developed to identify and analyze the trajectory and behavior patterns of moving targets, including speeding and illegal parking, while providing a visual display of the behavior analysis results and an alarm function.

[0043] Beneficial effects of the present invention:

[0044] (1) The SAM model can be used to quickly perform semantic segmentation on roads;

[0045] (2) The target can be detected quickly by using the YOLOv8 model;

[0046] (3) The judgment method can be used to determine whether the identified target is a highway infrastructure in different scenarios;

[0047] (4) Combining the SAM model with the YOLOv8 model can achieve real-time recognition and real-time segmentation;

[0048] (5) It is conducive to the management of highway infrastructure and can greatly improve efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 What is shown is a flow chart of the method for identifying infrastructure from the perspective of a drone based on the SAM and YOLOV8 models of the present invention;

[0050] Figure 2 What is shown is the improved YOLOv8 model architecture diagram of the drone perspective infrastructure recognition method based on SAM and YOLOV8 models of the present invention. DETAILED DESCRIPTION

[0051] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0052] See also Figure 1 ,The present invention provides an embodiment: a method for identifying infrastructure from a drone perspective based on SAM and YOLOV8 model, the steps of which are as follows: using SAM to segment scene areas, using SAM to perform semantic segmentation on the pictures taken by the drone, segmenting the highway area, and marking it as a highway area;

[0053] S11: Collect pictures of the scene area, mark the road boundaries in each picture, and use these pictures as training data sets;

[0054] S12: Use the collected dataset to train the SAM model, which consists of an image encoder, a hint encoder, and a mask decoder;

[0055] S121: Before inputting the image into the encoder of the SAM model, the image is first divided and converted into a series of image blocks. For each image block, a corresponding relative position encoding vector is initialized. These position encoding vectors are initialized based on preset rules. Assuming that the image is divided into N image blocks, the dimension of the position encoding vector corresponding to each image block is D, and the position encoding matrix P∈R N×D , after the image block is converted into the initial feature vector through linear projection and other operations, these feature vectors are added to the corresponding position encoding vector element by element, and the image encoding is self-encoded using the self-attention mechanism to obtain the feature vector of the image. This feature vector contains the global and local information of the image. The calculation formula is:

[0056]

[0057] where Q and K are query and key matrices respectively, d k is the dimension of the key vector, Used to scale points to prevent gradient problems;

[0058] S13: Input the image to be processed into the trained SAM model. The SAM model will perform semantic segmentation on the roads in the image. The road area is represented by white, and other areas are represented by black. The coordinates of the pixels in all white areas are saved. That is, the SAM model learns these intrinsic structural features by itself through a large amount of unlabeled image data, learns to extract useful features from the image, and applies them to new unlabeled images for segmentation.

[0059] S2: Use YOLOv8 for target detection to detect all infrastructure in the pictures taken by the drone. The Neck part of the YOLOv8 model uses a three-layer PAFPN structure:

[0060] S21: Collect pictures of the scene area, mark the infrastructure in each picture, record the pixel coordinates of the infrastructure in each picture, and use these pictures as training data sets;

[0061] S22: Train the YOLOv8 model using the collected dataset;

[0062] S23: Input the image to be processed into the trained YOLOv8 model. The SAM model will detect the infrastructure in the image, frame the detected target area, and record the four vertices of each detection frame.

[0063] S3: If the target location area detected by YOLOv8 intersects with the road area segmented by SAM, it means that the target is on the road or on the edge of the road;

[0064] S31: Take out the four vertices of each target detection box recorded in S23 to determine whether they are in the pixel point coordinates recorded in S13;

[0065] S32: According to the judgment in S31, if at least one of the four vertices is in the coordinates of the pixel points in the highway area recorded in S13, it means that the detected target is a highway infrastructure;

[0066] S4: Obtain infrastructure targets on the highway through step S3, and do not process data outside the highway area;

[0067] S5: Perform semantic segmentation on the infrastructure target area, semantically segment each detected entity, and then cancel the YOLOV8 detection border;

[0068] S51: giving the pixel coordinates of each highway infrastructure target determined in S32 to SAM for semantic segmentation;

[0069] S52: SAM performs semantic segmentation and returns the coordinates of the segmented pixels;

[0070] S53: Display the pixel coordinates returned by S52 in the original target detection frame;

[0071] S54: Cancel the original borders of the target detection box, leaving only the semantic segmentation edges of each entity.

[0072] Example 1

[0073] The identification method using this technical solution has the following steps:

[0074] SI uses SAM to segment scene areas, and uses SAM to perform semantic segmentation on the pictures taken by drones, segmenting the road area and marking it as the road area. In the process of semantic segmentation, the relative position encoding is integrated into the SAM model;

[0075] S11: Collect pictures of the scene area, mark the highway boundaries in each picture, and use these pictures as training data sets. In the recognition process, preprocess the pictures taken by the drone. In the preprocessing, first read the image file stored on the drone or ground station, load it into the image processing software, and use the denoising algorithm, including mean filtering, median filtering or Gaussian filtering, to remove these noise points and improve the clarity of the image. Then, adjust the histogram of the image, use the contrast stretching algorithm or the adaptive contrast enhancement algorithm to enhance the contrast, so that the target in the image is more prominent, which is convenient for subsequent feature extraction and recognition. Machine learning or deep learning models are also applied for processing. These models automatically extract features in the image by learning a large amount of image data, and realize the preprocessing and optimization of the image. At the same time, the quality of the pictures taken by the drone is evaluated to ensure that the picture quality meets the recognition requirements;

[0076] S12: Use the collected data set to train the SAM model. The SAM model consists of an image encoder, a hint encoder, and a mask decoder. The mask decoder decodes the feature vector generated by the encoder into a two-bit mask array of the same size as the original image. The mask array is used to perform pixel-level segmentation on the image, indicating whether each pixel in the image should be retained or deleted, and performing fine-grained control and manipulation of the image.

[0077] S121: Before inputting the image into the encoder of the SAM model, the image is first divided and converted into a series of image blocks. For each image block, a corresponding relative position encoding vector is initialized. These position encoding vectors are initialized based on preset rules. Assuming that the image is divided into N image blocks, the dimension of the position encoding vector corresponding to each image block is D, and the position encoding matrix P∈R N×D , after the image block is converted into the initial feature vector through linear projection and other operations, these feature vectors are added to the corresponding position encoding vector element by element, and the image encoding is self-encoded using the self-attention mechanism to obtain the feature vector of the image. This feature vector contains the global and local information of the image. The calculation formula is:

[0078]

[0079] where Q and K are query and key matrices respectively, d k is the dimension of the key vector, Used to scale points to prevent gradient problems;

[0080] S13: Input the image to be processed into the trained SAM model. The SAM model will perform semantic segmentation on the roads in the image. The road area is represented by white, and other areas are represented by black. The coordinates of the pixels in all white areas are saved. That is, the SAM model learns these intrinsic structural features by itself through a large amount of unlabeled image data, learns to extract useful features from the image, and applies them to new unlabeled images for segmentation. The image encoder of SAM converts the input image data into a feature representation form that the model can understand. This feature vector can capture the key features and information of the image, and the model can effectively understand and process the image data. The mask decoder in SAM decodes the feature vector generated by the encoder into the final segmentation mask to achieve accurate segmentation of the image.

[0081] S14: Separate 20%-30% of the training data set as the validation set. These images also contain marked road boundaries. Use the trained SAM model to segment the road area of ​​the images in the validation set. Evaluate the model performance by comparing the overlap between the model prediction results and the true labels, including using IoU, that is, Intersect ion over Union, to evaluate the model performance. Set a threshold, IoU>0.7, and predictions above this threshold are considered correct. If it is lower than this threshold, adjust the model parameters or retrain according to the evaluation indicators until satisfactory performance is achieved;

[0082] S2: Use YOLOv8 for target detection to detect all infrastructure in the pictures taken by drones. The Neck part of the YOLOv8 model adopts a three-layer PAFPN structure. Based on the YOLOv8 model, a multi-target tracking algorithm is introduced, including SORT and DeepSORT. A behavior analysis algorithm is developed to identify and analyze the trajectory and behavior patterns of mobile targets, including speeding and illegal parking. At the same time, a visualization display and alarm function of the behavior analysis results are provided:

[0083] S21: Collect pictures of the scene area, mark the infrastructure in each picture, record the pixel coordinates of the infrastructure in each picture, and use these pictures as training data sets;

[0084] S22: Train the YOLOv8 model using the collected dataset;

[0085] S23: Input the image to be processed into the trained YOLOv8 model. The SAM model will detect the infrastructure in the image, frame the detected target area, and record the four vertices of each detection frame.

[0086] S24: Separate a part of the training data set as the validation set. These pictures contain labeled infrastructure. Use the trained YOLOv8 model to detect infrastructure on the pictures in the validation set. Use the precision, recall, and F1 score indicators to evaluate the model performance. These indicators are calculated by comparing the model prediction results with the actual labels. Finally, adjust the model parameters or retrain according to the evaluation indicators until satisfactory performance is achieved.

[0087] S3: If the target location area detected by YOLOv8 intersects with the road area segmented by SAM, it means that the target is on the road or on the edge of the road;

[0088] S31: Take out the four vertices of each target detection box recorded in S23 to determine whether they are in the pixel point coordinates recorded in S13;

[0089] S32: According to the judgment in S31, if at least one of the four vertices is in the coordinates of the pixel points in the highway area recorded in S13, it means that the detected target is a highway infrastructure;

[0090] S4: Obtain infrastructure targets on the highway through step S3, and do not process data outside the highway area;

[0091] S5: Perform semantic segmentation on the infrastructure target area, semantically segment each detected entity, and then cancel the YOLOV8 detection border;

[0092] S51: giving the pixel coordinates of each highway infrastructure target determined in S32 to SAM for semantic segmentation;

[0093] S52: SAM performs semantic segmentation and returns the coordinates of the segmented pixels;

[0094] S53: Display the pixel coordinates returned by S52 in the original target detection frame;

[0095] S54: Cancel the original borders of the target detection box, leaving only the semantic segmentation edges of each entity;

[0096] S55: Visual interface module

[0097] The location and information of identified highway traffic infrastructure are displayed to facilitate user viewing and operation. At the same time, the visualization interface module is equipped with a user feedback function, allowing users to mark erroneous or missed infrastructure and interact with other traffic management systems to achieve information sharing and collaborative management. At the same time, a parameter adjustment function is set to optimize and adjust model parameters according to different scenarios and needs.

[0098] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge scope of those skilled in the art without departing from the spirit of the present invention.

Claims

1. The infrastructure recognition method from the drone perspective based on SAM and YOLOV8 model is characterized by: The steps are as follows: SI: Use SAM to segment the scene area, use SAM to perform semantic segmentation on the pictures taken by the drone, segment the road area and mark it as the road area; S11: Collect pictures of the scene area, mark the road boundaries in each picture, and use these pictures as training data sets; S12: Use the collected dataset to train the SAM model, which consists of an image encoder, a hint encoder, and a mask decoder; S121: Before inputting the image into the encoder of the SAM model, the image is first divided and converted into a series of image blocks. For each image block, a corresponding relative position encoding vector is initialized. These position encoding vectors are initialized based on preset rules. Assuming that the image is divided into N image blocks, the dimension of the position encoding vector corresponding to each image block is D, and the position encoding matrix P∈R N×D , after the image block is converted into the initial feature vector through linear projection and other operations, these feature vectors are added to the corresponding position encoding vector element by element, and the image encoding is self-encoded using the self-attention mechanism to obtain the feature vector of the image. This feature vector contains the global and local information of the image. The calculation formula is: where Q and K are query and key matrices respectively, d k is the dimension of the key vector, Used to scale points to prevent gradient problems; S13: Input the image to be processed into the trained SAM model. The SAM model will perform semantic segmentation on the roads in the image. The road area is represented by white, and other areas are represented by black. The coordinates of the pixels in all white areas are saved. That is, the SAM model learns these intrinsic structural features by itself through a large amount of unlabeled image data, learns to extract useful features from the image, and applies them to new unlabeled images for segmentation. S2: Use YOLOv8 for target detection to detect all infrastructure in the pictures taken by the drone. The Neck part of the YOLOv8 model uses a three-layer PAFPN structure: S21: Collect pictures of the scene area, mark the infrastructure in each picture, record the pixel coordinates of the infrastructure in each picture, and use these pictures as training data sets; S22: Train the YOLOv8 model using the collected dataset; S23: Input the image to be processed into the trained YOLOv8 model. The SAM model will detect the infrastructure in the image, frame the detected target area, and record the four vertices of each detection frame. S3: If the target location area detected by YOLOv8 intersects with the road area segmented by SAM, it means that the target is on the road or on the edge of the road; S31: Take out the four vertices of each target detection box recorded in S23 to determine whether they are in the pixel point coordinates recorded in S13; S32: According to the judgment in S31, if at least one of the four vertices is in the coordinates of the pixel points in the highway area recorded in S13, it means that the detected target is a highway infrastructure; S4: Obtain infrastructure targets on the highway through step S3, and do not process data outside the highway area; S5: Perform semantic segmentation on the infrastructure target area, semantically segment each detected entity, and then cancel the YOLOV8 detection border; S51: giving the pixel coordinates of each highway infrastructure target determined in S32 to SAM for semantic segmentation; S52: SAM performs semantic segmentation and returns the coordinates of the segmented pixels; S53: Display the pixel coordinates returned by S52 in the original target detection frame; S54: Cancel the original borders of the target detection box, leaving only the semantic segmentation edges of each entity.

2. The method for identifying infrastructure from the perspective of a drone based on the SAM and YOLOV8 models according to claim 1, characterized in that: The image encoder of SAM in S13 converts the input image data into a feature representation that the model can understand. This feature vector can capture the key features and information of the image, and the model can effectively understand and process the image data; the mask decoder in SAM decodes the feature vector generated by the encoder into the final segmentation mask to achieve accurate segmentation of the image.

3. The method for identifying infrastructure from the perspective of a drone based on the SAM and YOLOV8 models according to claim 2, characterized in that: In SI, relative position encoding is integrated into the SAM model during the semantic segmentation process.

4. The method for identifying infrastructure from the perspective of a drone based on the SAM and YOLOV8 models according to claim 3 is characterized in that: The mask decoder in S12 decodes the feature vector generated by the encoder into a two-bit mask array of the same size as the original image. The mask array is used to perform pixel-level segmentation of the image, indicating whether each pixel in the image should be retained or deleted, and performing fine-grained control and manipulation of the image.

5. The method for identifying infrastructure from the perspective of a drone based on the SAM and YOLOV8 models according to claim 4 is characterized in that: In SI, 20%-30% of the training data set is separated as a validation set. These pictures also contain marked road boundaries. The trained SAM model is used to segment the road areas of the pictures in the validation set. The model performance is evaluated by comparing the overlap between the model prediction results and the true labels, including using IoU, that is, Intersection over Union, to evaluate the model performance. A threshold is set, IoU>0.7, and predictions above this threshold are considered correct. If it is lower than this threshold, the model parameters are adjusted or retrained according to the evaluation indicators until satisfactory performance is achieved.

6. The method for identifying infrastructure from the perspective of a drone based on the SAM and YOLOV8 models according to claim 5, characterized in that: In S2, a part of the training data set is separated as a validation set. These pictures contain labeled infrastructure. The trained YOLOv8 model is used to detect infrastructure on the pictures in the validation set. The precision, recall and F1 score indicators are used to evaluate the model performance. These indicators are calculated by comparing the model prediction results with the actual labels. Finally, the model parameters are adjusted or retrained according to the evaluation indicators until satisfactory performance is achieved.

7. The method for identifying infrastructure from the perspective of a drone based on the SAM and YOLOV8 models according to claim 6, characterized in that: During the recognition process, the pictures taken by the drone are preprocessed. In the preprocessing, the image files stored on the drone or ground station are first read and loaded into the image processing software. The noise points are removed through denoising algorithms, including mean filtering, median filtering or Gaussian filtering, to improve the clarity of the image. Then, the image histogram is adjusted and the contrast is enhanced using a contrast stretching algorithm or an adaptive contrast enhancement algorithm to make the target in the image more prominent, which is convenient for subsequent feature extraction and recognition. Machine learning or deep learning models are also applied for processing. These models automatically extract features from images by learning a large amount of image data, and realize image preprocessing and optimization. At the same time, the quality of the pictures taken by the drone is evaluated to ensure that the image quality meets the recognition requirements.

8. The method for identifying infrastructure from the perspective of a drone based on the SAM and YOLOV8 models according to claim 7, characterized in that: The method also includes a visualization interface module that displays the location and information of the identified highway traffic infrastructure to facilitate user viewing and operation. At the same time, the visualization interface module is provided with a user feedback function that allows users to mark erroneous or missed infrastructure and interact with other traffic management systems to achieve information sharing and collaborative management. At the same time, a parameter adjustment function is set to optimize and adjust the model parameters according to different scenarios and needs.

9. The method for identifying infrastructure from the perspective of a drone based on the SAM and YOLOV8 models according to claim 8, characterized in that: Based on the YOLOV8 model, multi-target tracking algorithms, including SORT and DeepSORT, are introduced, and behavior analysis algorithms are developed to identify and analyze the trajectories and behavior patterns of moving targets, including speeding and illegal parking, while providing visual display and alarm functions of behavior analysis results.

Images