A yolo-based port facility remote sensing image detection and counting discrimination method

Abstract

The application discloses a kind of YOLO-based port facility remote sensing image detection and count discrimination method, specifically related to port target detection technical field, by YOLOv5 algorithm to the important facilities of port in remote sensing image oil tank and crane are carried out target detection and target counting to realize the evaluation of the number of port installation, target port image is identified and an edge region is set, threshold discrimination is carried out, then counting, according to the confidence of the frame on the edge region after target detection discriminates the integrity of target object, port facility discrimination counting is classified according to the target detection result Statistics on the number of target objects, and the integrity of the target object in the edge region is discriminated, when target object is complete facility, add 1 on the total value of counting;When target object is not complete facility, add 0.5 on the total value of counting, other regions are normally counted, can avoid a port facility multiple counting, realize the grasp of the situation of the port and increase the richness of port related evaluation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of port target detection, more particularly, the present application relates to a port facility remote sensing image detection and counting discrimination method based on YOLO. BACKGROUND

[0002] Target detection has a wide application prospect in the fields of industrial detection, unmanned driving, medical image processing, etc. Especially in the field of high-resolution remote sensing target detection, remote sensing target detection shows high research value and significance in civil fields. Port is a comprehensive transportation hub and a strategic resource and important support for economic and social development. Therefore, it is of great practical significance and urgent practical demand to accurately and efficiently extract port targets from remote sensing images for shipping services. This is also one of the research hotspots in the field of pattern recognition and image processing.

[0003] The task of target detection is to find all the targets (objects) of interest in the image, determine their categories and positions. Adding the counting function on the basis of target detection can get the number of detected targets in the image, which is of great help to data statistics in agriculture, medicine, construction, etc. In the field of target recognition, the classic algorithms are R-CNN series and YOLO series. YOLO series algorithm belongs to one-stage algorithm, while R-CNN algorithm belongs to two-stage algorithm. Due to the fast detection speed of YOLO algorithm, it can be applied to real-time detection and video detection, and has been concerned in recent years. Therefore, YOLO algorithm and counting function are combined to be widely used. In order to improve the accuracy of counting, the network structure, loss function, data enhancement, etc. of YOLO algorithm are improved to improve the detection accuracy and thus achieve the purpose of improving the counting accuracy. The optimization of this algorithm can effectively improve the accuracy of counting, but the degree of improvement of accuracy is limited for special scene requirements, and new optimization scheme needs to be added according to special scene.

[0004] In the process of realizing the number evaluation of port facilities by using YOLOv5 algorithm to identify and count the targets of important facilities such as oil tank and crane in remote sensing image, it is found that the defect in the prior art is that only the network structure, loss function, data enhancement, etc. of YOLO algorithm are improved to improve the detection accuracy and thus achieve the purpose of improving the counting accuracy. The influence of special situation on technical accuracy is not considered. In order to ensure that all facilities in the port can be counted, the remote sensing images of the port are cut into images of the same size and non-intersection when the data set is made. This will inevitably cause individual facilities to be cut off. When target detection is carried out, the facilities cut off at the edge of the image will also be recognized, which will cause a facility to be counted repeatedly, resulting in inaccurate counting.

[0005] To solve the above problems, a technical solution is provided. SUMMARY

[0006] In order to overcome the above-mentioned defects of the prior art, the present application provides a YOLO-based port facility remote sensing image detection and counting discrimination method, which realizes the evaluation of the number of port facilities by target detection and target counting of important port facilities such as oil tanks and cranes in remote sensing images through YOLOv5 algorithm, performs target recognition on the target port image and sets an edge region, performs threshold discrimination, then counts, discriminates the integrity of the target object according to the confidence on the frame of the edge region after target detection, and classifies and counts the number of target objects according to the target detection result, and discriminates the integrity of the target object in the edge region, adds 1 to the total count value when the target object is a complete facility, adds 0.5 to the total count value when the target object is not a complete facility, and normally counts other regions, which can avoid multiple counting of a port facility, realize the grasp of the port material situation and increase the richness of the port related evaluation, so as to solve the problems raised in the above background art.

[0007] To achieve the above object, the present application provides the following technical scheme:

[0008] A YOLO-based port facility remote sensing image detection and counting discrimination method, which performs screenshot on the target port remote sensing image to obtain a target port image set, then performs target detection and target counting on important port facilities such as oil tanks and cranes in the target port image set through YOLOv5 algorithm, realizes the evaluation of the number of port facilities, performs target recognition on the target port image and sets an edge region, performs threshold discrimination, then counts, discriminates the integrity of the target object according to the confidence on the frame of the edge region after target detection, and classifies and counts the number of target objects according to the target detection result, and discriminates the integrity of the target object in the edge region, adds 1 to the total count value when the target object is a complete facility, adds 0.5 to the total count value when the target object is not a complete facility, and normally counts other regions.

[0009] As a further scheme of the present application, the following specific steps are included:

[0010] Step one, image set making: select a target port in the ship news network for screenshot to obtain a target port image, the pixel of the target port image is 960x960, ensure that the content of the target port image is not overlapped, and generate a target port image set;

[0011] Step two, YOLOv5 model training: pre-process and label the target port image set, assign classes and bounding boxes to the target objects in the target port image set, and optimize the model using data augmentation strategy;

[0012] Step three, target detection of port oil tank and crane: perform target detection on the target port image set by the trained YOLOv5 model, and output the class, bounding box coordinates and confidence of the detected target object;

[0013] Step four, threshold discrimination: set an edge region for the target port image, calculate the accuracy and recall rate under different confidence thresholds, draw the precision-recall curve, select the optimal confidence according to the precision-recall curve, and discriminate whether the target object is a complete facility or a segmented facility according to the optimal confidence on the edge region frame after target detection;

[0014] Step five, port facility discrimination counting: according to the target detection result, classify and count the number of target objects, and discriminate the integrity of the target objects;

[0015] Step six, accuracy evaluation: establish an evaluation model through target image quality, model parameters and false detection rate, evaluate the accuracy of port facility discrimination counting and output the total number of port facilities.

[0016] As a further scheme of the present application, the target objects detected in step three target detection of port oil tank and crane are oil tanks and cranes in the port.

[0017] As a further scheme of the present application, in step two YOLOv5 model training, the target port image is pre-processed and labeled, the classes and bounding boxes of the target objects in the target port image are assigned, and the model is optimized using data augmentation strategy, the specific steps are as follows:

[0018] Step Q1, image pre-processing: increase the clarity of the target port images in the target port image set by super-resolution, and then label the target port images, assign classes and bounding boxes to each target object;

[0019] Step Q2, model selection: select YOLOv5 model as the port target detection model;

[0020] Step Q3, model training: train the YOLOv5 model through the labeled target port image set, and use data augmentation strategy to improve the performance of the model during training;

[0021] Step Q4, model verification: evaluate the performance of the model through the target port image set, the evaluation indicators include accuracy, recall rate and F1 score indicators, and optimize the model according to the verification result.

[0022] As a further scheme of the application, step Q3 model training trains the YOLOv5 model through the labeled target port image set, and adopts a data enhancement strategy to improve the model performance during the training process, wherein the data enhancement strategy is determined by a density matching algorithm, and the determination formula of the optimal data enhancement strategy is:

[0023] ;

[0024] In the formula: is the optimal data enhancement strategy, is the expected accuracy, is an image in the target port image set, is the data enhancement strategy, is the search parameter.

[0025] As a further scheme of the application, step three target detection of the port oil tank and the crane is performed by the trained YOLOv5 model on the target port image set, and the detected target object outputs the class, the bounding box coordinates and the confidence, and the specific steps are as follows:

[0026] Step Z1, image reprocessing: scaling, cropping and rotating the target port image set to adapt to the input requirements of the YOLOv5 model;

[0027] Step Z2, port target detection: target detection of the port facility remote sensing picture is performed by the trained YOLOv5 model, and the detected target object outputs the class, the bounding box coordinates and the confidence;

[0028] Step Z3, image post-processing: the bounding box coordinates output by the non-maximum suppression are used to eliminate overlapping bounding boxes.

[0029] As a further scheme of the application, step four threshold discrimination sets an edge region for the target port image, calculates the accuracy and recall rate under different confidence thresholds, and draws the precision-recall curve, selects the optimal confidence according to the precision-recall curve, and discriminates whether the target object is a complete facility or a segmented facility according to the optimal confidence on the frame of the edge region after target detection.

[0030] As a further scheme of the present application, the step five port facility discrimination counting classifies and counts the number of target objects according to the target detection result, and discriminates the integrity of the target objects in the edge area, when the target object is a complete facility, adds 1 to the total value of the counting; when the target object is not a complete facility, adds 0.5 to the total value of the counting, and the other area is normally counted.

[0031] As a further scheme of the present application, the step six accuracy evaluation establishes an accuracy evaluation model for the port facility discrimination counting by the target image quality, the model parameter and the false detection rate, wherein the formula of the accuracy evaluation model is:

[0032] ;

[0033] In the formula: is the accuracy of the port facility discrimination counting, is the target image quality, is the model parameter, is the false detection rate.

[0034] The technical effect and advantages of the present application, a port facility remote sensing image detection and counting discrimination method based on YOLO, are as follows:

[0035] 1. The present application can realize the grasp of the port material situation and increase the richness of the port related evaluation by the target detection and target counting of the important facilities of the port, oil tank and crane.

[0036] 2. The present application implements accurate counting for the incomplete facilities in the image edge, avoids multiple counting of a port facility, and improves the accuracy of target detection and counting. DETAILED DESCRIPTION

[0037] Figure 1 The present application is a flowchart of a port facility remote sensing image detection and counting discrimination method based on YOLO. CONCLUSION

[0038] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all the other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.

[0039] A YOLO-based port facility remote sensing image detection and counting discrimination method, the target port remote sensing image is captured to obtain a target port image set, and then the YOLOv5 algorithm is used to detect and count the target port important facilities oil tank and crane in the target port image set, so as to realize the evaluation of the number of port facilities, the target port image is subjected to target recognition and an edge region is set, threshold discrimination is carried out, and then counting is carried out, the integrity of the target object is discriminated according to the confidence of the frame on the edge region after target detection, the port facility discrimination counting is classified and counted according to the target detection result, and the integrity of the target object in the edge region is discriminated, when the target object is a complete facility, 1 is added to the total value of the count; when the target object is not a complete facility, 0.5 is added to the total value of the count, and other regions are normally counted.

[0040] The embodiment of the present application comprises the following specific steps:

[0041] Step one, image set making: select a target port in the ship news network for capturing to obtain a target port image, the target port image has a pixel of 960x960, the content of the target port image is ensured not to overlap, and a target port image set is generated;

[0042] Step two, YOLOv5 model training: pre-process the target port image set and label, assign a class and a boundary box to the target object in the target port image set, and optimize the model by using a data enhancement strategy;

[0043] Step three, target detection of port oil tank and crane: the trained YOLOv5 model is used to detect the target port image set, and the class, boundary box coordinates and confidence of the detected target object are outputted;

[0044] Step four, threshold discrimination: set an edge region for the target port image, calculate the accuracy and recall rate under different confidence thresholds, draw the precision-recall curve, select the optimal confidence according to the precision-recall curve, and discriminate whether the target object is a complete facility or a segmented facility according to the optimal confidence on the frame of the edge region after target detection;

[0045] Step five, port facility discrimination counting: according to the target detection result, the number of target objects is classified and counted, and the integrity of the target object is discriminated;

[0046] Step six, accuracy evaluation: an evaluation model is established by target image quality, model parameters and false detection rate, the accuracy of the port facility discrimination counting is evaluated, and the total number of port facilities is outputted. Embodiment 1

[0047] In the ship network, select a target port for screenshot, require each cut image to be 960x960 pixels, and ensure that the content in each image does not overlap. Since the clarity of remote sensing images in some areas is not enough, the port facilities cannot be clearly seen. Therefore, the dataset is first super-resolved to increase the clarity before being labeled. After the dataset is prepared, the YOLOv5 algorithm is used for target detection, and then the detection model is obtained through the training of various labels. The target image set is tested, and then the counting function is added to count the detected targets. The target port image is recognized and the edge region is set. Whether the detected port facilities in the edge region of the target port image are complete is determined by threshold value. When the confidence of the port facilities in the edge region is higher than the threshold value, the port facility count is 1; when the confidence of the port facilities in the edge region is lower than the threshold value, the port facility count is 0.5. Other regions are normally counted, and finally the number of detected port facilities in all target port images is output.

[0048] In the step three of the embodiment of the application, the target objects detected in the target detection of the port oil tank and the crane are oil tanks and cranes in the port.

[0049] In the step two of the embodiment of the application, the YOLOv5 model is trained. The target port images are preprocessed and labeled. The target objects in the target port images are assigned classes and bounding boxes. The data enhancement strategy is used to optimize the model. The specific steps are as follows:

[0050] Step Q1, image preprocessing: the target port images in the target port image set that are not clear are super-resolved to increase the clarity, and then the target port images are labeled. Each target object is assigned a class and a bounding box.

[0051] Step Q2, model selection: the YOLOv5 model is selected as the port target detection model.

[0052] Step Q3, model training: the YOLOv5 model is trained through the labeled target port image set. In the training process, the data enhancement strategy is used to improve the performance of the model.

[0053] Step Q4, model verification: the performance of the model is evaluated through the target port image set. The evaluation indicators include accuracy, recall rate and F1 score indicators. The model is optimized according to the verification result.

[0054] In the step Q3 of the embodiment of the application, the YOLOv5 model is trained through the labeled target port image set. In the training process, the data enhancement strategy is used to improve the performance of the model. The data enhancement strategy is determined by the density matching algorithm. The determination formula of the best data enhancement strategy is as follows:

[0055] ;

[0056] wherein: is the best data augmentation strategy, is the expected accuracy, is an image in the target port image set, is the data augmentation strategy, is the finding parameter.

[0057] The target detection of the port oil tank and the crane in step three in the embodiment of the application is performed through the target detection of the target port image set by the YOLOv5 model trained, and the category, the bounding box coordinate and the confidence of the detected target object are output, and the specific steps are as follows:

[0058] Step Z1, image reprocessing: scaling, cropping and rotating operations are performed on the target port image set to adapt to the input requirements of the YOLOv5 model;

[0059] Step Z2, port target detection: the target detection of the port facility remote sensing picture is performed through the YOLOv5 model trained, and the category, the bounding box coordinate and the confidence of the detected target object are output;

[0060] Step Z3, image post-processing: the bounding box coordinate output by the non-maximum suppression is used to eliminate the overlapping bounding box.

[0061] In step four in the embodiment of the application, a marginal area is set for the target port image, the accuracy and the recall rate under different confidence thresholds are calculated, and the precision-recall curve is drawn, the optimal confidence is selected according to the precision-recall curve, and whether the target object is a complete facility or a segmented facility is discriminated according to the optimal confidence on the frame of the marginal area after the target detection.

[0062] In step five in the embodiment of the application, the port facility discrimination counting is classified and counted according to the target detection result, and the integrity of the target object in the marginal area is discriminated, when the target object is a complete facility, 1 is added to the total value of the counting; when the target object is not a complete facility, 0.5 is added to the total value of the counting, and other areas are normally counted.

[0063] In step six in the embodiment of the application, the accuracy evaluation model is established through the target image quality, the model parameter and the false detection rate, and the accuracy of the port facility discrimination counting is evaluated, wherein the formula of the accuracy evaluation model is:

[0064] ;

[0065] wherein: is the accuracy of the port facility discrimination counting, For target image quality, For model parameters, For false detection rate.

[0066] The accuracy of the port facility discrimination count has a square function relationship with the target image quality, a square function relationship with four times the model parameters, and an inverse function relationship with the false detection rate, and the function relationship reflects that the accuracy of the port facility discrimination count is greatly affected by the target image quality and the model parameters, and is less affected by the false detection rate.

[0067] The embodiment of the application realizes the evaluation of the number of port facilities by target detection and target counting of the important port facilities oil tank and crane in the remote sensing image through the YOLOv5 algorithm, performs target recognition on the target port image and sets an edge region, performs threshold discrimination, then counts, discriminates the integrity of the target object according to the confidence on the edge region frame after target detection, the port facility discrimination count classifies and counts the number of target objects according to the target detection result, and discriminates the integrity of the target object in the edge region, when the target object is a complete facility, adds 1 to the total value of the count; when the target object is not a complete facility, adds 0.5 to the total value of the count, and the other regions are normally counted. Through target detection and target counting of the important port facilities oil tank and crane, the material situation of the port can be mastered, and the richness of the port related evaluation can be increased, accurate counting of the facilities with incomplete edges in the image can be implemented, multiple counting of a port facility can be avoided, and the accuracy of target detection and counting is improved.

[0068] The above is only a specific embodiment of the application, but the protection scope of the application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical range disclosed in the application, which should be covered in the protection scope of the application. Therefore, the protection scope of the application should be subject to the protection scope of the claims.

[0069] Finally: the above is only a preferred embodiment of the application, and is not used to limit the application, any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the application should be included in the protection scope of the application.

Claims

1. A YOLO-based port facility remote sensing image detection and counting discrimination method, characterized in that, The remote sensing image of the target port is screenshot to obtain a target port image set, and then the YOLOv5 algorithm is used to detect and count the important facilities of the port, such as oil tanks and cranes, in the target port image set, so as to evaluate the number of port facilities. The target port image is identified and an edge region is set, threshold discrimination is performed, and then counting is performed. The integrity of the target object in the edge region is judged according to the confidence of the bounding box after target detection. The number of target objects is classified and counted according to the target detection result, and the integrity of the target object in the edge region is judged. When the target object is a complete facility, 1 is added to the total count value; when the target object is not a complete facility, 0.5 is added to the total count value, and other regions are normally counted. Step one, image set making: select the target port in the ship news network to obtain the target port image, ensure that the target port image content does not overlap, and generate the target port image set; Step two, YOLOv5 model training: pre-process the target port image set and label it, assign classes and bounding boxes to target objects in the target port image set, and optimize the model using data augmentation strategy; Step three, target detection of port oil tank and crane: use the trained YOLOv5 model to detect the target port image set, output the class, bounding box coordinates and confidence of the detected target object; Step four, threshold discrimination: set an edge region for the target port image, calculate the accuracy and recall rate under different confidence thresholds, and draw the precision-recall curve to select the optimal confidence; Step five, port facility discrimination counting: according to the target detection result, the number of target objects is classified and counted, and the integrity of the target object is judged; Step six, accuracy evaluation: establish an evaluation model through target image quality, model parameters and false detection rate to evaluate the accuracy of the port facility discrimination counting and output the total number of port facilities.

2. The YOLO-based port facility remote sensing image detection and counting discrimination method according to claim 1, characterized in that, In step three, the detected target object is the oil tank and crane of the target port. 3.The YOLO-based port facility remote sensing image detection and counting discrimination method of claim 1, wherein, In step two, the YOLOv5 model is trained by pre-processing and labeling the target port image, assigning classes and bounding boxes to target objects in the target port image, and optimizing the model using data augmentation strategy. The specific steps are as follows: Step Q1, image preprocessing: increase the clarity of the target port image by super-resolution, and then label the target port image, assign classes and bounding boxes to each target object; Step Q2, model selection: select YOLOv5 model as the port target detection model; Step Q3, model training: train the YOLOv5 model with the labeled target port image set, and use data augmentation strategy to improve the performance of the model during training; Step Q4, model verification: evaluate the performance of the model with the target port image set, the evaluation indicators include accuracy, recall rate and F1 score, and optimize the model according to the verification result.

4. The YOLO-based port facility remote sensing image detection and counting discrimination method according to claim 3, characterized in that, Step Q3 model training trains the YOLOv5 model through the labeled target port image set, and adopts a data enhancement strategy to improve the performance of the model during the training process, wherein the data enhancement strategy is determined through a density matching algorithm, and the determination formula of the best data enhancement strategy is as follows: ； wherein: is the best data augmentation strategy, is the expected accuracy, is an image in the target port image set, is the data augmentation strategy, is the finding parameter.

5. The YOLO-based port facility remote sensing image detection and counting discrimination method according to claim 1, characterized in that, Step three, target detection of the port oil tank and the crane, the target detection of the target port image set is performed through the trained YOLOv5 model, the class, the boundary box coordinates and the confidence of the detected target object are output, and the specific steps are as follows: Step Z1, image reprocessing: scaling, cropping and rotating operations are performed on the target port image set to adapt to the input requirements of the YOLOv5 model; Step Z2, port target detection: the target detection of the port facility remote sensing picture is performed through the trained YOLOv5 model, and the class, the boundary box coordinates and the confidence of the detected target object are output; Step Z3, image post-processing: the boundary box coordinates output by the non-maximum suppression are used to eliminate the overlapping boundary boxes. 6.The YOLO-based port facility remote sensing image detection and counting discrimination method of claim 1, wherein, Step four, threshold discrimination: an edge region is set for the target port image, the accuracy and the recall rate under different confidence thresholds are calculated, and the precision-recall curve is drawn, the optimal confidence is selected according to the precision-recall curve, and the target object is discriminated according to the optimal confidence on the edge region of the frame after the target detection, whether it is a complete facility or a segmented facility.

7. The YOLO-based port facility remote sensing image detection and counting discrimination method according to claim 1, characterized in that, Step five, port facility discrimination counting: according to the target detection result, the number of target objects is classified and counted, and the integrity of the target object in the edge region is discriminated, when the target object is a complete facility, 1 is added to the total value of the count; When the target object is not a complete facility, 0.5 is added to the total value of the count, and the other regions are normally counted. 8.The YOLO-based port facility remote sensing image detection and counting discrimination method of claim 1, wherein, Step six, accuracy evaluation: the accuracy evaluation model is established through the target image quality, the model parameters and the false detection rate, and the accuracy of the port facility discrimination counting is evaluated, wherein the formula of the accuracy evaluation model is as follows: ； In the formula: is the accuracy of the port facility discrimination count, is the target image quality, is the model parameter, is the false detection rate.

Images