Railway environment color steel tile hidden danger identification and evaluation method based on Yolov8

Through the Yolov8-based railway environment color steel tile hidden danger identification evaluation method, the satellite remote sensing image and area weighted loss function optimization model is used to solve the problem of low recognition efficiency of color steel tile hidden dangers in railway environment, achieving higher recognition accuracy and better Fast troubleshooting cycle.

CN119091321APending Publication Date: 2024-12-06INFORMATION TECH INST OF CHINA RAILWAY ZHENGZHOU BUREAU GRP CO LTD

Patent Information

Application Number
CN202411282147.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-13
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The prior art is inefficient in identifying hidden dangers of color steel tiles in railway environments, relying on manpower inspections and unable to achieve immediate discovery and rapid response. In addition, drone aerial photography is limited in some areas, and the recognition accuracy is insufficient.

Method used

Using Yolov8-based railway environmental color steel tile hidden danger identification and evaluation method, multi-spectral and full-color images are obtained through national satellite remote sensing imaging, pre-processing and fusion, combined with the regional weighted loss function optimization model, YOLOv8 recognition model is trained to improve color steel tile recognition accuracy.

Benefits of technology

Acquisition of clear and rich images through remote sensing satellites, and using the YOLOv8 algorithm to train the model, significantly improve the recognition accuracy of color steel tile hidden dangers, avoid manual secondary judgment, save manpower and inspection cycle, and perform better in the recognition of small-sized object features .

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119091321A_ABST
    Figure CN119091321A_ABST
Patent Text Reader

Abstract

The invention relates to a Yolov8-based railway environment color steel tile hidden danger identification and evaluation method in the technical field of image identification. The method comprises the following steps: S1, obtaining a remote sensing image along a railway through satellite remote sensing imaging; s2, preprocessing the remote sensing image; performing radiometric calibration, atmospheric correction and ortho-rectification on the multispectral image, and fusing the multispectral image with the panchromatic image subjected to radiometric calibration and ortho-rectification to obtain remote sensing image data; s3, cutting and labeling the remote sensing image data, and dividing the constructed data set into a training set and a test set; s4, the training set is used for training the adopted Yov8 recognition model, the test set is used for evaluating the trained Yov8 recognition model, and an average precision mean value is used for representing the average precision of color steel tile recognition of the Yov8 recognition model in evaluation; according to the color steel tile hidden danger identification method, a more reasonable image acquisition, processing and identification mode can be adopted, and the identification precision of the color steel tile hidden danger is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a method for identifying and assessing hidden dangers of color steel tiles in a railway environment based on Yolov8. Background Art

[0002] Colored steel sheet hidden danger is one of the main hidden dangers around railways, and its existence cannot be ignored. In view of the vast territory of my country and the complex environment along the railway, the railway management department has to conduct regular inspections to follow the principle of prevention first, aiming to timely discover and eliminate safety hazards such as colored steel sheets; however, the current inspection work of the railway surrounding environment mainly relies on manpower, which not only consumes a lot of human resources, but also is limited by low efficiency and lengthy cycles, and cannot achieve immediate discovery and rapid response to hidden dangers of colored steel sheets;

[0003] There are currently two main methods for identifying hidden dangers of color-coated steel tiles. One is to use an edge extraction algorithm to automatically vectorize the remote sensing images along the railway, thereby generating vectorized images of houses around the railway. The vectorized images of houses extracted by this method cannot accurately determine whether the building is a color-coated steel tile house, and manual secondary confirmation is still required; the second is to obtain aerial orthophotos through drone aerial photography, and predict whether the building is a color-coated steel tile house by identifying the aerial orthophotos. For example, the Chinese invention patent with announcement number CN114239755A discloses an intelligent identification method for color-coated steel tile buildings along railways based on deep learning, which adopts this method. However, since some protection areas around railways do not allow the use of drones for aerial photography, this method has great limitations in obtaining aerial data. Summary of the invention

[0004] In order to overcome the shortcomings in the background technology and solve the existing technical problems, the present invention discloses a method for identifying and evaluating hidden dangers of color steel tiles in a railway environment based on Yolov8, which can adopt more reasonable image acquisition, processing and recognition methods to improve the recognition accuracy of hidden dangers of color steel tiles.

[0005] In order to achieve the above-mentioned invention object, the present invention adopts the following technical scheme:

[0006] A method for identifying and evaluating hidden dangers of color steel tiles in a railway environment based on Yolov8 comprises the following steps: S1, obtaining remote sensing images along the railway through national satellite remote sensing imaging, wherein the remote sensing images include multispectral images and panchromatic images; S2, preprocessing the remote sensing images; performing radiation calibration, atmospheric correction and orthorectification on the multispectral images, and fusing them with the panchromatic images that have undergone radiation calibration and orthorectification to obtain remote sensing image data; S3, cropping and annotating the remote sensing image data, and dividing the constructed data set into a training set and a test set; S4, using the training set to train the adopted Yolov8 recognition model, and using the test set to evaluate the trained Yolov8 recognition model, wherein the evaluation adopts the average precision mean to represent the average precision of the Yolov8 recognition model in identifying the color steel tiles.

[0007] Furthermore, the regional weighted loss function is introduced to optimize the loss function of the Yolov8 recognition model, and its formula is:

[0008] (1)

[0009] In formula (1), The first Color steel tile target; Indicates the total number of color steel tiles on an image; The probability that the object is predicted to be a colored steel tile; Indicates the true label of the object; It is a vector containing 4 parameters, which represent the predicted coordinates of the color steel tile; It is also a vector containing 4 parameters, representing the actual coordinates of the color steel tile; Indicates the degree of overlap between the predicted frame and the real frame area of ​​the color steel tile, 1 means that their overlapping area is greater than 50%, and 0 means that their overlapping area is less than 50%;

[0010] In formula (1) Represents the loss function of classification, and its formula is:

[0011] (2)

[0012] In formula (1) represents the regression loss function of the anchor box, and its formula is:

[0013] (3)

[0014] In formula (3), Represents the true area of ​​the i-th anchor box; Represents the predicted area of ​​the i-th anchor box; express and The area of ​​intersection; Represents the Euclidean distance between the center point of the real anchor box and the predicted anchor box, Represents the diagonal length of the circumscribed rectangle between the real anchor box and the predicted anchor box; Indicates the similarity of the aspect ratio between the real anchor box and the predicted anchor box. The more similar they are, the smaller the value.

[0015] In formula (1), Represents the weight coefficient of the anchor box, and introduces a monotone decreasing function so that The value of is always greater than 1, and its formula is: (4), where represents the area of ​​the i-th anchor box; Substituting formula (4) into formula (1), the formula of the Loss function is:

[0016] (5).

[0017] Furthermore, in S1, it is possible to directly obtain GF-2 remote sensing images from the National Remote Sensing Data and Application Platform. The spatial resolution of multispectral images is at least 4 meters, and the spatial resolution of panchromatic images is at least 1 meter.

[0018] Furthermore, in S2, a Python batch program based on the GDAL library is used to perform the preprocessing process.

[0019] Furthermore, in S3, several images cropped from remote sensing images are 640*640 in size, and the cropped images are labeled using LabelImg software.

[0020] Furthermore, in S3, map and navigation service data can be used simultaneously to build a multi-data source and multi-scale colored steel tile dataset.

[0021] Furthermore, in S4, the training parameters of the Yolov8 recognition model are set, where epoch is set to 1000; optimizer is set to Adam, the initial learning rate is 0.001; momentum is set to 0.937; weight_decay is set to 0.0005, patience is set to 100, and batch_size is set to 4.

[0022] Furthermore, in S4, the mean average precision (mAP) is divided into two types: mAP50 and mAP50-100; mAP50 means the proportion of correct detection results when the intersection over union (IoU) between the model prediction box and the true box is greater than 50%, that is, when the overlap between the detection box and the true box is greater than 50%; mAP50-95 refers to the average value of the correct proportion of detection results when the IoU threshold ranges from 50% to 95%.

[0023] Due to the adoption of the above-mentioned technical solution, the present invention has the following beneficial effects:

[0024] The Yolov8-based railway environment color steel tile hidden danger identification and assessment method disclosed by the present invention collects image data through remote sensing satellites and performs effective processing by multiple means, which not only avoids the limitations of traditional use of drones, but also can obtain clearer and more informative images. Therefore, after the recognition model is trained using the advanced Yolov8 target detection algorithm, the recognition accuracy of color steel tiles can be greatly improved, manual secondary judgment can be avoided, and manpower and investigation cycle can be effectively saved.

[0025] In addition, the Yolov8 recognition model can be optimized by introducing a regional weighted loss function, aiming to guide the neural network to more accurately capture and analyze the detailed features of small-sized objects, thereby further significantly enhancing the recognition accuracy of hidden dangers of color-coated steel tiles. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is the first comparison diagram of the original image, the Yolov8 recognition image and the optimized Yolov8 recognition image;

[0027] Figure 2 It is a second comparison diagram of the original image, the Yolov8 recognition image and the optimized Yolov8 recognition image;

[0028] Figure 3 3 is a third comparison diagram of the original image, the Yolov8 recognized image and the optimized Yolov8 recognized image. DETAILED DESCRIPTION

[0029] The technical solution of the present invention will be described below in conjunction with the accompanying drawings in the embodiments of the present invention:

[0030] The present invention is based on Yolov8 railway environment color steel tile hidden danger identification and assessment method, comprising the following steps:

[0031] Step 1: Obtain remote sensing images along the railway through national satellite remote sensing imaging. The remote sensing images include multispectral images and panchromatic images. Specifically, obtain the Gaofen-2 (GF-2) remote sensing images from the National Remote Sensing Data and Application Platform. A total of 974 remote sensing images were downloaded, of which 747 covered the railway. The GF-2 images include multispectral images with a spatial resolution of 4 meters and panchromatic images with a spatial resolution of 1 meter. These remote sensing image data are used to produce a dataset of color steel tiles around the railway.

[0032] Step 2: In order to produce a data set that meets the requirements of Yolov8 target detection, the remote sensing images need to be preprocessed; the preprocessing steps include radiation calibration, atmospheric correction, orthorectification and image fusion, and the preprocessing process is performed using a Python batch program based on the GDAL class library; first, the panchromatic image and multispectral image are radiometrically calibrated separately, and the sensor DN value is converted into radiation brightness and reflectivity, and then the multispectral image is atmospherically corrected to eliminate the effects of water vapor, oxygen, carbon dioxide, methane and ozone in the atmosphere on the reflection of the ground objects, and the effects of atmospheric molecules and aerosol scattering, and then the panchromatic image and multispectral image are orthorectified separately to obtain the correct geographic coordinate information, and finally the processed multispectral image is fused with the panchromatic image to obtain clearer and more informative remote sensing image data.

[0033] Step 3: Crop and label the remote sensing image data, and divide the constructed data set into a training set and a test set. Specifically, crop the remote sensing image into a 640*640 size picture. One remote sensing image can be cropped into about 800 remote sensing pictures. The cropped pictures are labeled using LabelImg software. Of course, map and navigation service data can also be used to construct a multi-data source and multi-scale color steel tile data set. For example, Google map tile data can be used for online labeling at the same time. The labeling format is the same as that of LabelImg, so that the total number of labeled color steel tile pictures can be increased to 17,816. After the picture labeling is completed, start to build the data set. The training set and test set are constructed using a ratio of 8:2.

[0034] Step 4: Use the training set to train the Yolov8 recognition model. Before training, set the training parameters of the Yolov8 recognition model, where epoch is set to 1000; optimizer is set to Adam, the initial learning rate is 0.001; momentum is set to 0.937; weight_decay is set to 0.0005, patience is set to 100, and batch_size is set to 4;

[0035] The trained Yolov8 recognition model was evaluated using the test set. The mean average precision was used to represent the average precision of the Yolov8 recognition model in identifying color steel tiles. The mean average precision mAP was divided into two types: mAP50 and mAP50-100. mAP50 represents the proportion of correct detection results when the intersection over union (IoU) between the model prediction box and the true box is greater than 50%, that is, when the overlap between the detection box and the true box is greater than 50%. mAP50-95 refers to the average value of the correct proportion of detection results when the IoU threshold ranges from 50% to 95%.

[0036] After completing the model evaluation, it will be found that the accuracy of the Yolov8 recognition model may not be very high. This is because the color steel tiles occupy a small number of pixels in the remote sensing image. Compared with these small features, YOLOv8 can easily identify some large target objects. In order to improve the detection accuracy of small objects, this paper proposes a regional weighted loss function, which aims to make YOLOv8 pay more attention to small target features, thereby improving the accuracy of color steel tile recognition;

[0037] The regional weighted loss function is introduced to optimize the loss function of the Yolov8 recognition model. The formula is:

[0038] (1)

[0039] In formula (1), The first Color steel tile target; Indicates the total number of color steel tiles on an image; The probability that the object is predicted to be a colored steel tile; Indicates the true label of the object; It is a vector containing 4 parameters, which represent the predicted coordinates of the color steel tile; It is also a vector containing 4 parameters, representing the actual coordinates of the color steel tile; Indicates the degree of overlap between the predicted frame and the real frame area of ​​the color steel tile, 1 means that their overlapping area is greater than 50%, and 0 means that their overlapping area is less than 50%;

[0040] In formula (1) Represents the loss function of classification, and its formula is:

[0041] (2)

[0042] In formula (1) represents the regression loss function of the anchor box, and its formula is:

[0043] (3)

[0044] In formula (3), Represents the true area of ​​the i-th anchor box; Represents the predicted area of ​​the i-th anchor box; express and The area of ​​intersection; Represents the Euclidean distance between the center point of the real anchor box and the predicted anchor box, Represents the diagonal length of the circumscribed rectangle between the real anchor box and the predicted anchor box; Indicates the similarity of the aspect ratio between the real anchor box and the predicted anchor box. The more similar they are, the smaller the value.

[0045] In formula (1), Represents the weight coefficient of the anchor box. In order to improve the accuracy of small target recognition, its value is inversely proportional to the area of ​​the anchor box, that is, the smaller the area of ​​the anchor box, the larger its value. However, since the length and width of the anchor box are normalized in the Yolov8 algorithm, resulting in the area of ​​the anchor box being less than 1, the present invention introduces a monotonically decreasing function so that The value of is always greater than 1, and its formula is: (4), where represents the area of ​​the i-th anchor box; Substituting formula (4) into formula (1), the formula of the Loss function is:

[0046] (5).

[0047] In order to intuitively demonstrate the effectiveness of the regional weighted loss function proposed in this invention, the YOLOv8 recognition model and the YOLOv8 recognition model optimized after adding the regional function loss (YOLOv8+AreaLoss) are used for comparison. The model training accuracy is shown in the following table: Model Precision Recall F1-Score mAP50 mAP50-95 YOLOv8 0.791 0.782 0.786 0.794 0.533 YOLOv8+AreaLoss 0.814 0.796 0.805 0.820 0.558

[0048] As shown in the table, compared with YOLOv8, after using the regional loss function, the F1-Score value, mAP50 and mAP50-95 increased by 0.023, 0.026, and 0.025 respectively;

[0049] The original YOLOv8 recognition model and the YOLOv8 recognition model with AreaLoss added have the following actual recognition effects: Figure 1 , two As shown in Figure 3, the first column is the original image to be detected, the second column is the effect of original YOLOv8 recognition, and the third column is the effect of YOLOv8 recognition after adding AreaLoss function.

[0050] The parts of the present invention that are not described in detail are prior art. It is obvious to those skilled in the art that the present invention is not limited to the details of the above-mentioned exemplary embodiments, and the present invention can be implemented in other specific forms without departing from the spirit or basic characteristics of the present invention. Therefore, no matter from which point of view, the above-mentioned embodiments should be regarded as exemplary and non-restrictive. The scope of the present invention is limited by the attached claims rather than the above description. Therefore, it is intended to include all changes within the meaning and scope of the equivalent elements of the claims in the present invention, and any figure marks in the claims should not be regarded as limiting the content of the claims involved.

Claims

1. A railway environmental color steel tile hidden danger identification and assessment method based on Yolov8, characterized by: Follow these steps: S1. Obtain remote sensing images along the railway line through national satellite remote sensing imaging, which include multispectral images and panchromatic images; S2. Preprocessing the remote sensing image: after radiometric calibration, atmospheric correction and orthorectification of the multispectral image, fuse it with the radiometric calibration and orthorectified panchromatic image to obtain remote sensing image data; S3, cropping and annotating the remote sensing image data, and dividing the constructed data set into a training set and a test set; S4. Use the training set to train the adopted Yolov8 recognition model, and use the test set to evaluate the trained Yolov8 recognition model. The evaluation uses the average precision mean to represent the average precision of the Yolov8 recognition model in identifying the color steel tiles.

2. The method for identifying and assessing hidden dangers of color steel tiles in railway environment based on Yolov8 according to claim 1 is characterized by: introducing The regional weighted loss function optimizes the loss function of the Yolov8 recognition model, and its formula is: (1) In formula (1), The first Color steel tile target; Indicates the total number of color steel tiles on an image; The probability that the object is predicted to be a colored steel tile; Indicates the true label of the object; It is a vector containing 4 parameters, which represent the predicted coordinates of the color steel tile; It is also a vector containing 4 parameters, representing the actual coordinates of the color steel tile; Indicates the degree of overlap between the predicted frame and the real frame area of ​​the color steel tile, 1 means that their overlapping area is greater than 50%, and 0 means that their overlapping area is less than 50%; In formula (1) Represents the loss function of classification, and its formula is: (2) In formula (1) represents the regression loss function of the anchor box, and its formula is: (3) In formula (3), Represents the true area of ​​the i-th anchor box; Represents the predicted area of ​​the i-th anchor box; express and The area of ​​intersection; Represents the Euclidean distance between the center point of the real anchor box and the predicted anchor box, Represents the diagonal length of the circumscribed rectangle between the real anchor box and the predicted anchor box; Indicates the similarity of the aspect ratio between the real anchor box and the predicted anchor box. The more similar they are, the smaller the value. In formula (1), Represents the weight coefficient of the anchor box, and introduces a monotone decreasing function so that The value of is always greater than 1, and its formula is: (4), where represents the area of ​​the i-th anchor box; Substituting formula (4) into formula (1), the formula of the Loss function is: (5)。 3. The method for identifying and assessing hidden dangers of railway environmental colored steel tiles based on Yolov8 according to claim 1 is characterized in that: In S1, the GF-2 remote sensing images can be directly obtained from the national remote sensing data and application platform. The spatial resolution of multispectral images is at least 4 meters, and the spatial resolution of panchromatic images is at least 1 meter.

4. The method for identifying and assessing hidden dangers of railway environmental colored steel tiles based on Yolov8 according to claim 1 is characterized by: In S2, the preprocessing process is performed using a Python batch program implemented based on the GDAL class library.

5. The method for identifying and assessing hidden dangers of railway environmental colored steel tiles based on Yolov8 according to claim 1 is characterized by: In S3, several images cropped from remote sensing images are 640*640 in size, and the cropped images are labeled using LabelImg software.

6. The railway environment color steel tile hidden danger identification and assessment method based on Yolov8 according to claim 1 is characterized by: In S3, map and navigation service data can be used simultaneously to build a multi-data source, multi-scale colored steel tile dataset.

7. The method for identifying and assessing hidden dangers of railway environmental colored steel tiles based on Yolov8 according to claim 1 is characterized in that: In S4, the training parameters of the Yolov8 recognition model are set, where epoch is set to 1000; optimizer is set to Adam, the initial learning rate is 0.001; momentum is set to 0.937; weight_decay is set to 0.0005, patience is set to 100, and batch_size is set to 4.

8. The method for identifying and assessing hidden dangers of railway environmental colored steel tiles based on Yolov8 according to claim 1 is characterized by: In S4, the mean average precision (mAP) is divided into two types: mAP50 and mAP50-100. mAP50 means the proportion of correct detection results when the intersection over union (IoU) between the model prediction box and the true box is greater than 50%, that is, when the overlap between the detection box and the true box is greater than 50%. mAP50-95 refers to the average value of the correct proportion of detection results when the IoU threshold ranges from 50% to 95%.

Citation Information

Patent Citations

  • Deep learning-based intelligent identification method for color steel tile building along railway

    CN114239755A

Cited By

  • Method and system for identifying hidden danger of color steel tile along railway based on remote sensing image

    CN121904616A