Color steel room detection and danger level classification method based on unmanned aerial vehicle inspection

The drone-based inspection method effectively addresses inefficiencies in existing color steel house detection by integrating image processing and classification modules to provide accurate and efficient detection and hazard assessment of color steel houses.

CN119068365APending Publication Date: 2024-12-03BEIJING UNIV OF CIVIL ENG & ARCHITECTURE +1

Patent Information

Application Number
CN202411172158.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-26
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

Existing methods for detecting and classifying color steel houses along tracks are inefficient, as track inspection vehicles struggle to identify roof-mounted plates, and manual inspections are labor-intensive and time-consuming, posing safety risks due to potential plate detachment.

Method used

A drone-based inspection method utilizing a drone inspection module, image data transmission and processing, model detection and classification, and geographical location and hazard level display modules to accurately detect and classify color steel houses, integrating image segmentation, feature extraction, and damage type classification.

Benefits of technology

The method enables efficient and accurate detection of color steel houses, providing precise geographical location and hazard level classification, facilitating targeted maintenance and reducing safety hazards.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068365A_ABST
    Figure CN119068365A_ABST
Patent Text Reader

Abstract

The invention discloses a color steel room detection and danger level classification method based on unmanned aerial vehicle inspection, and the method comprises the steps: carrying out the correlation storage of image data and corresponding position coordinate information through an image data transmission and processing module, and carrying out the segmentation processing of the image data, the segmentation module is used for segmenting the image data, numbering the segmented image data to ensure splicing of subsequent image data, and transmitting the segmented image data to the model detection and classification module; and the model detection and classification module transmits the processed picture data into a trained detection and classification model, detects and extracts various color steel rooms in the pictures, classifies the color steel rooms in the pictures according to different damage types, and outputs geographical positions, area sizes and damage type information of the color steel rooms.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned aerial vehicles, and in particular relates to a method for detecting and classifying color steel houses based on unmanned aerial vehicle inspections and danger levels. Background Art

[0002] The color steel plates along the existing tracks often have problems such as improper installation, aging and rust. In windy weather, they may fall off and fly into the track, causing danger. The current mainstream method of environmental monitoring along the track is track inspection vehicle inspection. This method is difficult to detect color steel plates located on the roof in the environment along the track, and manual inspection is time-consuming and labor-intensive, and the effect is not good. Summary of the invention

[0003] The invention provides a method for detecting and classifying color steel houses based on drone inspection, and the method comprises:

[0004] S1. The drone inspection module, the image data transmission and processing module, the model detection and classification module, and the color steel room geographical location and hazard level display module are sequentially connected;

[0005] S2. Use the drone inspection module to inspect the environment along the track, obtain image data of the color steel house cluster area around the track and its corresponding position coordinate information, and transmit it to the image data transmission and processing module;

[0006] S3. Use the image data transmission and processing module to associate and save the image data and its corresponding position coordinate information, segment the image data, number the segmented image data to ensure the splicing of subsequent image data, and transmit the segmented image data to the model detection and classification module;

[0007] S4. The model detection and classification module transfers the processed image data to the trained detection and classification model, detects and extracts various color steel houses in the image, classifies the color steel houses in the image according to different damage types, and outputs the geographical location, area size, and damage type information of the color steel houses;

[0008] S5. The module for displaying the geographical location and hazard level of color steel houses stitches the pictures into a complete picture according to the numbers and displays it in the UI interface, including the number of color steel houses in the picture, the geographical location of the color steel houses, the size of the area, and the type of damage. The hazard level of the color steel houses is classified according to the size of the area and the type of damage, and they are divided into high-risk targets, medium-risk targets, and low-risk targets.

[0009] Furthermore, the location coordinate information is specifically the longitude and latitude information obtained based on the GNSS module in the UAV inspection module.

[0010] Further, the step S3 specifically includes:

[0011] The image data of each picture is divided into a plurality of sub-image data according to preset fixed row pixel and column pixel intervals, and the sub-image data are numbered consecutively in the order from left to right and from top to bottom in the picture.

[0012] Further, in step S4, the processed multiple groups of sub-image data are input into the trained detection and classification model to detect and extract various color steel houses in the picture, which specifically includes the following steps:

[0013] S411. Get sub-image data extraction, get the processed pixel value I(x,y) of the point (x,y) in the sub-image,

[0014] I'(x,y)=0.3I R (x,y)+0.6I G (x,y)+0.1I B (x,y)

[0015] I(x,y)=∑ m ∑ n I'(xm,yn)*K(m,n)

[0016] Among them, I R (x, y) is the pixel value of the red channel point (x, y) of the sub-image, I G (x, y) is the pixel value of the green channel point (x, y) of the sub-image, I B (x, y) is the pixel value of the blue channel point (x, y) of the sub-image, m is the offset of the convolution kernel in the x direction, n is the offset of the convolution kernel in the y direction, and K is the convolution kernel;

[0017] S412. Calculate the characteristic attribute S(x,y) of the point (x,y),

[0018] S(x,y)=∑ m ∑ n (M(m,n)-u M )×(I(x+m,y+n)-u I )

[0019] Among them, M(m,n) is the pixel value of point (m,n) in the template image, u M is the mean value of the pixel values ​​at each point in the sample image, u I is the mean value of the pixel values ​​at each point of the sub-image data;

[0020] S413. Extract the color steel house data H(x,y), where H(x,y) is the pixel value of the color steel house area.

[0021] H(x,y)=I(x,y),if S(x,y)≥T

[0022] H(x,y)=0,if S(x,y) <T

[0023] Among them, T is the set threshold;

[0024] S414. Calculate the area of ​​the color steel house by counting the total number of pixels where H(x,y)≠0.

[0025] Further, in step S4, classifying the color steel houses in the picture according to the damage types specifically includes the following steps:

[0026] S421. Extract feature vector F,

[0027] F=[H(1,1),H(1,1),…,H(1,N),H(2,1),…,H(M,N)]

[0028] Wherein, M is the height of the sub-image data, and N is the width of the sub-image data;

[0029] S422. Calculate intermediate output z l ,

[0030] z l =W l ·F+b l

[0031] Among them, W l is the weight corresponding to the lth damage type, and b l Bias parameter corresponding to the lth damage type;

[0032] S423. Calculate the probability p l ,

[0033]

[0034] Among them, k is the independent variable, L is the total number of damage types;

[0035] S424. Each probability p corresponding to L damage types l Sort, and the damage type corresponding to the highest ranking is used as the damage type classification result.

[0036] The present invention also relates to a computer program product, which includes a computer program, and the computer program is executed by a processor to execute the above-mentioned color steel house detection and hazard level classification method based on drone inspection.

[0037] The present invention also relates to a computer-readable storage medium, characterized in that the computer-readable storage medium is used to store a computer program, and the computer program is executed by a processor to execute the above-mentioned color steel house detection and hazard level classification method based on drone inspection.

[0038] The technical solution of the present invention can obtain the top picture and location information of the environment along the track. The improved detection model can accurately and efficiently detect the color steel house in the picture data. The integrated detection grading display software facilitates subsequent processing personnel to select appropriate processing methods and locate the problem color steel house. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a flow chart of the method for detecting and classifying color steel houses and danger levels based on drone inspections of the present invention. DETAILED DESCRIPTION

[0040] The present invention will be further described below in conjunction with the accompanying drawings. The following examples are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the scope of protection of the present invention. It should be pointed out that the following detailed description is exemplary and is intended to provide further explanation of the present invention.

[0041] Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those generally understood by those of ordinary skill in the art to which the present invention belongs. It should be noted that the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0042] Embodiment 1 of the present invention relates to a method for detecting and classifying color steel houses based on drone inspection and hazard level, the method comprising the following steps:

[0043] S1. The UAV inspection module, the image data transmission and processing module, the model detection and classification module, and the color steel room geographical location and hazard level display module are connected in sequence.

[0044] S2. Use the drone inspection module to inspect the environment along the track, obtain image data of the color steel house cluster area around the track and its corresponding location coordinate information, and transmit it to the image data transmission and processing module.

[0045] The location coordinate information is specifically the longitude and latitude information obtained by the GNSS module in the drone inspection module.

[0046] S3. Use the image data transmission and processing module to associate and save the image data and its corresponding position coordinate information, segment the image data, and number the segmented image data to ensure the splicing of subsequent pictures, and transmit the segmented image data to the model detection and classification module.

[0047] Among them, the image data of each picture is divided into multiple sub-image data according to preset fixed row pixel and column pixel intervals, so as to reduce the workload of single processing and improve work efficiency, and the sub-image data are numbered consecutively in the order from left to right and from top to bottom in the picture.

[0048] S4. The model detection and classification module passes the processed image data into the trained detection and classification model, detects and extracts various color-coated steel houses in the image, classifies the color-coated steel houses in the image according to the different damage types, and outputs the geographical location, area size, and damage type information of the color-coated steel houses.

[0049] The processed multiple groups of sub-image data are input into the trained detection and classification model to detect and extract various color steel houses in the picture, which specifically includes the following steps:

[0050] S411. Get sub-image data extraction, get the processed pixel value I(x,y) of the point (x,y) in the sub-image,

[0051] I'(x,y)=0.3I R (x,y)+0.6I G (x,y)+0.1I B (x,y)

[0052] I(x,y)=∑ m ∑ n I'(xm,yn)*K(m,n)

[0053] Among them, I R (x, y) is the pixel value of the red channel point (x, y) of the sub-image, I G (x, y) is the pixel value of the green channel point (x, y) of the sub-image, I B (x, y) is the pixel value of the blue channel point (x, y) of the sub-image, m is the offset of the convolution kernel in the x direction, n is the offset of the convolution kernel in the y direction, and K is the convolution kernel.

[0054] S412. Calculate the characteristic attribute S(x,y) of the point (x,y),

[0055] S(x,y)=∑ m ∑ n (M(m,n)-u M )×(I(x+m,y+n)-u I )

[0056] Among them, M(m,n) is the pixel value of point (m,n) in the template image, u M is the mean value of the pixel values ​​at each point in the sample image, u I is the mean value of the pixel values ​​at each point in the sub-image data.

[0057] S413. Extract the color steel house data H(x,y), where H(x,y) is the pixel value of the color steel house area.

[0058] H(x,y)=I(x,y),if S(x,y)≥T

[0059] H(x,y)=0,if S(x,y) <T

[0060] Wherein, T is the set threshold.

[0061] S414. Calculate the area of ​​the color steel house by counting the total number of pixels where H(x,y)≠0.

[0062] The classification of the color steel houses in the picture according to the damage types specifically includes the following steps:

[0063] S421. Extract feature vector F,

[0064] F=[H(1,1),H(1,1),…,H(1,N),H(2,1),…,H(M,N)]

[0065] Wherein, M is the height of the sub-image data, and N is the width of the sub-image data.

[0066] S422. Calculate intermediate output z l ,

[0067] z l =W l ·F+b l

[0068] Among them, W l is the weight corresponding to the lth damage type, and b l Bias parameter corresponding to the lth damage type.

[0069] S423. Calculate the probability p l ,

[0070]

[0071] Among them, k is the independent variable, L is the total number of damage types;

[0072] S424. Each probability p corresponding to L damage types lSort, and the damage type corresponding to the highest ranking is used as the damage type classification result.

[0073] S5. The module for displaying the geographical location and hazard level of color steel houses stitches the pictures into a complete picture according to the numbers and displays it in the UI interface, including the number of color steel houses in the picture, the geographical location of the color steel houses, the size of the area, and the type of damage. The hazard level of the color steel houses is classified according to the size of the area and the type of damage, and they are divided into high-risk targets, medium-risk targets, and low-risk targets.

[0074] Embodiment 2 of the present invention relates to a computer program product, which includes a computer program. The computer program is executed by a processor to execute the method for detecting and classifying color steel houses and danger levels based on drone inspection of embodiment 1.

[0075] Embodiment 3 of the present invention relates to a computer-readable storage medium, which is used to store a computer program. The computer program is executed by a processor to execute the method for detecting and classifying color steel houses and danger levels based on drone inspections of Embodiment 1.

[0076] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the protection scope of the present invention.

Claims

1. A method for detecting and classifying color steel houses based on drone inspections and their danger levels, characterized in that: The method comprises: S1. The drone inspection module, the image data transmission and processing module, the model detection and classification module, and the color steel room geographical location and hazard level display module are sequentially connected; S2. Use the drone inspection module to inspect the environment along the track, obtain image data of the color steel house cluster area around the track and its corresponding position coordinate information, and transmit it to the image data transmission and processing module; S3. Use the image data transmission and processing module to associate and save the image data and its corresponding position coordinate information, segment the image data, number the segmented image data to ensure the splicing of subsequent image data, and transmit the segmented image data to the model detection and classification module; S4. The model detection and classification module transfers the processed image data to the trained detection and classification model, detects and extracts various color steel houses in the image, classifies the color steel houses in the image according to different damage types, and outputs the geographical location, area size, and damage type information of the color steel houses; S5. The module for displaying the geographical location and hazard level of color steel houses stitches the pictures into a complete picture according to the numbers and displays it in the UI interface, including the number of color steel houses in the picture, the geographical location of the color steel houses, the size of the area, and the type of damage. The hazard level of the color steel houses is classified according to the size of the area and the type of damage, and they are divided into high-risk targets, medium-risk targets, and low-risk targets.

2. The method for detecting and classifying color steel houses and danger levels based on drone inspection according to claim 1 is characterized in that: The location coordinate information is specifically the longitude and latitude information obtained by the GNSS module in the drone inspection module.

3. The method for color steel house detection and hazard level classification based on drone inspection according to claim 1 is characterized in that: The step S3 specifically includes: The image data of each picture is divided into a plurality of sub-image data according to preset fixed row pixel and column pixel intervals, and the sub-image data are numbered consecutively in the order from left to right and from top to bottom in the picture.

4. The method for detecting and classifying color steel houses and danger levels based on drone inspection according to claim 1 is characterized in that: In step S4, the processed multiple groups of sub-image data are input into the trained detection and classification model to detect and extract various color steel houses in the picture, which specifically includes the following steps: S411. Get sub-image data extraction, get the processed pixel value I(x,y) of the point (x,y) in the sub-image, I'(x,y)=0.3I R (x,y)+0.6I G (x,y)+0.1I B (x,y) I(x,y)=∑ m ∑ n I'(x-m,y-n)*K(m,n) Among them, I R (x, y) is the pixel value of the red channel point (x, y) of the sub-image, I G (x, y) is the pixel value of the green channel point (x, y) of the sub-image, I B (x, y) is the pixel value of the blue channel point (x, y) of the sub-image, m is the offset of the convolution kernel in the x direction, n is the offset of the convolution kernel in the y direction, and K is the convolution kernel; S412. Calculate the characteristic attribute S(x,y) of the point (x,y), S(x,y)=∑ m ∑ n (M(m,n)-u M )×(I(x+m,y+n)-u I ) Among them, M(m,n) is the pixel value of point (m,n) in the template image, u M is the mean value of the pixel values ​​at each point in the sample image, u I is the mean value of the pixel values ​​at each point of the sub-image data; S413. Extract the color steel house data H(x,y), where H(x,y) is the pixel value of the color steel house area. H(x,y)=I(x,y),if S(x,y)≥T H(x,y)=0,if S(x,y) <T Where, T is the set threshold; S414. Calculate the area of ​​the color steel house by counting the total number of pixels where H(x,y)≠0.

5. The method for color steel house detection and hazard level classification based on drone inspection according to claim 1 is characterized in that: In step S4, classifying the color steel houses in the picture according to the damage types specifically includes the following steps: S421. Extract feature vector F, F=[H(1,1),H(1,1),…,H(1,N),H(2,1),…,H(M,N)] Wherein, M is the height of the sub-image data, and N is the width of the sub-image data; S422. Calculate intermediate output z l , z l =W l ·F+b l Among them, W l is the weight corresponding to the lth damage type, and b l Bias parameter corresponding to the lth damage type; S423. Calculate the probability p l , Among them, k is the independent variable, L is the total number of damage types; S424. Each probability p corresponding to L damage types l Sort, and the damage type corresponding to the highest ranking is used as the damage type classification result.

6. A computer program product, characterized in that The computer program product includes a computer program, which is executed by a processor and is used to execute the method for detecting and classifying color steel houses and danger levels based on drone inspection as described in any one of claims 1 to 5.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and the computer program is executed by a processor to execute the method for detecting and classifying color steel houses and danger levels based on drone inspection as described in any one of claims 1 to 5.

Citation Information

Patent Citations

  • Welding spot recognitio and positioning method based on template matching and distribution characteristics

    CN112101411A

  • Railway operation environment abnormity identification method based on image and laser data fusion

    CN114266891A

  • Rail transit surrounding hazard source detection method based on deep learning

    CN114898204A

  • Method for automatically extracting color steel rooms along railway

    CN115359351A

  • Wind turbine generator blade multi-type damage detection method and device based on image processing

    CN116402764A

Cited By

  • Deep learning-based intelligent detection method and system for color steel rooms along railway

    CN121214262A