Method and equipment for correcting the detection results of apparent defects of municipal roads and bridges

Abstract

The present invention provides a method and device for correcting the results of target detection of apparent defects of municipal roads and bridges. The method automatically identifies the apparent defect elements of municipal roads and bridges with the help of target detection algorithms, extracts the edge information of apparent defects based on computer vision technology, and then builds a buffer zone according to the gradient direction of the edge points to solve the size deviation and positioning deviation of the target detection frame; in addition, by optimizing the gradient direction and threshold segmentation strategy in the boundary detection algorithm, the algorithm's adaptive ability is improved to ensure the accuracy and continuity of edge extraction; a single object association and merging strategy is constructed to ensure the scale consistency of the target object. Finally, an efficient and accurate solution is provided for the target detection of apparent defects of roads and bridges in municipal work.

Description

Technical Field

[0001] The invention relates to a method and equipment for correcting detection results of apparent defects of municipal roads and bridges. Background Art

[0002] Common apparent defects of municipal roads and bridges include concrete cracks, peeling, exposed steel bar corrosion, water seepage and alkali efflux, and bridge deck damage. Each type of defect shows different visual characteristics. For example, exposed steel bar corrosion is manifested as surface concrete shedding and exposed steel bar corrosion; water seepage and alkali efflux is manifested as obvious water stains or white flocculent crystals on the concrete surface. Based on this, the target detection method can be used to realize intelligent diagnosis of bridge apparent defects.

[0003] At present, the rectangular area described by the target detection box roughly determines the location of the target object, but does not provide a detailed description of the target's geometric features and cannot be directly used for quantitative analysis of diseases. At the same time, the YOLO algorithm generates the category, location information, and confidence of the detection box based on the convolutional neural network. However, due to factors such as the target disease characteristics, data set limitations, and changes in viewing angles and scales, the detection box has two prediction biases:

[0004] Positioning deviation, that is, the center point of the detection box does not match the center of the target;

[0005] Scale deviation: The scale of the detection frame is inconsistent with the actual size of the target object, which is often manifested as a large-scale target being divided into multiple small-scale detection frames with incomplete coverage. Summary of the invention

[0006] The purpose of the present invention is to provide a method and device for correcting the detection results of apparent defects of municipal roads and bridges.

[0007] In order to solve the above problems, the present invention provides a method for correcting the detection results of apparent defects of municipal roads and bridges, comprising:

[0008] Step 1: Collect the apparent disease image and determine the apparent disease type of the apparent disease image;

[0009] Step 2: Extract edge features of the apparent disease based on the image marked with the detection frame, the apparent disease type of the detection frame, and the geometric information of the detection frame;

[0010] Step 3: Based on the final edge features of the apparent disease, correct the positioning deviation and scale deviation of the detection frame in the image after edge refinement.

[0011] Furthermore, in the above method, step 1: collecting the apparent disease image and determining the apparent disease type of the apparent disease image includes:

[0012] Step 1.1: photograph the apparent disease area of ​​the object from any angle and position to generate each apparent disease image;

[0013] Step 1.2: Record the visual features of various types of apparent diseases, define the apparent disease types based on the visual features, and obtain various apparent disease types;

[0014] Step 1.3: Perform target detection on the images in the apparent disease image set P according to a preset image target detection model; wherein the preset image target model is used to automatically identify the defined apparent disease type.

[0015] Furthermore, in the above method, step 2: extracting edge features of the apparent disease based on the image with the detection frame mark, the apparent disease type of the detection frame and the geometric information of the detection frame, includes:

[0016] Step 2.1: Convert the image marked with the detection frame into a grayscale image, and perform Gaussian filtering, image smoothing and noise removal on the grayscale image to obtain a processed image;

[0017] Step 2.2: Calculate the gradient of the processed image using a differential template of size m*m, obtain the gradient magnitude and gradient direction of each pixel in the processed image, and perform non-maximum suppression on the gradient magnitude to obtain an image with refined edges;

[0018] Step 2.3: Construct a three-threshold segmentation method; based on the three-threshold segmentation method and the image after edge refinement, determine the strong edge pixels and the weak edge pixels;

[0019] Step 2.4: Determine the final edge of the apparent disease by connecting the strong edge pixels and the weak edge pixels.

[0020] Furthermore, in the above method, step 2.3: constructing a three-threshold segmentation method; determining strong edge pixels and weak edge pixels in the image after the three-threshold segmentation method and edge refinement, includes:

[0021] Step 2.3.1: Based on the image after edge refinement, determine the threshold using the maximum inter-class variance method d ;

[0022] Assume that the grayscale range of the image after edge refinement is [ 0, L-1 ], the pixels in the image after edge refinement are divided into two categories: the first category has gray levels in [ 0, t ], accounting for the full ratio p 0 , the average gray level of the first type of gray level u 0 ; The second gray level is [ t, L-1 ], accounting for the full ratiop 1 , the average gray level of the second type of gray level u 1 ;use u represents the total average grayscale of the image, s 2 represents the between-class variance, then:

[0023] (1)

[0024] (2)

[0025] in, L Represents the maximum value of the grayscale range of the image; then traverse all possible thresholds t , t The value of 0 arrive L Integer in ; for each t Calculate the corresponding between-class variance s 2 ;when s 2 When it is maximum, there is an optimal gray level threshold d;

[0026] Step 2.3.2: Set the highest threshold d h = d At the same time, add a middle threshold d m = 0.7 d , low threshold d l = 0.4 d ; The definition of edge points includes: strong edge pixels, weak edge pixels and non-edge pixels;

[0027] Step 2.3.3: If the gradient magnitude of the pixel is greater than d h , then the pixel is a strong edge pixel and is retained;

[0028] Step 2.3.4: If the gradient magnitude of the pixel is less than d l , then the pixel is a non-edge pixel and is discarded;

[0029] Step 2.3.5: If the gradient magnitude of the pixel is d h ~ dm If there are strong edge pixels in the 8-neighborhood of the pixel, the pixel is searched. If there are one or more strong edges, the pixel is determined to be a strong edge pixel and is retained; otherwise, it is determined to be a weak edge pixel and is retained.

[0030] Step 2.3.6: If the gradient amplitude of the pixel is d m ~d l If there are one or more strong edge pixels or two or more weak edge pixels, the pixel is determined to be a weak edge pixel and is retained; otherwise, it is a non-edge pixel and is discarded.

[0031] Furthermore, in the above method, step 2.4: determining the final edge of the apparent disease by connecting the strong edge pixels and the weak edge pixels, comprises:

[0032] The weak edge pixels connected to the strong edge pixels are retained; if there are no other strong edge pixels or weak edge pixels in the 8-neighborhood of the weak edge pixel, the weak edge pixel is determined to be an isolated pixel, and the isolated weak edge pixel is discarded; the final edge features of the apparent disease are determined by connecting the currently retained strong edge pixels and weak edge pixels.

[0033] Furthermore, in the above method, step 3: based on the final edge features of the apparent disease, correcting the positioning deviation and scale deviation of the detection frame in the image after the edge is refined, includes:

[0034] Step 3.1: Based on the positioning deviation of the detection frame in the image after edge refinement, a buffer is constructed according to the number of edge points on the boundary of the detection frame to adjust the position of the detection frame;

[0035] Step 3.2: To address the problem of mismatch between detection box and target scale, especially the problem of large-scale targets being cut by small-size target prediction boxes, a single object association and merging strategy is constructed.

[0036] Furthermore, in the above method, step 3.1: for the positioning deviation of the detection frame in the image after the edge is refined, a buffer is constructed according to the number of edge points on the boundary of the detection frame, and the position of the detection frame is adjusted, including:

[0037] Step 3.1.1: The final edge feature of the apparent disease is composed of continuous edge points, and the gradient direction pointing to the outside of the final edge is taken as the gradient direction of the edge point;

[0038] Step 3.1.2: The possible positioning deviation of the detection frame is divided into four characteristics as follows:

[0039] (a) There are several edge points on a certain side of the detection box, and one or more edge points have only one adjacent edge point;

[0040] (b) There are several edge points on each of two adjacent sides of the detection box, and one or more edge points have only one adjacent edge point;

[0041] (c) There are several edge points on each of two non-adjacent sides of the detection box, and one or more edge points have only one adjacent edge point;

[0042] (d) There are several edge points on each of the three sides of the detection box, and one or more edge points have only one adjacent edge point;

[0043] Step 3.1.3: Determine whether there is an edge point on the edge line of the detection box with only one adjacent edge point; if so, expand it outward once in the direction perpendicular to the edge. j pixels, build a buffer;

[0044] Step 3.1.4: According to the method in step 2, the edge features of the apparent disease in the buffer zone are extracted again;

[0045] Step 3.1.5: Based on the edge features of the apparent disease in the extracted buffer zone, and according to the methods of steps 3.1.2 and 3.1.3, the positional relationship between the edge points of the buffer zone and the apparent disease is determined again, and the edge points of the buffer zone are adjusted;

[0046] Step 3.1.6: Repeat steps 3.1.3 to 3.1.5 until the edge points of the apparent disease in the detection frame form a complete closed outline.

[0047] Furthermore, in the above method, step 3.2: for the problem that the detection frame does not match the target scale, especially the problem that the large-scale target is cut by the small-scale target prediction frame, a single object association and merging strategy is constructed, including:

[0048] Step 3.2.1: For two overlapping detection frames anchor_l and anchor_s of the same apparent disease type, if (anchor_l - anchor_s) / anchor_s > K%, then directly merge the detection frames anchor_s and anchor_l; where K% represents the degree of overlap;

[0049] Step 3.2.2: For two adjacent detection frames of the same apparent disease type, based on steps 3.1.1 to 3.1.3, respectively construct buffer zones for the two detection frames; if the two detection frames intersect after the buffer zones are constructed, it is considered that the apparent diseases in the two detection frames are the same object, and after the two detection frames are merged across the frames, the edges of the apparent diseases are further extracted based on steps 3.1.4 to 3.1.6; otherwise, based on steps 3.1.4 to 3.1.6, only the edges of the apparent diseases in the two corrected detection frames are extracted.

[0050] According to another aspect of the present invention, there is further provided a computer-readable storage medium having computer-executable instructions stored thereon, wherein when the computer-executable instructions are executed by a processor, the processor is caused to: adopt any of the methods described above.

[0051] According to another aspect of the present invention, there is also provided a computer device, comprising:

[0052] Processor; and

[0053] A memory arranged to store computer executable instructions which, when executed, cause the processor to: adopt any of the methods described above.

[0054] Compared with the prior art, in order to solve a series of problems in the target detection frame area, the present invention proposes a method for correcting the detection results of municipal road and bridge apparent disease targets. The method automatically identifies the apparent disease elements of municipal roads and bridges with the help of target detection algorithms, extracts the apparent disease edge information based on computer vision technology, and then builds a buffer zone according to the gradient direction of the edge point to solve the size deviation and positioning deviation of the target detection frame; in addition, by optimizing the gradient direction and threshold segmentation strategy in the boundary detection algorithm, the algorithm's adaptive ability is improved to ensure the accuracy and continuity of edge extraction; a single object association and merging strategy is constructed to ensure the scale consistency of the target object. Finally, an efficient and accurate solution is provided for the target detection of apparent diseases of roads and bridges in municipal work.

[0055] In order to more carefully judge the gradient direction of a pixel point and its relationship with neighboring pixels and adapt to the diversity of gradient directions of apparent disease edges, the present invention divides the gradient directions into the following categories: Figure 2The six intervals shown in the figure are as follows; the present invention adopts an improved three-threshold segmentation detection method to improve the adaptive ability of the algorithm and ensure the accuracy and continuity of edge extraction; the present invention constructs a buffer zone by the number of edge points on the boundary of the detection frame, adjusts the position of the detection frame, and realizes the positioning deviation correction of the detection frame; the present invention proposes a single object association and merging strategy to solve the problem that the detection frame does not match the target scale, especially the problem that a large-scale target is cut by a small-size target prediction frame; the present invention can quickly and accurately determine the geometric information of the apparent defects of municipal roads and bridges, and effectively guide the bridge maintenance work. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 It is a flow chart of a method for correcting the detection results of apparent defects of municipal roads and bridges according to an embodiment of the present invention;

[0057] Figure 2 is a schematic diagram of an improved gradient direction division interval according to an embodiment of the present invention;

[0058] Figure 3 is a first schematic diagram of determining the extension direction of a buffer zone according to an embodiment of the present invention;

[0059] Figure 4 is a second schematic diagram of determining the extension direction of a buffer zone according to an embodiment of the present invention;

[0060] Figure 5 is a third schematic diagram of determining the extension direction of a buffer zone according to an embodiment of the present invention;

[0061] Figure 6 A fourth schematic diagram of determining the extension direction of a buffer zone according to an embodiment of the present invention;

[0062] Figure 7a A schematic diagram of the apparent disease contour result of the original target detection according to an embodiment of the present invention;

[0063] Figure 7b It is a schematic diagram of a corrected and complete apparent disease result according to an embodiment of the present invention. DETAILED DESCRIPTION

[0064] The present invention is further described in detail below in conjunction with the accompanying drawings.

[0065] In a typical configuration of the present application, the terminal, the device of the service network and the trusted party all include one or more processors (CPU), input / output interfaces, network interfaces and memories.

[0066] Memory may include non-permanent storage in a computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0067] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include non-transitory media such as modulated data signals and carrier waves.

[0068] like Figure 1 As shown, the present invention provides a method for correcting the detection results of apparent defects of municipal roads and bridges, comprising:

[0069] Step 1: Collect the apparent disease image and determine the apparent disease type of the apparent disease image;

[0070] Step 1.1: Collect apparent disease data;

[0071] Among them, the apparent disease area of ​​the object is photographed from any angle and position to generate an apparent disease image set P= { Pic_1, Pic_2, Pic_3,…, Pic_n}, n The serial number of the apparent disease image;

[0072] For example, photograph the diseased area on the surface of an object from any angle and position to generate a collection of apparent disease images P= { Pic_ 1, Pic_2, Pic_3,…, Pic_56};

[0073] Step 1.2: Define the apparent disease type;

[0074] Among them, the visual features of various types of apparent diseases are recorded, and the apparent disease types are defined based on the visual features to obtain the apparent disease type set. C= { Class_1, Class_2, Class_3, …, Class_i}, i A serial number indicating the apparent disease type;

[0075] For example, record the visual features of various types of apparent diseases, define the apparent disease types, and obtain the apparent disease type set C={Concrete Crack, Spallation, Efflorescence, Exposed Bars, Corrosion Stain};

[0076] Step 1.3: Perform target detection on the images in the apparent disease image set P according to the preset image target detection model; wherein the preset image target model is used to automatically identify the defined apparent disease type, and the output is an image with a detection frame mark, the apparent disease type of the detection frame, and the geometric information of the detection frame;

[0077] Step 2: Extract edge features of the apparent disease based on the image marked with the detection frame, the apparent disease type of the detection frame, and the geometric information of the detection frame;

[0078] Step 2.1: Convert the image marked with the detection frame into a grayscale image, and perform Gaussian filtering, image smoothing and noise removal on the grayscale image to obtain a processed image;

[0079] Step 2.2: Calculate the gradient of the processed image using a differential template of size m*m, obtain the gradient magnitude and gradient direction of each pixel in the processed image, and perform non-maximum suppression on the gradient magnitude to obtain an image with refined edges;

[0080] Here, the image gradient can be calculated using a 2×2 pixel difference template to obtain the gradient magnitude and direction of each pixel, and the gradient magnitude can be non-maximum suppressed to refine the edge;

[0081] Among them, Figure 2 As shown in the figure, the gradient direction is divided into 6 directions to more carefully judge the gradient direction of the pixel point and its relationship with the neighboring pixels, thereby improving the accuracy of edge point detection;

[0082] Step 2.3: Improve the dual threshold detection method, construct a three-threshold segmentation method, and improve the algorithm's adaptability; based on the three-threshold segmentation method and the image after edge refinement, determine the strong edge pixels and weak edge pixels to ensure the accuracy and continuity of edge extraction;

[0083] Step 2.3.1: Based on the image after edge refinement, determine the threshold using the maximum inter-class variance method d ;

[0084] Specifically, assume that the grayscale range of the image after edge refinement is [ 0, L-1 ], the pixels in the image after edge refinement are divided into two categories: the first category has gray levels in [ 0, t ], accounting for the full ratio p 0 , the average gray level of the first type of gray levelu 0 ; The second gray level is [ t, L-1 ], accounting for the full ratio p 1 , the average gray level of the second type of gray level u 1 ;use u represents the total average grayscale of the image, s 2 represents the between-class variance, then:

[0085] (1)

[0086] (2)

[0087] in, L Represents the maximum value of the grayscale range of the image; then traverse all possible thresholds t , t The value of 0 arrive L Integer in ; for each t Calculate the corresponding between-class variance s 2 ;when s 2 When it is maximum, there is an optimal gray level threshold d;

[0088] Step 2.3.2: Set the highest threshold d h = d At the same time, add a medium threshold d m = 0.7 d , low threshold d l = 0.4 d ; The definition of edge points includes: strong edge pixels, weak edge pixels and non-edge pixels;

[0089] Step 2.3.3: If the gradient magnitude of the pixel is greater than d h , then the pixel is a strong edge pixel and is retained;

[0090] Step 2.3.4: If the gradient magnitude of the pixel is less than d l , then the pixel is a non-edge pixel and is discarded;

[0091] Step 2.3.5: If the gradient magnitude of the pixel is d h ~ d m If there are strong edge pixels in the 8-neighborhood of the pixel, the pixel is searched. If there are one or more strong edges, the pixel is determined to be a strong edge pixel and is retained; otherwise, it is determined to be a weak edge pixel and is retained.

[0092] Step 2.3.6: If the gradient amplitude of the pixel is d m ~d l If there are one or more strong edge pixels or two or more weak edge pixels in the 8-neighborhood of the pixel, the pixel is determined to be a weak edge pixel and is retained; otherwise, it is a non-edge pixel and is discarded.

[0093] Step 2.4: Determine the final edge of the apparent disease by connecting the strong edge pixels and the weak edge pixels;

[0094] Specifically, retain the weak edge pixels connected to the strong edge pixels; if there are no other strong edge pixels or weak edge pixels in the 8-neighborhood of the weak edge pixel, the weak edge pixel is determined to be an isolated pixel, and the isolated weak edge pixel is discarded; the final edge features of the apparent disease are determined by connecting the currently retained strong edge pixels and weak edge pixels.

[0095] Step 3: Based on the final edge features of the apparent disease, correct the positioning deviation and scale deviation of the detection frame in the image after the edge is refined;

[0096] Step 3.1: Based on the positioning deviation of the detection frame in the image after edge refinement, a buffer is constructed according to the number of edge points on the boundary of the detection frame to adjust the position of the detection frame;

[0097] Step 3.1.1: The final edge feature of the apparent disease is composed of continuous edge points, and the gradient direction pointing to the outside of the final edge is taken as the gradient direction of the edge point;

[0098] Step 3.1.2: The possible positioning deviation of the detection frame is divided into four characteristics as follows:

[0099] (a) If Figure 3 As shown, there are several edge points on a certain side of the detection box, and one or more edge points have only one adjacent edge point;

[0100] (b) If Figure 4As shown, there are several edge points on each of two adjacent sides of the detection frame, and there are one or more edge points with only one adjacent edge point;

[0101] (c) If Figure 5 As shown, there are several edge points on each of two non-adjacent sides of the detection frame, and there are one or more edge points with only one adjacent edge point;

[0102] (d) If Figure 6 As shown, there are several edge points on each of the three sides of the detection box, and there are one or more edge points with only one adjacent edge point;

[0103] Step 3.1.3: Determine whether there is an edge point on the edge line of the detection box with only one adjacent edge point; if so, expand it outward once in the direction perpendicular to the edge. j pixels, build a buffer;

[0104] Step 3.1.4: According to the method in step 2, the edge features of the apparent disease in the buffer zone are extracted again;

[0105] Step 3.1.5: Based on the edge features of the apparent disease in the extracted buffer zone, and according to the methods of steps 3.1.2 and 3.1.3, the positional relationship between the edge points of the buffer zone and the apparent disease is determined again, and the edge points of the buffer zone are adjusted;

[0106] Step 3.1.6: Repeat steps 3.1.3 to 3.1.5 until the edge points of the apparent disease in the detection frame form a complete closed outline, such as Figure 7a and 7b As shown;

[0107] Step 3.2: To address the problem of mismatch between detection box and target scale, especially the problem of large-scale targets being cut by small-scale target prediction boxes, a single object association and merging strategy is constructed;

[0108] Step 3.2.1: For two overlapping detection frames anchor_l and anchor_s of the same apparent disease type, if (anchor_l - anchor_s) / anchor_s > K%, then directly merge the detection frames anchor_s and anchor_l; where K% represents the degree of overlap;

[0109] Specifically, for two overlapping detection frames of the same type, anchor_l and anchor_s, if (anchor_l - anchor_s) / anchor_s > 45%, then the detection frames anchor_s and anchor_l are directly merged;

[0110] Step 3.2.2: For two adjacent detection frames of the same apparent disease type, based on steps 3.1.1 to 3.1.3, respectively construct buffer zones for the two detection frames; if the two detection frames intersect after the buffer zones are constructed, it is considered that the apparent diseases in the two detection frames are the same object, and after the two detection frames are merged across the frames, the edges of the apparent diseases are further extracted based on steps 3.1.4 to 3.1.6; otherwise, based on steps 3.1.4 to 3.1.6, only the edges of the apparent diseases in the two corrected detection frames are extracted.

[0111] In summary, in order to solve a series of problems in the target detection frame area mentioned above, the present invention proposes a method for correcting the detection results of municipal road and bridge apparent disease targets. This method automatically identifies the apparent disease elements of municipal roads and bridges with the help of target detection algorithms, extracts the apparent disease edge information based on computer vision technology, and then builds a buffer zone according to the gradient direction of the edge point to solve the size deviation and positioning deviation of the target detection frame; in addition, by optimizing the gradient direction and threshold segmentation strategy in the boundary detection algorithm, the algorithm's adaptive ability is improved to ensure the accuracy and continuity of edge extraction; a single object association and merging strategy is constructed to ensure the scale consistency of the target object. Finally, an efficient and accurate solution is provided for the target detection of apparent diseases of roads and bridges in municipal work.

[0112] In order to more carefully judge the gradient direction of a pixel point and its relationship with neighboring pixels and adapt to the diversity of gradient directions of apparent disease edges, the present invention divides the gradient directions into the following categories: Figure 2 The six intervals shown in the figure are as follows; the present invention adopts an improved three-threshold segmentation detection method to improve the adaptive ability of the algorithm and ensure the accuracy and continuity of edge extraction; the present invention constructs a buffer zone by the number of edge points on the boundary of the detection frame, adjusts the position of the detection frame, and realizes the positioning deviation correction of the detection frame; the present invention proposes a single object association and merging strategy to solve the problem that the detection frame does not match the target scale, especially the problem that a large-scale target is cut by a small-size target prediction frame; the present invention can quickly and accurately determine the geometric information of the apparent defects of municipal roads and bridges, and effectively guide the bridge maintenance work.

[0113] According to another aspect of the present invention, there is further provided a computer-readable storage medium having computer-executable instructions stored thereon, wherein when the computer-executable instructions are executed by a processor, the processor is caused to: adopt any of the methods described above.

[0114] According to another aspect of the present invention, there is also provided a computer device, comprising:

[0115] Processor; and

[0116] A memory arranged to store computer executable instructions which, when executed, cause the processor to: adopt any of the methods described above.

[0117] The detailed contents of the device embodiments of the present invention can be found in the corresponding parts of the method embodiments, which will not be described in detail here.

[0118] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

[0119] It should be noted that the present invention can be implemented in software and / or a combination of software and hardware, for example, it can be implemented using an application-specific integrated circuit (ASIC), a general-purpose computer or any other similar hardware device. In one embodiment, the software program of the present invention can be executed by a processor to implement the steps or functions described above. Similarly, the software program of the present invention (including related data structures) can be stored in a computer-readable recording medium, for example, a RAM memory, a magnetic or optical drive or a floppy disk and the like. In addition, some steps or functions of the present invention can be implemented using hardware, for example, as a circuit that cooperates with a processor to perform various steps or functions.

[0120] In addition, a part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. The program instruction for calling the method of the present invention may be stored in a fixed or removable recording medium, and / or transmitted through a data stream in a broadcast or other signal-bearing medium, and / or stored in a working memory of a computer device that runs according to the program instruction. Here, according to an embodiment of the present invention, a device is included, the device including a memory for storing computer program instructions and a processor for executing program instructions, wherein, when the computer program instruction is executed by the processor, the device is triggered to run the method and / or technical solution based on the aforementioned multiple embodiments of the present invention.

[0121] It is obvious to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential features of the present invention. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-restrictive, and the scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any figure mark in the claims should not be regarded as limiting the claims involved. In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the device claim can also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any particular order.

Claims

1. A method for correcting the detection results of apparent defects of municipal roads and bridges, characterized in that: include: Step 1: Collect the apparent disease image and determine the apparent disease type of the apparent disease image; Step 2: Extract edge features of the apparent disease based on the image marked with the detection frame, the apparent disease type of the detection frame, and the geometric information of the detection frame; Step 3: Based on the final edge features of the apparent disease, correct the positioning deviation and scale deviation of the detection frame in the image after the edge is refined; Step 2: Based on the image with the detection frame mark, the apparent disease type of the detection frame and the geometric information of the detection frame, the edge features of the apparent disease are extracted, including: Step 2.1: Convert the image marked with the detection frame into a grayscale image, and perform Gaussian filtering, image smoothing and noise removal on the grayscale image to obtain a processed image; Step 2.2: Calculate the gradient of the processed image using a differential template of size m*m, obtain the gradient magnitude and gradient direction of each pixel in the processed image, and perform non-maximum suppression on the gradient magnitude to obtain an image with refined edges; Step 2.3: Construct a three-threshold segmentation method; based on the three-threshold segmentation method and the image after edge refinement, determine the strong edge pixels and the weak edge pixels; Step 2.4: Determine the final edge of the apparent disease by connecting the strong edge pixels and the weak edge pixels; Step 2.3: Construct a three-threshold segmentation method; based on the three-threshold segmentation method and the image after edge refinement, determine the strong edge pixels and the weak edge pixels, including: Step 2.3.1: Based on the image after edge refinement, determine the threshold δ by the maximum inter-class variance method; Assume that the grayscale range of the image after edge refinement is [0, L-1], and divide the pixels in the image after edge refinement into two categories: the first category has a grayscale between [0, t], accounting for a full-scale ratio of p0, and an average grayscale of the first category u0; the second category has a grayscale between [t, L-1], accounting for a full-scale ratio of p1, and an average grayscale of the second category u1; Use u to represent the total average grayscale of the image, σ 2 represents the between-class variance, then: u=p0u0+p1u1 (1) σ 2 =p0(u0-u) 2 +p1(u1-u) 2 (2) Among them, L represents the maximum value of the grayscale range of the image; then traverse all possible thresholds t, where t is an integer from 0 to L; for each t, ​​calculate the corresponding inter-class variance σ 2 ; When σ 2 When it is maximum, there is an optimal gray level threshold δ; Step 2.3.2: Set the highest threshold δ h =δ, at the same time, add a middle threshold δ m =0.7δ, low threshold δ l =0.4δ; the definition of edge points includes: strong edge pixels, weak edge pixels and non-edge pixels; Step 2.3.3: If the gradient magnitude of the pixel is greater than δ h , then the pixel is a strong edge pixel and is retained; Step 2.3.4: If the gradient magnitude of the pixel is less than δ l , then the pixel is a non-edge pixel and is discarded; Step 2.3.5: If the gradient magnitude of the pixel is within δ h ~δ m If there are strong edge pixels in the 8-neighborhood of the pixel, the pixel is searched. If there are one or more strong edges, the pixel is determined to be a strong edge pixel and is retained; otherwise, it is determined to be a weak edge pixel and is retained. Step 2.3.6: If the gradient amplitude of the pixel is within δ m ~δ l If there are one or more strong edge pixels or two or more weak edge pixels in the 8-neighborhood of the pixel, the pixel is determined to be a weak edge pixel and is retained; otherwise, it is a non-edge pixel and is discarded. Step 2.4: Determine the final edge of the apparent disease by connecting strong edge pixels and weak edge pixels, including: Keep the weak edge pixels connected to the strong edge pixels; if there are no other strong edge pixels or weak edge pixels in the 8-neighborhood of the weak edge pixel, the weak edge pixel is determined to be an isolated pixel, and the isolated weak edge pixel is discarded; the final edge features of the apparent disease are determined by connecting the currently retained strong edge pixels and weak edge pixels; Step 3: Based on the final edge features of the apparent disease, correct the positioning deviation and scale deviation of the detection frame in the image after the edge is refined, including: Step 3.1: Based on the positioning deviation of the detection frame in the image after edge refinement, a buffer is constructed according to the number of edge points on the boundary of the detection frame to adjust the position of the detection frame; Step 3.2: To solve the problem that the detection box does not match the target scale and the large-scale target is cut by the small-scale target prediction box, a single object association and merging strategy is constructed; Step 3.2: To solve the problem that the detection box does not match the target scale and the large-scale target is cut by the small-scale target prediction box, a single object association and merging strategy is constructed, including: Step 3.2.1: For two overlapping detection frames anchor_l and anchor_s of the same apparent disease type, if (anchor_l-anchor_s) / anchor_s>K%, then directly merge the detection frames anchor_s and anchor_l; where K% represents the degree of overlap; Step 3.2.2: For two adjacent detection frames of the same apparent disease type, based on steps 3.1.1 to 3.1.3, respectively construct buffer zones for the two detection frames; if the two detection frames intersect after the buffer zones are constructed, it is considered that the apparent diseases in the two detection frames are the same object, and after the two detection frames are merged across the frames, the edges of the apparent diseases are further extracted based on steps 3.1.4 to 3.1.6; otherwise, based on steps 3.1.4 to 3.1.6, only the edges of the apparent diseases in the two corrected detection frames are extracted.

2. The method for correcting the detection results of apparent defects of municipal roads and bridges as claimed in claim 1, characterized in that: Step 1: Collect apparent disease images and determine the apparent disease type of the apparent disease images, including: Step 1.1: photograph the apparent disease area of ​​the object from any angle and position to generate each apparent disease image; Step 1.2: Record the visual features of various types of apparent diseases, define the apparent disease types based on the visual features, and obtain various apparent disease types; Step 1.3: Perform target detection on the images in the apparent disease image set P according to a preset image target detection model; wherein the preset image target model is used to automatically identify the defined apparent disease type.

3. The method for correcting the detection results of apparent defects of municipal roads and bridges as claimed in claim 1, characterized in that: Step 3.1: Based on the positioning deviation of the detection frame in the image after edge refinement, a buffer is constructed according to the number of edge points on the boundary of the detection frame, and the position of the detection frame is adjusted, including: Step 3.1.1: The final edge feature of the apparent disease is composed of continuous edge points, and the gradient direction pointing to the outside of the final edge is taken as the gradient direction of the edge point; Step 3.1.2: The possible positioning deviation of the detection frame is divided into four characteristics as follows: (a) There are several edge points on a certain side of the detection box, and one or more edge points have only one adjacent edge point; (b) There are several edge points on each of two adjacent sides of the detection box, and one or more edge points have only one adjacent edge point; (c) There are several edge points on each of two non-adjacent sides of the detection box, and one or more edge points have only one adjacent edge point; (d) There are several edge points on each of the three sides of the detection box, and one or more edge points have only one adjacent edge point; Step 3.1.3: Determine whether there is an edge point on the edge line of the detection box that has only one adjacent edge point; If yes, then expand outward by j pixels at a time in the direction perpendicular to the edge to construct a buffer zone; Step 3.1.4: According to the method in step 2, the edge features of the apparent disease in the buffer zone are extracted again; Step 3.1.5: Based on the edge features of the apparent disease in the extracted buffer zone, and according to the methods of steps 3.1.2 and 3.1.3, the positional relationship between the edge points of the buffer zone and the apparent disease is determined again, and the edge points of the buffer zone are adjusted; Step 3.1.6: Repeat steps 3.1.3 to 3.1.5 until the edge points of the apparent disease in the detection frame form a complete closed outline.

4. A computer-readable storage medium having computer-executable instructions stored thereon, wherein: When the computer executable instructions are executed by a processor, the processor is caused to: adopt the method according to any one of claims 1 to 3.

5. A computer device, wherein: include: processor; as well as A memory arranged to store computer executable instructions which, when executed, cause the processor to: employ a method as claimed in any one of claims 1 to 3.

Images