Crack detection method and device based on prediction frame and computer equipment
By using a pre-trained crack detection network and a confidence update algorithm, target prediction boxes are selected, which solves the problem of low crack detection accuracy in existing technologies and achieves higher accuracy crack detection and accurate crack rate calculation.
Patent Information
- Application Number
- CN202211135957.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-09-19
- Publication Date
- 2026-01-27
- Estimated Expiration
- 2042-09-19
AI Technical Summary
Existing crack detection technologies are not very accurate in detecting cracks on engineering structures such as asphalt pavements, and it is difficult to effectively identify and distinguish the same crack detected by multiple prediction frames.
A pre-trained crack detection network is used for initial bounding box detection. The confidence parameter is updated by calculating the overlap area between the initial bounding box and the candidate bounding boxes, and the target bounding box is selected. Redundant bounding boxes are filtered out using a non-maximum suppression algorithm and a preset confidence threshold.
It improves the accuracy of crack detection and crack rate calculation, reduces the number of cases where multiple prediction boxes detect the same crack simultaneously, and enhances detection accuracy.
Smart Images
Figure CN115546119B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of artificial intelligence technology, and in particular to a crack detection method, apparatus, computer device, storage medium, and computer program product based on a prediction box. Background Technology
[0002] Crack detection, such as on the surfaces of asphalt pavements, bridges, and other building structures, is a crucial aspect of preventative maintenance of engineering structures. For example, cracks in asphalt pavements not only affect driving safety and comfort but can also lead to decreased pavement stability and other secondary damage due to water seepage. Therefore, timely maintenance and repair of early-stage cracks in asphalt pavements can extend their service life. Regular inspection and evaluation of the surfaces of engineering structures such as asphalt pavements provide a basis for timely maintenance and repair of cracks and other defects, and are of great significance for maintaining the health of engineering structure surfaces.
[0003] With the development of deep learning technology, object detection algorithms have emerged, which can be used to detect cracks on the surface of engineering structures such as asphalt pavement. However, the accuracy of crack detection is not high. Summary of the Invention
[0004] Therefore, it is necessary to provide a crack detection method, apparatus, computer equipment, computer-readable storage medium, and computer program product based on a prediction box that can improve the accuracy of crack detection, addressing the aforementioned technical problems.
[0005] Firstly, this application provides a crack detection method based on a predicted bounding box. The method includes:
[0006] Acquire the target image of the object to be detected;
[0007] A pre-trained crack detection network is used to detect cracks in the target image, resulting in multiple initial prediction boxes containing prediction confidence and location information.
[0008] For each initial prediction box, a candidate prediction box with a prediction confidence greater than that of the initial prediction box is determined in each initial prediction box. Based on the overlap area between the candidate prediction box and the initial prediction box, a confidence update parameter corresponding to the initial prediction box is determined, and the prediction confidence of the initial prediction box is updated based on the confidence update parameter to obtain the target confidence of the initial prediction box.
[0009] Based on the target confidence level and preset confidence threshold corresponding to each initial prediction box, the initial prediction boxes are filtered to obtain target prediction boxes containing cracks.
[0010] In one embodiment, determining the confidence update parameter corresponding to the initial prediction box based on the overlap area between the candidate prediction box and the initial prediction box includes:
[0011] For each candidate prediction box, the overlap area between the candidate prediction box and the initial prediction box is calculated. Based on the overlap area and the area of the initial prediction box, the overlap area ratio is calculated, and the non-overlap area ratio corresponding to the candidate prediction box is determined based on the overlap area ratio.
[0012] Calculate the first product of the non-overlapping area ratios corresponding to each candidate prediction box, and determine the confidence update parameters corresponding to the initial prediction box based on the first product.
[0013] In one embodiment, determining the confidence update parameters corresponding to the initial prediction box based on the first product includes:
[0014] Based on the first product and the preset mapping function, the confidence update parameters corresponding to the initial prediction box are calculated.
[0015] In one embodiment, updating the prediction confidence of the initial prediction box based on the confidence update parameter to obtain the target confidence corresponding to the initial prediction box includes:
[0016] Calculate the second product of the confidence update parameter and the predicted confidence of the initial prediction box, and use the second product as the target confidence of the initial prediction box.
[0017] In one embodiment, the pre-trained crack detection network is used to detect cracks in the target image, resulting in multiple initial prediction boxes containing prediction confidence and location information, including:
[0018] A pre-trained crack detection network is used to detect cracks in the target image, resulting in multiple first prediction boxes containing prediction confidence and location information.
[0019] The first prediction box is filtered using a non-maximum suppression algorithm to obtain an initial prediction box.
[0020] In one embodiment, the training process of the crack detection network includes:
[0021] Obtain a sample image containing the crack;
[0022] Cracks are annotated in the sample images to obtain a sample dataset containing the bounding boxes;
[0023] The initial target detection network is trained based on the sample dataset to obtain the crack detection network.
[0024] Secondly, this application also provides a crack detection device. The device includes:
[0025] The first acquisition module is used to acquire the target image of the object to be detected;
[0026] The detection module is used to perform crack detection on the target image using a pre-trained crack detection network to obtain multiple initial prediction boxes containing prediction confidence and location information.
[0027] The update module is configured to, for each initial prediction box, determine candidate prediction boxes whose prediction confidence is greater than that of the initial prediction box, determine confidence update parameters corresponding to the initial prediction box based on the overlap area between the candidate prediction boxes and the initial prediction boxes, and update the prediction confidence of the initial prediction boxes based on the confidence update parameters to obtain the target confidence of the initial prediction boxes.
[0028] The filtering module is used to filter the initial prediction boxes according to the target confidence level and the preset confidence threshold corresponding to each initial prediction box, so as to obtain the target prediction boxes containing cracks.
[0029] In one embodiment, the update module is specifically used for:
[0030] For each candidate prediction box, the overlap area between the candidate prediction box and the initial prediction box is calculated. Based on the overlap area and the area of the initial prediction box, the overlap area ratio is calculated, and the non-overlapping area ratio corresponding to the candidate prediction box is determined based on the overlap area ratio. The first product of the non-overlapping area ratios corresponding to each candidate prediction box is calculated, and the confidence update parameter corresponding to the initial prediction box is determined based on the first product.
[0031] In one embodiment, the update module is specifically used for:
[0032] Based on the first product and the preset mapping function, the confidence update parameters corresponding to the initial prediction box are calculated.
[0033] In one embodiment, the update module is specifically used for:
[0034] Calculate the second product of the confidence update parameter and the predicted confidence of the initial prediction box, and use the second product as the target confidence of the initial prediction box.
[0035] In one embodiment, the detection module is specifically used for:
[0036] A pre-trained crack detection network is used to detect cracks in the target image, resulting in multiple first prediction boxes containing prediction confidence and location information. A non-maximum suppression algorithm is then used to filter the first prediction boxes to obtain initial prediction boxes.
[0037] In one embodiment, the device further includes:
[0038] The second acquisition module is used to acquire sample images containing cracks;
[0039] The annotation module is used to annotate the cracks in the sample image to obtain a sample dataset containing the bounding boxes;
[0040] The training module is used to train the initial target detection network based on the sample dataset to obtain the crack detection network.
[0041] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method described in the first aspect.
[0042] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the method described in the first aspect.
[0043] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.
[0044] The aforementioned crack detection method, apparatus, computer equipment, storage medium, and computer program products based on predicted bounding boxes detect cracks in the target image of the object to be detected using a pre-trained crack detection network, obtaining multiple initial predicted bounding boxes containing prediction confidence and location information. Then, based on the overlap area between the initial predicted bounding boxes and candidate predicted bounding boxes, a confidence update parameter is calculated for each initial predicted bounding box. The prediction confidence of each initial predicted bounding box is then updated based on the confidence update parameter to obtain a target confidence. Finally, target predicted bounding boxes containing cracks are selected based on the target confidence. Here, a candidate predicted bounding box refers to an initial predicted bounding box whose prediction confidence is greater than that of the initial predicted bounding box itself. In this method, the target confidence is obtained by updating the prediction confidence based on the overlap area between the candidate predicted bounding box and the initial predicted bounding box. Filtering the initial predicted bounding boxes based on the target confidence and a pre-set confidence threshold can filter redundant predicted bounding boxes. The resulting target predicted bounding boxes can suppress or reduce the situation where multiple target predicted bounding boxes simultaneously detect the same crack, thereby improving the accuracy of crack detection. Attached Figure Description
[0045] Figure 1 This is a flowchart illustrating a crack detection method based on a prediction box in one embodiment;
[0046] Figure 2a This is a schematic diagram of the target image in an example;
[0047] Figure 2b This is a schematic diagram of the initial prediction bounding box obtained after crack detection on a target image in an example.
[0048] Figure 2c This is a schematic diagram of the target prediction boxes obtained after filtering the initial prediction boxes in an example.
[0049] Figure 3 This is a flowchart illustrating the process of determining confidence update parameters in one embodiment;
[0050] Figure 4 This is a schematic diagram of the initial prediction box obtained in another example;
[0051] Figure 5 This is a flowchart illustrating the training process of a crack detection network in one embodiment.
[0052] Figure 6 This is a structural block diagram of a crack detection device based on a prediction box in one embodiment;
[0053] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation
[0054] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0055] First, before introducing the technical solutions of the embodiments of this application, the technical background or evolution of the embodiments of this application is first introduced. Regular crack detection and evaluation of the surface of engineering structures such as asphalt pavements for timely maintenance and repair is of great significance for maintaining the health of the surface of engineering structures. With the development of deep learning technology, object detection algorithms have emerged and can be used to detect cracks on the surface of engineering structures such as asphalt pavements. However, due to the topological structure of cracks, they differ significantly from common target detection objects (such as people, animals, vehicles, etc.). For example, cracks have more typical linear features, and the overlapping and occlusion between cracks manifests as crack connectivity, etc. Therefore, the accuracy of crack detection using current object detection algorithms is not high. Based on this background, the applicant, through long-term research and development and experimental verification, proposes the crack detection method based on prediction boxes of this application, which can improve the accuracy of crack detection and thus improve the accuracy of crack rate calculation. Furthermore, it should be noted that the applicant has devoted a great deal of creative effort to the discovery of the technical problems of this application and the technical solutions described in the following embodiments.
[0056] In one embodiment, such as Figure 1 As shown, a crack detection method based on a predicted bounding box is provided. This embodiment illustrates the application of this method to a terminal. It is understood that this method can also be applied to a server, or to a system including both a terminal and a server, and is implemented through interaction between the terminal and the server. The terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. The server can be a standalone server or a server cluster consisting of multiple servers. In this embodiment, the method includes the following steps:
[0057] Step 101: Obtain the target image of the object to be detected.
[0058] The object to be inspected refers to the object that needs to be crack detected, which can be an asphalt road, concrete road, bridge, or other building structure. The target image of the object to be inspected is generally an image obtained by acquiring the surface of the aforementioned engineering structure. The image viewpoint can be a top-down viewpoint; if the image is from another viewpoint, relevant algorithms can be used to process the image to obtain a top-down viewpoint image.
[0059] In implementation, the terminal can use a camera device (such as a webcam or a detection device equipped with a camera) to capture an image of the object to be detected, thus obtaining a target image. In one example, the target image is as follows: Figure 2a As shown, this is a digital image of an asphalt pavement, containing four cracks: a, b, c, and d.
[0060] Step 102: A pre-trained crack detection network is used to detect cracks in the target image, resulting in multiple initial prediction boxes containing prediction confidence and location information.
[0061] In implementation, the terminal can use a pre-trained crack detection network to detect cracks in the target image. The crack detection network is built and trained based on an object detection algorithm. For example, training sample images with bounding boxes labeled with crack locations can be used to train an initial network built based on an object detection algorithm (such as YOLO-v5) to obtain a crack detection network with optimal network parameters (such as weights). By using the crack detection network to detect cracks in the target image, multiple initial prediction boxes containing prediction confidence and location information can be obtained. In one example, the crack detection network is used to detect cracks in an image such as... Figure 2a After crack detection is performed on the target image shown, the following results can be obtained: Figure 2b The eight initial prediction boxes shown are Box1 to Box8. It can be seen that the initial prediction box Box1 misdetects cracks a and b as one crack. Therefore, crack a is detected by both initial prediction boxes Box1 and Box2, and crack b is detected by both initial prediction boxes Box1 and Box3. That is, there is a situation where the same crack is detected repeatedly by multiple prediction boxes.
[0062] Step 103: For each initial prediction box, determine candidate prediction boxes whose prediction confidence is greater than that of the initial prediction box. Based on the overlap area between the candidate prediction boxes and the initial prediction boxes, determine the confidence update parameter corresponding to the initial prediction box. Update the prediction confidence of the initial prediction box based on the confidence update parameter to obtain the target confidence of the initial prediction box.
[0063] In implementation, for each initial prediction box obtained in step 102, the terminal selects the prediction boxes with a prediction confidence greater than that of the initial prediction box as candidate prediction boxes. Specifically, the terminal can sort the initial prediction boxes according to their prediction confidence, such as sorting them from highest to lowest prediction confidence. Then, for each initial prediction box, the terminal can select the initial prediction boxes ranked before it as candidate prediction boxes.
[0064] Then, the terminal can determine the confidence update parameter corresponding to the initial prediction box based on the overlap area between the candidate prediction boxes and the initial prediction box. The confidence update parameter is used to update the prediction confidence of the initial prediction box, and it is related to the overlap area between the candidate prediction boxes and the initial prediction box. For example, the confidence update parameter can be negatively correlated with the aforementioned overlap area; that is, the larger the overlap area between the candidate prediction boxes and the initial prediction box, the smaller the confidence update parameter. Then, the terminal can update the prediction confidence of the initial prediction box based on the confidence update parameter to obtain the target confidence corresponding to the initial prediction box. For example, the confidence update parameter can be multiplied by the prediction confidence to obtain the target confidence.
[0065] Step 104: Based on the target confidence level and preset confidence threshold corresponding to each initial prediction box, the initial prediction boxes are filtered to obtain target prediction boxes containing cracks.
[0066] In implementation, after the terminal obtains the target confidence level corresponding to each initial prediction box according to step 103, it can compare each target confidence level with a preset confidence threshold to filter the initial prediction boxes. Initial prediction boxes that meet the preset confidence threshold condition are selected as target prediction boxes containing cracks. The preset confidence threshold can be pre-set as needed; initial prediction boxes with a target confidence level greater than or equal to the preset confidence threshold can be used as target prediction boxes. In one example, this method is used to... Figure 2b After filtering the 8 initial prediction boxes shown, the following can be obtained: Figure 2c The four target prediction boxes shown are Box2 for detecting crack a, Box3 for detecting crack b, Box7 for detecting crack c, and Box5 for detecting crack d. This filters out redundant prediction boxes that repeatedly detect the same crack, improving crack detection accuracy. Furthermore, the crack rate can be calculated based on this detection result, yielding a crack rate with high accuracy.
[0067] In the aforementioned crack detection method based on prediction boxes, a pre-trained crack detection network is used to detect cracks in the target image of the object to be detected, resulting in multiple initial prediction boxes containing prediction confidence and location information. Then, based on the overlap area between the initial prediction boxes and candidate prediction boxes, a confidence update parameter is calculated for each initial prediction box. The prediction confidence of each initial prediction box is then updated based on this update parameter to obtain a target confidence. Finally, target prediction boxes containing cracks are selected based on the target confidence. Here, a candidate prediction box refers to an initial prediction box whose prediction confidence is greater than the initial prediction box's own prediction confidence. In this method, the target confidence is obtained by updating the prediction confidence based on the overlap area between the candidate prediction box and the initial prediction box. Filtering the initial prediction boxes based on the target confidence and a pre-set confidence threshold can filter redundant prediction boxes, and the resulting target prediction boxes can suppress or reduce the situation where multiple target prediction boxes simultaneously detect the same crack. This method considers the topological structure of cracks and filters redundant prediction boxes, thereby improving the accuracy of crack detection. Furthermore, calculating the crack rate based on the target prediction boxes obtained using this method can further improve the accuracy of crack rate calculation.
[0068] In one embodiment, such as Figure 3 As shown, the process of determining the confidence update parameters in step 103 specifically includes the following steps:
[0069] Step 301: For each candidate prediction box, calculate the overlap area between the candidate prediction box and the initial prediction box. Calculate the overlap area ratio based on the overlap area and the area of the initial prediction box, and determine the non-overlapping area ratio corresponding to the candidate prediction box based on the overlap area ratio.
[0070] In implementation, after the terminal determines the candidate prediction boxes corresponding to each initial prediction box, it can calculate the overlap area between the candidate prediction box and the initial prediction box for each candidate prediction box. Then, it calculates the overlap area ratio based on the overlap area and the area of the initial prediction box. Specifically, the terminal can calculate the overlap area based on the position information of the initial prediction box and the candidate prediction boxes, and calculate the area of the initial prediction box based on its position information. In one example, the formula for calculating the overlap area ratio is as follows:
[0071]
[0072] Among them, b m Indicates the initial prediction boxes m and b i Represents candidate prediction box i, IoA(b) m ,b iarea(b) represents the ratio of the overlap area between the initial predicted bounding box m and the candidate predicted bounding box i; m ∩b i ) represents the overlap area between the initial predicted bounding box m and the candidate predicted bounding box i, where area ab m This represents the area of the initial prediction box m.
[0073] Then, the terminal can determine the non-overlapping area ratio corresponding to the candidate prediction box based on the overlap area ratio. The non-overlapping area ratio refers to the proportion of the total area of the initial prediction box, excluding the overlapping area (i.e., the non-overlapping area), to the total area of the initial prediction box. In one example, the formula for calculating the non-overlapping area ratio is as follows:
[0074]
[0075] in, This represents the ratio of the non-overlapping areas of the initial predicted box m and the candidate predicted box i.
[0076] In another implementation, the terminal can calculate the overlap area between the candidate prediction box and the initial prediction box for each candidate prediction box. Then, based on the overlap area and the area of the initial prediction box, it can calculate the non-overlapping area. Finally, based on the non-overlapping area and the area of the initial prediction box, it can calculate the ratio of the non-overlapping area of the candidate prediction box to the area of the initial prediction box, as shown in the following formula:
[0077]
[0078] Step 302: Calculate the first product of the non-overlapping area ratios corresponding to each candidate prediction box, and determine the confidence update parameters corresponding to the initial prediction box based on the first product.
[0079] In implementation, the terminal can multiply the non-overlapping area ratios of each candidate prediction box determined in step 301, and use the resulting product as the confidence update parameter for the initial prediction box. In one example, the formula for calculating the confidence update parameter is as follows:
[0080]
[0081] Where, δ m This represents the confidence update parameter corresponding to the initial predicted box m, and M represents the number of candidate predicted boxes corresponding to the initial predicted box m, which is M. This formula represents the ratio of the non-overlapping area of the initial predicted box m to its corresponding candidate predicted boxes i (i = 1, 2, ..., M). Multiplying them together yields the confidence update parameter δ corresponding to the initial prediction box m. m .
[0082] In this embodiment, the overlap area ratio (IoA) is calculated based on the overlap area between the initial prediction box and each candidate prediction box, and then the non-overlap area ratio is calculated. Then the ratio of each non-overlapping area The product is used as the confidence update parameter for the initial prediction box. Based on this confidence update parameter, the prediction confidence of the initial prediction box is updated. Based on the updated target confidence, the initial prediction box is filtered to filter redundant prediction boxes, thereby suppressing or reducing the simultaneous detection of the same crack by multiple target prediction boxes and improving the accuracy of crack detection.
[0083] In one embodiment, the process of determining the confidence update parameters in step 302 specifically includes the following steps: calculating the confidence update parameters corresponding to the initial prediction box based on the first product and the preset mapping function.
[0084] In implementation, the terminal calculates the product of the non-overlapping area ratios of the initial predicted box m and its corresponding candidate predicted boxes i (which can be denoted as...). After that, the product α can be used as a basis. m And a preset mapping function f(x) is used to calculate the confidence update parameter δ corresponding to the initial prediction box m. m The preset mapping function f(x) is used to map the product α. m Mapping within a reasonable range, such as [0,1] or [0.5,1] or other ranges, yields a more reasonable confidence update parameter. The target confidence obtained based on this confidence update parameter can more effectively filter redundant prediction boxes.
[0085] In one example, the formula for the mapping function f(x) is shown below:
[0086]
[0087] Then according to the product α m Using the mapping function f(x) mentioned above, the confidence update parameter δ corresponding to the initial prediction box m can be calculated. m The calculation formula is as follows:
[0088]
[0089] In this embodiment, a preset mapping function is used to map the non-overlapping area ratios. The product obtained by multiplication is mapped to a preset range to obtain a more reasonable confidence update parameter. Based on the target confidence obtained by the confidence update parameter, redundant prediction boxes can be filtered more effectively. That is, the obtained target prediction boxes can more effectively suppress or reduce the situation where multiple target prediction boxes detect the same crack at the same time, thereby improving the accuracy of crack detection.
[0090] In one embodiment, the process of obtaining the target confidence in step 103 specifically includes the following steps: calculating the second product of the confidence update parameter and the prediction confidence of the initial prediction box, and using the second product as the target confidence corresponding to the initial prediction box.
[0091] In implementation, the terminal calculates the confidence update parameter δ corresponding to the initial prediction box m. m Then, the confidence level can be updated to the parameter δ. m The prediction confidence of the initial prediction box m (which can be denoted as s) m Multiplying these two products yields the target confidence score (which can be denoted as p) corresponding to the initial prediction box m. m The formula is as follows:
[0092] p m =δ m ×s m
[0093] In this embodiment, the confidence level is updated by parameter δ. m The prediction confidence s of the initial prediction box m m The product of p is used as the target confidence level. m This allows for the filtering of target prediction boxes based on target confidence and pre-set confidence thresholds, thereby filtering out redundant prediction boxes and improving the accuracy of crack detection.
[0094] In one embodiment, the process of obtaining the initial prediction box in step 102 specifically includes the following steps: using a pre-trained crack detection network to detect cracks in the target image and obtain multiple first prediction boxes containing prediction confidence and location information; using a non-maximum suppression algorithm to filter the first prediction boxes and obtain the initial prediction boxes.
[0095] In implementation, the terminal can use a pre-trained crack detection network to detect cracks in the target image, obtaining multiple first predicted bounding boxes containing prediction confidence and location information. Then, the terminal can use a non-maximum suppression (NMS) algorithm to filter the first predicted bounding boxes, obtaining initial predicted bounding boxes. The specific process of the NMS algorithm will not be elaborated here. In one example, a crack detection network is used to detect cracks in a target image such as... Figure 2a After crack detection is performed on the target image shown, the following results can be obtained: Figure 2b The eight first predicted bounding boxes shown are then filtered using the NMS algorithm to obtain the following results: Figure 4The five initial prediction boxes are shown. Among them, the area intersection-union ratios (IoU(1,2) = 28.8%) of prediction boxes Box1 and Box2, and the area intersection-union ratios (IoU(1,3) = 33.8%) of prediction boxes Box1 and Box3 are both low, so the NMS algorithm cannot filter out prediction box Box1, which detects cracks a and b as a single crack. Further, this method is used to... Figure 4 The initial prediction boxes shown are filtered (steps 103 and 104) to obtain the following results: Figure 2c The target prediction box shown can be used to filter out prediction box Box1, thereby improving the accuracy of crack detection.
[0096] In this embodiment, for each first predicted bounding box obtained by recognizing the target image using the crack detection network, the terminal can use the NMS algorithm to filter each first predicted bounding box to obtain an initial predicted bounding box, that is, to perform preliminary filtering of redundant predicted bounding boxes. Then, the prediction confidence of each initial predicted bounding box is updated, and based on the updated target confidence, the initial predicted bounding boxes are filtered again (see steps 103 and 104 for details; this method can be called the ARS algorithm) to obtain the target predicted bounding box. Thus, after two filterings, the situation where multiple target predicted bounding boxes detect the same crack at the same time can be suppressed or reduced, thereby improving the accuracy of crack detection.
[0097] In one embodiment, such as Figure 5 As shown, the training process of the crack detection network includes the following steps:
[0098] Step 501: Obtain a sample image containing the crack.
[0099] In practice, sample images containing cracks can be acquired using acquisition devices. The object to which the sample image belongs can be of the same category as the object to be detected. For example, if the object to be detected is an asphalt pavement, an image of an asphalt pavement containing cracks can be acquired as a sample image. The sample image can be captured from a top-down view. If the image is captured from another viewpoint, relevant algorithms can be used to process the image to obtain a top-down view sample image.
[0100] Step 502: Crack annotation is performed on the sample images to obtain a sample dataset containing the bounding boxes.
[0101] In implementation, the terminal can annotate cracks in the sample image to obtain a sample dataset containing bounding boxes. For example, a bounding box-based image annotation algorithm can be used to annotate cracks in the sample image to obtain a sample dataset containing bounding box information.
[0102] Optionally, data augmentation algorithms can be used to augment each sample image containing a bounding box in the sample dataset to obtain new sample data, which can then be used for network training to ensure the richness and diversity of the sample data.
[0103] Step 503: Train the initial target detection network based on the sample dataset to obtain the crack detection network.
[0104] In implementation, the terminal can train an initial object detection network based on a sample dataset containing bounding boxes. Specifically, the terminal can use clustering algorithms (such as k-means) and genetic algorithms to obtain adaptive anchor boxes for cracks based on the bounding box sizes in the sample dataset, which are then used for regression of the predicted detection boxes. The initial object detection network can be a network built based on an object detection algorithm (such as YOLO-v5). Then, the terminal can use the sample dataset containing the adaptive anchor boxes to perform end-to-end distributed training on the initial object detection network to obtain a crack detection network with optimal parameters (such as network weights).
[0105] This embodiment provides an implementation method for training a crack detection network, which can be used to detect cracks in target images of objects to be detected.
[0106] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.
[0107] Based on the same inventive concept, this application also provides a crack detection device based on prediction boxes for implementing the crack detection method based on prediction boxes described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations of one or more crack detection device embodiments based on prediction boxes provided below can be found in the limitations of the crack detection method based on prediction boxes described above, and will not be repeated here.
[0108] In one embodiment, such as Figure 6 As shown, a crack detection device 600 based on a prediction box is provided, including: a first acquisition module 601, a detection module 602, an update module 603, and a filtering module 604, wherein:
[0109] The first acquisition module 601 is used to acquire the target image of the object to be detected.
[0110] The detection module 602 is used to perform crack detection on the target image using a pre-trained crack detection network to obtain multiple initial prediction boxes containing prediction confidence and location information.
[0111] The update module 603 is used to determine, for each initial prediction box, a candidate prediction box whose prediction confidence is greater than that of the initial prediction box, determine the confidence update parameter corresponding to the initial prediction box based on the overlap area between the candidate prediction box and the initial prediction box, and update the prediction confidence of the initial prediction box based on the confidence update parameter to obtain the target confidence of the initial prediction box.
[0112] The filtering module 604 is used to filter the initial prediction boxes according to the target confidence level and the preset confidence threshold corresponding to each initial prediction box, so as to obtain the target prediction box containing the crack.
[0113] In one embodiment, the update module 603 is specifically configured to: for each candidate prediction box, calculate the overlap area between the candidate prediction box and the initial prediction box; calculate the overlap area ratio based on the overlap area and the area of the initial prediction box; determine the non-overlapping area ratio corresponding to the candidate prediction box based on the overlap area ratio; calculate the first product of the non-overlapping area ratios corresponding to each candidate prediction box; and determine the confidence update parameter corresponding to the initial prediction box based on the first product.
[0114] In one embodiment, the update module 603 is specifically used to: calculate the confidence update parameters corresponding to the initial prediction box based on the first product and the preset mapping function.
[0115] In one embodiment, the update module 603 is specifically used to: calculate the second product of the confidence update parameter and the predicted confidence of the initial prediction box, and use the second product as the target confidence corresponding to the initial prediction box.
[0116] In one embodiment, the detection module 602 is specifically used to: use a pre-trained crack detection network to detect cracks in the target image and obtain multiple first prediction boxes containing prediction confidence and location information; use a non-maximum suppression algorithm to filter the first prediction boxes and obtain initial prediction boxes.
[0117] In one embodiment, the device further includes a second acquisition module, an annotation module, and a training module, wherein:
[0118] The second acquisition module is used to acquire sample images containing cracks.
[0119] The annotation module is used to annotate cracks in sample images to obtain a sample dataset containing the bounding boxes.
[0120] The training module is used to train the initial target detection network based on the sample dataset to obtain the crack detection network.
[0121] Each module in the aforementioned crack detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.
[0122] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 7 As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a crack detection method based on a predictive frame. The display screen can be an LCD screen or an e-ink display. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.
[0123] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0124] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.
[0125] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.
[0126] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.
[0127] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.
[0128] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.
[0129] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0130] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. A crack detection method based on a prediction box, characterized in that, The method includes: Acquire the target image of the object to be detected; A pre-trained crack detection network is used to detect cracks in the target image, resulting in multiple initial prediction boxes containing prediction confidence and location information. For each initial prediction box, a candidate prediction box with a prediction confidence greater than that of the initial prediction box is determined in each initial prediction box. Based on the overlap area between the candidate prediction box and the initial prediction box, a confidence update parameter corresponding to the initial prediction box is determined, and the prediction confidence of the initial prediction box is updated based on the confidence update parameter to obtain the target confidence of the initial prediction box. Based on the target confidence level and preset confidence threshold corresponding to each initial prediction box, the initial prediction boxes are filtered to obtain target prediction boxes containing cracks; The step of determining the confidence update parameter corresponding to the initial prediction box based on the overlap area between the candidate prediction box and the initial prediction box includes: for each candidate prediction box, calculating the overlap area between the candidate prediction box and the initial prediction box; calculating the overlap area ratio based on the overlap area and the area of the initial prediction box; determining the non-overlapping area ratio corresponding to the candidate prediction box based on the overlap area ratio; calculating the first product of the non-overlapping area ratios corresponding to each candidate prediction box; and determining the confidence update parameter corresponding to the initial prediction box based on the first product.
2. The method according to claim 1, characterized in that, The step of determining the confidence update parameters corresponding to the initial prediction box based on the first product includes: Based on the first product and the preset mapping function, the confidence update parameters corresponding to the initial prediction box are calculated.
3. The method according to claim 1, characterized in that, The step of updating the prediction confidence of the initial prediction box based on the confidence update parameters to obtain the target confidence corresponding to the initial prediction box includes: Calculate the second product of the confidence update parameter and the predicted confidence of the initial prediction box, and use the second product as the target confidence of the initial prediction box.
4. The method according to claim 1, characterized in that, The pre-trained crack detection network performs crack detection on the target image, obtaining multiple initial prediction boxes containing prediction confidence and location information, including: A pre-trained crack detection network is used to detect cracks in the target image, resulting in multiple first prediction boxes containing prediction confidence and location information. The first prediction box is filtered using a non-maximum suppression algorithm to obtain an initial prediction box.
5. The method according to claim 1, characterized in that, The training process of the crack detection network includes: Obtain a sample image containing the crack; Cracks are annotated in the sample images to obtain a sample dataset containing the bounding boxes; The initial target detection network is trained based on the sample dataset to obtain the crack detection network.
6. A crack detection device based on a prediction frame, characterized in that, The device includes: The first acquisition module is used to acquire the target image of the object to be detected; The detection module is used to perform crack detection on the target image using a pre-trained crack detection network to obtain multiple initial prediction boxes containing prediction confidence and location information. The update module is configured to, for each initial prediction box, determine candidate prediction boxes whose prediction confidence is greater than that of the initial prediction box, determine confidence update parameters corresponding to the initial prediction box based on the overlap area between the candidate prediction boxes and the initial prediction boxes, and update the prediction confidence of the initial prediction boxes based on the confidence update parameters to obtain the target confidence of the initial prediction boxes. The filtering module is used to filter the initial prediction boxes according to the target confidence level and the preset confidence threshold corresponding to each initial prediction box, so as to obtain the target prediction box containing the crack. The update module is specifically used for: for each candidate prediction box, calculating the overlap area between the candidate prediction box and the initial prediction box; calculating the overlap area ratio based on the overlap area and the area of the initial prediction box; determining the non-overlapping area ratio corresponding to the candidate prediction box based on the overlap area ratio; calculating the first product of the non-overlapping area ratios corresponding to each candidate prediction box; and determining the confidence update parameter corresponding to the initial prediction box based on the first product.
7. The apparatus according to claim 6, characterized in that, The update module is specifically used for: Based on the first product and the preset mapping function, the confidence update parameters corresponding to the initial prediction box are calculated.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.
Citation Information
Patent Citations
Improved non-maximum suppression YOLOv4 traffic sign identification and positioning method
CN114419584A